14.3.1 The main sources of statistics in India as elsewhere are:
Administrative Statistics – generally collected by State Governments; consisting of statutory administrative returns and data derived as a by-product of general administration; and
Other important sources namely, censuses and sample surveys.
14.3.2 Administrative Statistics are very much needed for effective planning of censuses and sample surveys. The state of the Indian Statistical System thus depends largely on the state of functioning of the Administrative Statistical System. In case of the system of direct data collection through sample surveys, the main failure had been in timely processing of data and release of results. But, with effective computerisation, the problem of delays in publication of results of sample surveys has, by and large, been resolved.
14.3.3 The major failure is in the Administrative Statistical System. This system came into being as administrative information system whose essential purpose was to aid the Government Departments in the execution of their functions of implementation of different Acts, Rules and Regulations of Governments. Even when such Acts were passed by the Central Government, their implementation was decentralised through the State Government Departments and their district or other sub-offices. The statistics thus had a direct purpose of being not only of interest to but also necessary for the working of the departments. The regularity, quality, and completeness in the collection of these statistics, interwoven with the working of these departments, were thus indirectly ensured. The quality of this system is thus directly related to the interest the administrative departments take in it and the effective use they make of it. It is however a fact that strictness in the administrative functions of several departments of most State Governments is waning, resulting in a virtual neglect of the information system. The main reason for the near collapse of the Administrative Statistical System is this near total failure of the administrative machinery of the Governments. This results in (a) incomplete coverage, (b) delays in availability of information, and (c) unsatisfactory quality of Administrative Statistics. The other reason is the lack of effective coordination between different statistical agencies, especially at the Centre. Though charged with the responsibility of coordination as a nodal agency, the CSO could not effectively carry out this function and influence the other Central Ministries and State Governments to remedy the ills of their Administrative Statistical Systems.
Centralised and Decentralised Systems of Collection of Administrative Statistics
14.3.4 Subjects such as money and finance, international trade, balance of payments, incorporated businesses, have meaning only at the all-India level. There are sectors, which straddle across more than one State and for which statistics are collected directly by Central agencies such as railways, postal services and telecommunications. Statistics on both types of subjects are collected by the Central Administrative Statistical System. All other Administrative Statistics are collected by the State Statistical System.
14.3.5 Administrative statistics provide information that is relevant to the working of the Departments. They serve the major purpose of aiding the Departments in the execution of their administrative functions of implementation and execution of different Acts, Rules and Regulations with which the Departments are charged. Consequently, the concerned Departments have a vital interest in the proper collection of the administrative statistics.
14.3.6 There are other advantages too in the system of collection of statistics through the administrative set up. The collection of data by departmental agencies does not involve special costs. The collection is oriented to definite purposes, and the record and verification of information is part of administration. Departmental agencies and officials have not only good knowledge of the subject, but also of local language and local conditions, especially rural. Information collected is relevant and direct, and the respondents do not have to make calculations before answering a query. It is handled by agencies that have special knowledge of the subject Finally, there is an identifiable purpose in their data collection and they are in the best position to interpret the data. All this has lent a solid foundation to the decentralised administrative statistical system, and in turn, to the Indian Statistical System. An impression is carried by many that data collected by substantive Government departments are likely to suffer from bias. Therefore, they suggest that an independent agency should collect data to ensure objectivity. But, ignorance should not pass off as objectivity, making the solution worse than the problem. While the impression might be true for certain departments at certain times, it is easy to overstress the point as a justification for the solution suggested.
14.3.7 As against these advantages of the decentralised Administrative Statistical System, it will be apposite to point out the disadvantages of a centralised system of sample surveys. First, it is simply an unmanageable system for a large country such as India, except when it is a sample survey system of reasonable size. Second, on the very same counts on which a decentralised system possesses advantages, the centralised system fares poorly. Third, the very process of centralisation has a natural tendency to gather momentum.
14.3.8 A variant of this system is the one where a central agency collects data directly from the district offices of the State Government departments. When the State departments have to process the data and produce results, before they transmit them to the Centre, they are per force required to pay attention to timeliness in collection of data and their quality, and take corrective actions. But when the reporting units are to send the data directly to the Centre, which is a far distant agency, timeliness and quality of data will be affected adversely. For the States will have no responsibility for either. Worse, the State-level results, in this scheme of things, will become disaggregations of the National totals, and the States will be dependent on the Centre for State-level statistics, which are in fact in their domain.
14.3.9 Centralisation will thus inevitably raise questions about the Centre-State relationship in the field of statistics, about their authority for collection and use of data, and the authenticity and acceptability of such data.
Failure of Administrative Statistical System
14.3.10 But over the years, the Administrative Statistical System has been deteriorating and has now almost collapsed in certain sectors. The deterioration had taken place at its very roots namely, at the very first stage of collection and recording of data, and has been reported so far in four sectors: agriculture, labour, industry and commerce. The foundation on which the entire edifice of Administrative Statistical System was built appears to be crumbling, pulling down the whole system and paralysing a large part of the Indian Statistical System. This indisputably is the major problem facing the Indian Statistical System today.
Weak Lateral Coordination
14.3.11 A similar criticism is that of the weakness of lateral coordination, which has come to be viewed as another major problem. First an explanation of its nature is required because the very perception of this as a problem depends upon the view one takes of the Indian Statistical System. It also needs an analysis mainly because the solution to this is implicitly considered also as a solution to the first major problem. A brief history of the failure of the lateral coordination is presented below.
14.3.12 As stated above, the creation of the CSO brought in the question of lateral coordination in statistics between this and other ministries. The CSO carried out its function of coordination mainly by means of the technical committees or working groups, either appointed by it, generally under the Chairmanship of the CSO Director, or by the Ministries in which case the CSO was generally represented on them. The other mode was the bi-annual Conference of the Central and State Statistical Organisations (COCSSO) organised by the CSO. The COCSSO provided a forum for exchange of views and experiences concerning development of statistical activities in the country.
14.3.13 The success of the CSO in its role as a co-ordinator depended, on the one hand, upon the degree of its initiative and ability to persuade, and on the other hand, on the co-operation of the ministries, and their willingness to participate in this process as a team and to be persuaded to accept the conclusions of the team about their statistical work. However, given the historical background, the statisticians from the ministry had a less flexible mode of thinking, being generally averse to change, and “outside” influence. The unsatisfactory experience in coordinating with the ministries led the CSO to search for an institution outside of itself and the ministries from which it could derive authority. The idea was concretised by the Committee to Review the National Statistical System (1980), referred to as Kripa Narain Committee hereafter, in its two recommendations. The first required the Government of India to formally declare by Executive Order that the Department of Statistics, to which the CSO belonged, was the “Nodal” department (for statistics) for undertaking integration of data required for Government’s decision-making, for setting and maintaining standards, and for improvement and development of statistics in all respect. The second was to create a National Advisory Board on Statistics (NABS) with the Deputy Chairman or Member in charge for Statistics of the Planning Commission as the Chairman and the Director General of CSO as the Vice-Chairman. The Government had constituted NABS in 1982 with Member in charge of Statistics and Surveys Division, Planning Commission as Chairman. Later on from 1992, the Minister in charge of Statistics was appointed as the Chairman.
14.3.14 The NABS covered Governmental statistical programmes and systems both at the Centre and in the States and was meant for providing technical guidance for policy issues (regarding statistics), for ensuring effective and better coordination in all matters concerned with statistics. However, the NABS was not effective, primarily because of lack of official, legal or constitutional support.
14.3.15 The cumulative result of different actions by the Department of Statistics was a considerable weakening of the CSO. The available institutional arrangements for coordination in the shape of Technical Working Groups on various subjects or Committees were either not continued or were terminated by the Department of Statistics. The Conference of Central and State Statistical Organisations was not convened for many years till 2000 after the NSC started working. And for long periods the post of the Director General of the CSO was not filled up. A vacuum at the top was thus created. The separate identities of the Department of Statistics, a purely administrative office, and of the CSO, a professional institution, was thus erased. So far, the post of DG, CSO is still vacant, and during the work of the Commission, the views of the DG, CSO have not been represented before the Commission.
Modernisation Project.
14.3.16 In the late nineties, the Department of Statistics was designated as the nodal agency for Real Sector data categories of International Monetary Fund’s Special Data Dissemination Standards (SDDS). Faced with this responsibility, and the perceived weakness of the statistical system, the DOS came up with a proposal for creating a Statistical Authority to have centralised control over all official statistical agencies. It also put up a project for modernisation of the Indian Statistical System, to be financed through a loan from the World Bank. The main thrust of the project is on three measures: (a) Conduct of additional surveys to cater to the need of SDDS requiring expansion of NSSO to replace the failing Administrative Statistical System, (b) Expanded use of Information Technology, and (c) Creating a Central Training Institute for all levels of statistical personnel. The project has not considered adequately the problem of strengthening of the statistical system in the States.
14.3.17 The first measure, the massive expansion of National Sample Surveys, as a quick means for data collection for GDP estimation needs a re-examination. Also, the employment of these surveys as an alternative system to the failing Administrative Statistical System will divert attention from the solution of the real systemic problem of the decentralised Indian Statistical System.
14.3.18 Without any real effort to improve collection of data or their quality, the second measure of computerisation and application of Information Technology is likely to result in quick processing of data deficient in quality, coverage, timeliness, accuracy or precision. The real apprehension is that the desirable flow of information via the route: Reporting unit Þ State Statistical System Þ Indian Statistical System will be replaced for the sake of management efficiency by a “fly-over” approach: Reporting unit Þ Indian Statistical System.
14.3.19 That there is a great need for training of statistical personnel goes without saying. However, one must distinguish here between two different types of training. The first type of training is at the operational level - on standard or routinised methods of data collection, processing and summarisation. This kind of training is best given at the work site. A large majority of statistical operatives in India are university graduates, and short in-service type of training is all that is needed. It is the second type of training – training on statistical methods to improve the practice of statistics, is much more important and necessary, particularly in the context of almost total lack of use of so called “applicable theoretical techniques” in official statistical work in India today. Centralised training is essential here, but this has to wait till a body of competent trainers is available. The immediate necessity is for training of trainers.
Recommendations relating to the Modernisation Project
14.3.20 It is understood that the project is under review by the MoS&PI. While reviewing and reformulating it, the MoS&PI should consider the recommendations made by this Commission on the various subjects and the components of the project may be modified accordingly, if necessary. For this, the project will have to shift its focus from expansion of sample surveys to improvement of the systemic issues of the Administrative Statistical System. Modernisation may be considered as a means for that purpose. It should also keep in view the essentially decentralised character of the Indian Statistical System and ensure that the States’ Statistical Systems are interwoven in the project architecture.
Bringing all technical-cum-supervisory positions under ISS
14.3.21 Statistical units of Central Ministries and Departments can be grouped into three main types:
Those headed by ISS officers;
Those headed by officers from other organised services;
Those headed by officers not belonging to any organised service.
14.3.22 With an organised Indian Statistical Service (ISS) in place, the continued existence of ministries and departments of types (b) and (c) above is a serious anomaly. It appears to be a historical legacy of the birth of the ISS from voluntary offering of posts by different ministries and departments. To bring about uniformity in management and strengthen coordination, the Commission recommends that all technical-cum-supervisory statistical positions in the Central Government should be brought under the umbrella of the ISS. However, this does not preclude Government from appointing in a few senior positions, professional statisticians with proven capability on an appropriate arrangement.
Filling up of Vacancies
14.3.23 The Commission notes that a number of positions at the highest and middle levels of the hierarchy in the ISS have remained vacant for long periods. For the sake of proper functioning of the statistical system, these positions must be filled up immediately.
Implementation Of Above Recommendations
14.3.24 The recommendations made so far apply to problems currently faced by different statistical units of the MoS&PI under its present structure. But they do not address systemic issues. The Commission is of the view that for correcting the systemic problems of the Indian Statistical System, it is necessary to revamp and restructure the statistical system, starting from the top, by laying down a firm and lasting foundation for the Indian Statistical System. This is taken up later in this report (see paragraphs 14.5.1 to 14.5.27), where the constitution of National Commission on Statistics (NCS) is recommended. However, implementation of the above recommendations should proceed immediately and independently of the creation of the NCS. The existing institutions of National Advisory Board on Statistics (NABS), Governing Council of NSSO, Advisory committee on National Accounts Statistics and other technical Committees and institutional arrangements should also continue till the NCS is created.
Systemic Deficiencies
14.3.25 The Commission has noted that in recent years certain deficiencies of the statistical system of India have attracted serious media attention. These are:
Existence of gaps in availability of needed information;
Delays in publication of results;
Large and frequent revisions of published results;
Gross discrepancies between official statistics from different sources;
Occasional disagreement between tabulated summary results and publicly available basic data from which the summary has been produced; and
Lack of transparency in statistical operations.
14.3.26 These deficiencies have led to a serious loss of credibility of official statistics. These have been largely due to the following reasons:
There is no policy-making and coordination body with legal authority, independent of the producers of statistics and free from covert or overt political and bureaucratic pressures that can serve as a link between the producers and users of statistics.
There is a lack of a system of assurance of the quality of the statistics that are disseminated by the system.
Over the years, the system of statutory administrative returns, which are the major sources of official statistics, have been seriously weakened.
Time tested means of coordination with various agencies of the decentralised statistical system – technical working groups, advisory committees, conference of State and Central statistical officers, etc. have withered away. The biennial Conference of Central and State statistical organisations was not held from the year 1992 till 2000.
Effectiveness of the Central Statistical Organisation, the nodal agency for coordination and standards, has been seriously affected by not filling up the position of Director General, CSO since 1997.
There is a lack of appropriate legislation to provide a legal basis for collection of statistics, and to penalise official failures.
Absence of a human resource development policy has been responsible for a marked lack of motivation of official statisticians.
There is no spirit of innovation and research, to the extent that sample survey methodology developed in the fifties are still considered the only method to use whenever there is a demand for more information. Failure to adopt methods to “borrow strength from auxiliary information” has made dependence on larger and larger sizes of sample the only method available to cope with ever increasing demands for data. Time Series analysis, Use of statistical models, Classificatory Techniques, Hot deck methods of imputation, and other “applicable theoretical methods” have never been made use of.
Use of Information Technology has been restricted mainly to processing survey data. It has not been used in any significant way too improve the efficiency of the statistical system at large.
Absence of an explicit Citizen’s Charter or Mission Statement without which the expectations of the citizens from the statistical system cannot be formalised.
14.3.27 Before suggesting institutional changes necessary to solve the problems identified above, the Commission reviewed the international guidelines and the creation of apex institutions and supporting systemic arrangements by some statistically developed countries. A brief summary of this review is given below.
