review would not obviate the need for each agency to evaluate the quality of its new data series. Publications would include hard copy, as well as documentation of computer tapes giving public-use samples. OFSPS would prepare a set of guidelines for the presentation of data, as well as their limitations, in new statistical series which should be followed in preparing reports.

The proposed publication would be reviewed to ensure that users are informed of the limitations of the data. Data not adequate for general publication could be obtained on request by specific interested users in order to preclude any appearance of political or administrative suppression; appropriate caveats pointing out the shortcomings of the data would be included whenever data of inadequate quality are released; in addition, special agreements would be required to preclude further release of the data to other users or publications without specific authorization from the releasing agency.

Monitoring the quality of the data released would serve to improve statistics produced by Federal agencies. Clearly this task would require the assignment of specific staff resources to be done properly.

Statistics Obtained From Indirect Methodologies

The problem.-In addition to statistics obtained directly from censuses, surveys and administrative records, Federal statistical agencies can derive indirect statistical estimates by combining data from various sources. These indirect methodologies include synthetic and other regression estimates as well as weighted indexes.

The use of these indirect methodologies would minimize the burden on respondents, if the indirect estimates are based on existing statistical data. Although at present many agencies are using indirect methodologies, the procedures used are not always properly documented. In addition, such methodologies being used have not always been subjected to vigorous statistical standards.

Policy recommendations.-Indirect statistical measures should be investigated by OFSPS in conjunction with the Federal statistical agencies to document the use of such procedures and to determine whether their accuracy is sufficient for the purposes needed. The basic data input needed to derive an indirect estimate, as well as the availability and quality of the data, must be reviewed. In addition, further uses of indirect estimators should be investigated. These indirect measures may provide a cost-effective solution to the need for greater geo

graphic detail in statistics, as well as more frequent releases. The additional cost of computing the indirect statistics would be minimal in comparison to the cost of collecting statistical data in a census or survey.

Statistical Adjustment of Data to Compensate for
Known Biases

The problem.-Data are often subject to biases of which the producer may have some information as to size and direction. For example, the decennial census count of population is subject to a net underenumeration. Another example involves the income reported by individuals in surveys; reported income from a specified source is often less than actual income from that source.

Policy recommendations.-Procedures should be established to improve statistical estimates by improving survey methodology or adjusting for known biases. For example, some of the characteristics of the undercounted population in the decennial census can be determined via supplemental surveys or in matching studies. If the geographic distribution of the undercount can be approximated, then the basic decennial population estimates could be improved by carrying out a proper adjustment procedure. The adjustment of the various statistical series would require differing inputs. All statistical series should be examined to determine the availability of measures which might be used to improve the estimates and to determine the type of adjustment which would be most effective.

"Nested Survey" Methodology

The problem.-No single survey questionnaire can cover all topics of interest to the user of statistics. The length and complexity of survey questionnaires must be limited by considerations of respondent burden and level of comprehension as well as cost considerations. These limitations give rise to serious problems in accommodating in any questionnaire the detailed items needed to cover a variety of social, economic, and demographic subjects relating to the same universe. One way to overcome these limitations, at least in part, is to employ a "nested survey" methodology whereby different subsamples of the same universe are asked (partially) different questions; all samples may be asked certain core questions; one sample may cover all aspects of the survey. The idea is illustrated below.

In principle, this technique would permit coverage of an extensive range of topics without imposing an excessive burden on any single respondent unit. This data collection technique has been used, for example, in the 1970 Census of Population and Housing where some questions were asked of 5%, 15% and 20% of the population.

Furthermore, the data obtained from the several surveys could be consolidated at the processing stage by means of statistical matching to yield a more complete set of statistical measures. This would permit more extensive comparative analyses to be carried out, using data from each survey in different combinations.

Policy recommendations.-The "nested survey" approach might provide a new methodology which could be employed in connection with the mid-decade census. It would be necessary to investigate the feasibility of such an approach by means of a pretest to determine its implications for control of the field operation and for the consolidation of data based on statistical matching of the various survey instruments for analytic purposes. An evaluation of the resulting data file based on statistical matching of sample cases covering different subject matter areas would be needed.

Additional Topics Suggested by Various Other Committees

The report of the President's Commission on Federal Statistics' suggested many additional areas where further work on statistical methodology should be done. For example, recommendation 5-1 suggested that SPD/OMB (now the Office of Federal Statistical Policy and Standards) "should be expanded to allow an audit of the statistical activities of all agencies at appropriate intervals...." Recommendation 5-10 recommends "...that proposed new programs have monies specifically set aside for

'Federal Statistics, Report of the President's Commission, Volumes I and II, Washington, D.C., 1971.

evaluation in the proposed budget for the pro gram..." Recommendations 5-16 through 5-23 relate to small area statistics. These recommendations include "...developing procedures and techniques for estimating intercensal changes...."

The Joint Ad Hoc Committee on Government Statistics' prepared a report on the Federal Statistical System. This report included recommendations on topics relevant to standards for statistical methodology. On Federal contracts for statistical services:

The Committee recommends a thoroughgoing review of the procedures for awarding Federal contracts for statistical services to nongovernmental organizations. The review should include after-the-fact audits of the quality of the services provided, performance in relation to time and costs, and a look into comparative quality and costs of doing the work in-house.

On the need for analysis:

The Committee recommends that the major statistical agencies reassess their allocation of resources between collection and processing of data and analysis, with a view to providing more adequate analyses of the data that are collected. Throughout the Federal statistical system there is also a need for more resources devoted to the development and application of statistical methodology in the collection, analysis, and presentation of statistical data.

Both of these recommendations have far-reaching implications for the methodological aspects of the Federal Statistical System.

A panel of the Committee on National Statistics of the National Academy of Sciences under the sponsorship of the Department of Health, Education, and Welfare prepared a report on Setting Statistical Priorities.' This report emphasizes the difficulty of establishing priorities in a rational manner among the statistical activities of a government agency. One of its recommendations seems particularly relevant to the task of developing standards for statistical methodology. Recommendation 2 states:

Each statistical agency should have budgeted activities to support analyses (a) to be used in setting the agency's priorities and (b) to improve the collection, analysis, and dissemination of data.

2"Report of the Joint Ad Hoc Committee on Government Statistics," August 20, 1976, Statistical Reporter, No. 76-15 (September 1976), pp. 301-310.

'Committee on National Statistics, National Academy of Sciences, Setting Statistical Priorities, Washington, D.C., 1976.

This recommendation would tend to help produce a good statistical product.

A report of the Commission on Federal Paperwork on Statistics published in 1977 states in Recommendation 3 that:

Statistical service units should be established to provide technical assistance to the Congress and to Federal agencies in the planning and design of statistical studies and in the development of guidelines for awarding contracts and grants for statistical studies.

Summary of Recommendations

1. The agencies should begin to assess the implications of the recommendations given in Statistical Policy Working Paper 1, Report on Statistics for Allocation of Funds regarding the use of alternative data and formulas for the distribution of Federal funds to State and local

areas.

2. The agencies should begin to assess the implications of the recommendations given in Statistical Policy Working Paper 2, Report on Statistical Disclosure and Disclosure-Avoidance Techniques. These recommendations cover topics such as the concept of statistical disclosure, how to decide what data to release, disclosure-avoidance techniques in tabulations, the release of microdata files, effects of disclosure on data subjects and users, and needs for research.

3. The development of improved standards for statistical surveys are needed to improve the quality of Federal statistical surveys. In addition, the standards must be adequately monitored for maximum effectiveness. Standards for contracting for statistical surveys

also need to be developed. For the adequate design of new major statistical surveys statistical expertise should be available to offices in need of consultation. Projects of major importance should be submitted for clearance to the OMB and OFSPS at an early stage of development; this would lead to better methodologies, adherence to standards, and expedited final clearance of the project.

4. The feasibility of a centralized clearance of new data series to be published should be studied. Such a clearance system would be aimed at developing a more uniform quality of Federal statistical releases.

5. Investigation of the present use and further utility of indirect estimation methods should be carried out. These procedures may help reduce the response burden of collecting statistical data. In addition, estimates for small areas and at more frequent intervals may be obtained using these procedures.

6. Statistical adjustment of data series to compensate for known biases should be made whenever feasible. It is necessary to evaluate alternative methodologies which could be used for statistical adjustments.

7. The "nested survey" approach to analyze data files based on statistical matching should be tested in a real situation to determine the feasibility of using this procedure, the difficulties involved, and the quality of the resulting data.

In conclusion, the different Federal agencies sponsoring and producing statistical data have programs which address the various methodological issues discussed in this chapter. Our recommendations suggest further investigation of these issues in terms of broader use of efficient statistical methodology and of improved monitoring of the quality of statistical data.

A Developmental Strategy

Introduction

The objective of this chapter is to present a strategy for social indicator development in terms of two closely related activities: (1) the preparation of social indicator reports, including periodical publications containing such indicators, and (2) the construction of a social accounting framework which would facilitate the derivation of improved indicators of social processes and changes.

Recent efforts to prepare social indicator reports and related publications are reviewed. Prospective efforts in the area of social indicator research and development, focusing on improvements in the data base and the incorporation of these data into some kind of social accounting framework, are also considered. The concluding part provides a summary of some issues and options which emerge from the preceding considerations.

Social indicators may be described as social statistics (direct counts or derived measures) which are considered to reflect some important aspect of social conditions and trends. They constitute a subset of the available body of demographic, economic, and social statistics which may be obtained from a variety of public and private sources. The criteria whereby social indicators may be selected depend upon the purposes they are intended to serve. In general terms, three broad types of indicators may be distinguished on this basis: descriptive indicators, analytical indicators, and programmatic indicators.'

Descriptive indicators are needed to depict, in statistical terms, the current condition of the society

'Compare Eleanor Bernert Sheldon, "Social Reporting for the 1970's," in the President's Commission on Federal Statistics, Federal Statistics, Vol. II, Chapter 7 (Washington, D.C.: U.S. Governmen Printing Office, 1971). Also pertinent with respect to the general nature, purposes, and limitations of social indicators are articles by Eleanor Bernert Sheldon and Howard E. Freeman, "Notes on Social Indicators: Promises and Potential," Policy Sciences, I (Spring, 1970), by Eleanor Bernert Sheldon and Robert Parke, "Social Indicators," Science, 188 (May 16, 1975), and by Robert Parke and David Seidman, "Social Indicators and Social Reporting," The Annals, Volume 435 (January 1978), pp. 1-22.

and emerging trends in its evolution. For this purpose, it is necessary to select statistics related to the well-being of different population groups—their health, housing, education, modes of livelihood, income, protection against economic and other hazards, access to goods and services, opportunity for advancement, and the like. The usefulness of such data is enhanced when they are available in timeseries form, so that trends may be revealed.

Analytic indicators are intended to provide a deeper level of understanding than can be acquired by examining descriptive statistics alone. Analysis aims to explain "why" or "how" (in terms of related variables) a given condition has come about and thereby offers some clues as to what might be done to alter, improve, or adjust to that condition. The analytic process calls for the identification and measurement of those "independent" variables which significantly influence the condition or variable of interest. The distinction between descriptive and analytic indicators is a matter of degree. When descriptive indicators are presented either in timeseries form or disaggregated according to certain relevant characteristics, they yield useful insights in their own right by revealing differences which would otherwise be obscured. Futhermore, statistical analysis can reveal the interrelations among selected variables and thereby suggest possible causal influences, but these can seldom be demonstrated with finality.

Programmatic indicators, finally, are statistical observations and measurements which are designed to aid in monitoring and evaluating specific policies and related programs. Such indicators frequently reflect program costs and impacts so that program effectiveness can be assessed. In short, when a particular set of statistics or statistical measures is selected as a social indicator, the underlying rationale may be that the data are thought to depict an important trend or characteristic, or that they provide some explanatory insight as to why some development of interest has occurred or failed to occur, or that they reflect the effectiveness of a particular policy or program.