Page images

graphic detail in statistics, as well as more frequent releases. The additional cost of computing the indirect statistics would be minimal in comparison to the cost of collecting statistical data in a census or survey.

review would not obviate the need for each agency to evaluate the quality of its new data series. Publications would include hard copy, as well as documentation of computer tapes giving public-use samples. OFSPS would prepare a set of guidelines for the presentation of data, as well as their limitations, in new statistical series which should be followed in preparing reports.

The proposed publication would be reviewed to ensure that users are informed of the limitations of the data. Data not adequate for general publication could be obtained on request by specific interested users in order to preclude any appearance of political or administrative suppression; appropriate caveats pointing out the shortcomings of the data would be included whenever data of inadequate quality are released; in addition, special agreements would be required to preclude further release of the data to other users or publications without specific authorization from the releasing agency.

Monitoring the quality of the data released would serve to improve statistics produced by Federal agencies. Clearly this task would require the assignment of specific staff resources to be done properly.

Statistical Adjustment of Data to Compensate for
Known Biases

The problem.-Data are often subject to biases of which the producer may have some information as to size and direction. For example, the decennial census count of population is subject to a net underenumeration. Another example involves the income reported by individuals in surveys; reported income from a specified source is often less than actual income from that source.

Policy recommendations. Procedures should be established to improve statistical estimates by improving survey methodology or adjusting for known biases. For example, some of the characteristics of the undercounted population in the decennial census can be determined via supplemental surveys or in matching studies. If the geographic distribution of the undercount can be approximated, then the basic decennial population estimates could be improved by carrying out a proper adjustment procedure. The adjustment of the various statistical series would require differing inputs. All statistical series should be examined to determine the availability of measures which might be used to improve the estimates and to determine the type of adjustment which would be most effective.

Statistics Obtained From Indirect Methodologies

The problem. In addition to statistics obtained directly from censuses, surveys and administrative records, Federal statistical agencies can derive indirect statistical estimates by combining data from various sources. These indirect methodologies include synthetic and other regression estimates as well as weighted indexes.

The use of these indirect methodologies would minimize the burden on respondents, if the indirect estimates are based on existing statistical data. Although at present many agencies are using indirect methodologies, the procedures used are not always properly documented. In addition, such methodologies being used have not always been subjected to vigorous statistical standards.

“Nested Survey” Methodology

Policy recommendations.-Indirect statistical measures should be investigated by OFSPS in conjunction with the Federal statistical agencies to document the use of such procedures and to determine whether their accuracy is sufficient for the purposes needed. The basic data input needed to derive an indirect estimate, as well as the availability and quality of the data, must be reviewed. In addition, further uses of indirect estimators should be investigated. These indirect measures may provide a cost-effective solution to the need for greater geo

The problem.—No single survey questionnaire can cover all topics of interest to the user of statistics. The length and complexity of survey questionnaires must be limited by considerations of respondent burden and level of comprehension as well as cost considerations. These limitations give rise to serious problems in accommodating in any questionnaire the detailed items needed to cover a variety of social, economic, and demographic subjects relating to the same universe. One way to overcome these limitations, at least in part, is to employ a "nested survey" methodology whereby different subsamples of the same universe are asked (partially) different questions; all samples may be asked certain core questions; one sample may cover all aspects of the survey. The idea is illustrated below.

evaluation in the proposed budget for the pro Questionnaire

gram...” Recommendations 5-16 through 5-23 relate

to small area statistics. These recommendations inSub

clude “...developing procedures and techniques for sample

1 2 3 4 5 estimating intercensal changes.... 1 X X X X X

The Joint Ad Hoc Committee on Government 2 X X

Statistics prepared a report on the Federal Statistical 3


System. This report included recommendations on 4 X

topics relevant to standards for statistical X

methodology. On Federal contracts for statistical 5 x


services: 6

x x


The Committee recommends a thoroughgoing In principle, this technique would permit coverage

review of the procedures for awarding Federal

contracts for statistical services to nonof an extensive range of topics without imposing an excessive burden on any single respondent unit. This

governmental organizations. The review should in

clude after-the-fact audits of the quality of the data collection technique has been used, for example,

services provided, performance in relation to time in the 1970 Census of Population and Housing where

and costs, and a look into comparative quality and some questions were asked of 5%, 15% and 20% of the

costs of doing the work in-house. population.

On the need for analysis: Furthermore, the data obtained from the several surveys could be consolidated at the processing stage The Committee recommends that the major by means of statistical matching to yield a more statistical agencies reassess their allocation of complete set of statistical measures. This would resources between collection and processing of permit more extensive comparative analyses to be data and analysis, with a view to providing more carried out, using data from each survey in different adequate analyses of the data that are collected. combinations.

Throughout the Federal statistical system there is

also a need for more resources devoted to the Policy recommendations. The "nested surveyap

development and application of statistical proach might provide a new methodology which could be employed in connection with the mid-decade census.

methodology in the collection, analysis, and

presentation of statistical data. It would be necessary to investigate the feasibility of such an approach by means of a pretest to determine its Both of these recommendations have far-reaching implications for control of the field operation and for implications for the methodological aspects of the the consolidation of data based on statistical matching Federal Statistical System. of the various survey instruments for analytic purposes. An evaluation of the resulting data file based on

A panel of the Committee on National Statistics of statistical matching of sample cases covering different

the National Academy of Sciences under the subject matter areas would be needed.

sponsorship of the Department of Health, Education,

and Welfare prepared a report on Setting Statistical Additional Topics Suggested by Various

Priorities. This report emphasizes the difficulty of

establishing priorities in a rational manner among the Other Committees

statistical activities of a government agency. One of The report of the President's Commission on its recommendations seems particularly relevant to Federal Statistics' suggested many additional areas

the task of developing standards for statistical where further work on statistical methodology

methodology. Recommendation 2 states: should be done. For example, recommendation 5-1

Each statistical agency should have budgeted suggested that SPD/OMB (now the Office of Federal

activities to support analyses (a) to be used in Statistical Policy and Standards) "should be

setting the agency's priorities and (b) to improve expanded to allow an audit of the statistical activities

the collection, analysis, and dissemination of data. of all agencies at appropriate intervals...." Recommendation 5-10 recommends "...that proposed new

2“Report of the Joint Ad Hoc Committee on Government programs have monies specifically set aside for

Statistics,” August 20, 1976, Statistical Reporter, No. 76-15

(September 1976), pp. 301-310. 'Federal Statistics, Report of the President's Commission, Vol. 'Committee on National Statistics, National Academy of umes I and II, Washington, D.C., 1971.

Sciences, Setting Statistical Priorities, Washington, D.C.,1976.

[ocr errors][merged small]

also need to be developed. For the adequate design of new major statistical surveys statistical expertise should be available to offices in need of consultation. Projects of major importance should be submitted for clearance to the OMB and OFSPS at an early stage of development; this would lead to better methodologies, adherence to standards, and expedited final

clearance of the project. 4. The feasibility of a centralized clearance of

new data series to be published should be studied. Such a clearance system would be aimed at developing a more uniform quality of

Federal statistical releases. 5. Investigation of the present use and further

utility of indirect estimation methods should be carried out. These procedures may help reduce the response burden of collecting statistical data. In addition, estimates for small areas and at more frequent intervals may be

obtained using these procedures. 6. Statistical adjustment of data series to

compensate for known biases should be made whenever feasible. It is necessary to evaluate alternative methodologies which could be used

for statistical adjustments. 7. The nested surveyapproach to analyze data

files based on statistical matching should be tested in a real situation to determine the feasibility of using this procedure, the difficulties

involved, and the quality of the resulting data. In conclusion, the different Federal agencies sponsoring and producing statistical data have programs which address the various methodological issues discussed in this chapter. Our recommendations suggest further investigation of these issues in terms of broader use of efficient statistical methodology and of improved monitoring of the quality of statistical data.

Summary of Recommendations 1. The agencies should begin to assess the

implications of the recommendations given in Statistical Policy Working Paper 1, Report on Statistics for Allocation of Funds regarding the use of alternative data and formulas for the distribution of Federal funds to State and local

areas. 2. The agencies should begin to assess the

implications of the recommendations given in Statistical Policy Working Paper 2, Report on Statistical Disclosure and Disclosure-Avoidance Techniques. These recommendations cover topics such as the concept of statistical disclosure, how to decide what data to release, disclosure-avoidance techniques in tabulations, the release of microdata files, effects of disclosure on data subjects and users,

and needs for research. 3. The development of improved standards for

statistical surveys are needed to improve the quality of Federal statistical surveys. In addition, the standards must be adequately monitored for maximum effectiveness. Standards for contracting for statistical surveys

[ocr errors]


A Developmental Strategy


and emerging trends in its evolution. For this purpose, it is necessary to select statistics related to the well-being of different population groups—their health, housing, education, modes of livelihood, income, protection against economic and other ards, access to goods and services, opportunity for advancement, and the like. The usefulness of such data is enhanced when they are available in timeseries form, so that trends may be revealed.

The objective of this chapter is to present a strategy for social indicator development in terms of two closely related activities: (1) the preparation of social indicator reports, including periodical publications containing such indicators, and (2) the construction of a social accounting framework which would facilitate the derivation of improved indicators of social processes and changes.

Recent efforts to prepare social indicator reports and related publications are reviewed. Prospective efforts in the area of social indicator research and development, focusing on improvements in the data base and the incorporation of these data into some kind of social accounting framework, are also considered. The concluding part provides a summary of some issues and options which emerge from the preceding considerations.

Social indicators may be described as social statistics (direct counts or derived measures) which are considered to reflect some important aspect of social conditions and trends. They constitute a subset of the available body of demographic, economic, and social statistics which may be obtained from a variety of public and private sources. The criteria whereby social indicators may be selected depend upon the purposes they are intended to serve. In general terms, three broad types of indicators may be distinguished on this basis: descriptive indicators, analytical indicators, and programmatic indicators.'

Descriptive indicators are needed to depict, in statistical terms, the current condition of the society

'Compare Eleanor Bernert Sheldon, "Social Reporting for the 1970's," in the President's Commission on Federal Statistics, Federal Statistics, Vol. II, Chapter 7 (Washington, D.C.: U.S. Governmen Printing Office, 1971). Also pertinent with respect to the general nature, purposes, and limitations of social indicators are articles by Eleanor Bernert Sheldon and Howard E. Freeman, “Notes on Social Indicators: Promises and Potential,” Policy Sciences, I (Spring, 1970), by Eleanor Bernert Sheldon and Robert Parke, “Social Indicators,” Science, 188 (May 16, 1975), and by Robert Parke and David Seidman, “Social Indicators and Social Reporting,” The Annals, Volume 435 (January 1978), pp. 1-22.

Analytic indicators are intended to provide a deeper level of understanding than can be acquired by examining descriptive statistics alone. Analysis aims to explain “why” or “how" (in terms of related variables) a given condition has come about and thereby offers some clues as to what might be done to alter, improve, or adjust to that condition. The analytic process calls for the identification and measurement of those "independent" variables which significantly influence the condition or variable of interest. The distinction between descriptive and analytic indicators is a matter of degree. When descriptive indicators are presented either in timeseries form or disaggregated according to certain relevant characteristics, they yield useful insights in their own right by revealing differences which would otherwise be obscured. Futhermore, statistical analysis can reveal the interrelations among selected variables and thereby suggest possible causal influences, but these can seldom be demonstrated with finality.

Programmatic indicators, finally, are statistical observations and measurements which are designed to aid in monitoring and evaluating specific policies and related programs. Such indicators frequently reflect program costs and impacts so that program effectiveness can be assessed. In short, when a particular set of statistics or statistical measures is selected as a social indicator, the underlying rationale may be that the data are thought to depict an important trend or characteristic, or that they provide some explanatory insight as to why some development of interest has occurred or failed to occur, or that they reflect the effectiveness of a particular policy or program.

« PreviousContinue »