Monday, December 6, 2010

The American Community Survey: Development to 2004

Introduction

The American Community Survey (ACS) uses a continuous measurement methodology that employs a monthly sample of the U.S. population, an innovation that traces its roots back to the 1980s. The ACS is designed to provide the same kinds of information previously provided the decennial census long-form sample used for censuses from 1960 through 2000.

Why the ACS Was Developed

For decades, the long-form sample from the decennial census served as the only source of detailed social and demographic statistics for small areas and population groups in the United States. The long-form sample, however, had two major shortcomings.

First, since the information was collected as part of the census count, it was an added burden to the selected respondents and complicated the data collection effort by the Census Bureau. By 1990, the mail response rate for long-form recipients had dropped significantly below the rate for short-form recipients, increasing the costs of nonresponse follow-up (NRFU) and degrading the quality of the long-form data. In addition, because the census count is the Census Bureau’s top priority, the editing, processing, and tabulation of long-form data had to take a back seat to the full count enumeration, delaying the release of long-form data for two or three years after the actual enumeration.

Second, perhaps the biggest drawback of the census long-form sample is that the data were collected only once every ten years. The sample provided a snapshot of one point in time for the decade and quickly became out-of-date in many communities throughout the nation. Couple this with the delay in release of the long-form data, and it became increasingly apparent that an alternative to the census long form was needed.

ACS Development—Initial Concepts

As far back as 1941, Philip Hauser, then at the Census Bureau (and its deputy director in 1947), proposed an “annual sample census” as a way to obtain continuously updated information for the population. In 1976, Congress passed a law requiring a mid-decade census beginning in 1985, but funding was never appropriated, and it failed to become a reality.

The concept of a rolling sample census, which became the ACS, was first proposed by Leslie Kish in a 1981 paper prepared for the Subcommittee on Census and Population of the U.S. House of Representatives Committee on Post Office and Civil Service. Kish proposed that an independent sample of one-tenth of the population be surveyed every year, cumulating the estimates over two or more years for detailed characteristics and small geographic areas.

In the late 1980s, as part of the early Census 2000 planning work, three options were identified to simplify the basic census and provide subnational demographic and housing data in a timelier manner throughout the decade: (1) a mid-decade census; (2) a rolling sample of states taken over a ten-year period; and (3) a large scale monthly sample survey. In 1989, a plan, called the Decade Census Program (DCP), envisioned a basic short-form-only census with the detailed social, economic, and housing data collected by a sample rolling across states during the decade with the goal of obtaining at least two measurements of data in each state, every decade. This proposal noted that most uses of small-area data involve comparisons within states; therefore, the absence of detailed data for every state in the same year was not considered to be a critical issue.

In the early 1990s, in response to not only renewed congressional interest in more up-to-date small-area data, but also to congressional distaste for the more burdensome long form and decreased mail response rates, the Census Bureau developed a proposal for a continuous measurement alternative to the long-form sample for the next census in 2000. Beginning in 1992, Charles Alexander, a statistician at the Census Bureau, led an effort to examine design alternatives that would eliminate the census long form. This work would soon develop into a proposal for a Continuous Measurement Survey (CMS), which included three major components:

1. a continuously updated Master Address File (MAF) to support survey sample selection;

2. a continuous intercensal survey using monthly samples to produce multi-year moving averages for small geographic areas such as census tracts and single-year estimates for larger areas; and

3. an estimates program that would produce control totals for the continuous survey.

These three components offered several major advantages for the Census Bureau. First, the decennial census enumeration itself would be simplified. Second, having a sample every month would put more emphasis on developing methods to regularly update the MAF as a frame for sample selection, which would, in turn, benefit the decennial census. Third, a continuous survey would allow for a permanent and much better-trained staff to collect data for complicated topics such as disability. The advantages for users would be more frequent and timely information for small geographic areas and population groups.

It was clear from the beginning that the new CMS, which now became known as the American Community Survey (ACS), would require an extensive development period, resulting in some trade-offs involving timeliness, reliability, and costs. One trade-off presented itself very early. Designers of the sample found that five-year estimates from the survey with comparable reliability to the census long-form sample would require a monthly sample of approximately 500,000 addresses, which would be costly. Ultimately, budget realities resulted in a reduced monthly sample of about 250,000 addresses. This reduction would have little effect on the reliability of single-year estimates for larger population areas, but reliability would suffer for the five-year estimates for smaller populations, Some early thought was given to expanding the monthly sample from about 250,000 to 400,000 in selected years as part of a cycle, but in the end no such sample expansion was introduced.

Assuming that about one-half of the sample each month would respond by mail, to control costs the designers proposed that both telephone and personal visit follow-ups be done using subsampling. In 1993, however, it was recommended that phone interviews be conducted for all eligible cases and that only personal visit follow-up cases would be subsampled. The ACS would also need to parallel the census long-form sample design parameters, particularly the oversampling of small governmental units.

ACS Development—1995–2004

In 1994, the Census Bureau established a staff to test the new approach. Throughout 1996 a questionnaire similar to the long form was fielded in four counties. The decision was made in the mid-1990s to retain the long-form sample in the Census 2000 and implement the ACS on a gradual basis so that its operational feasibility could be determined and the results compared with the 2000 results. The test sites were expanded to 36 counties in 31 sites for the years 1999–2004. In addition, a nationwide Census 2000 Supplementary Survey (C2SS) was successfully fielded during the 12 months of 2000 in more than one-third of U.S. counties; together, the C2SS and the sites obtained information during 2000 for about 587,000 housing units.

Monthly data collected in the test sites were used to produce three-year period estimates from the ACS centered on 2000 to be compared to the Census 2000 results for counties and census tracts. The C2SS permitted comparisons at the national level. The Census Bureau conducted ACS and Census 2000 comparability research in 2002–2004 and issued six comparison reports in 2004. Although the 2004 Evaluation Report Series noted differences, these evaluations provided support for the proposed change from the census long-form sample to the ACS model.

Around the same time, the Census Bureau contracted with researchers who possessed local knowledge of the ACS test areas to closely examine ACS and census sample data for selected counties. A number of studies used data from administrative records and other corroborating evidence to explore differences detected in ACS and Census 2000 data, for example, regarding house heating fuel and the year housing structure was built. This work found that the ACS information more closely resembled the administrative records data. Much of this work concluded that meaningful differences between the ACS and Census 2000 estimates at the local level were largely explained in favor of the ACS and were attributed to the use of professional interviewers who were better able to obtain information from respondents. Research on the rural and seasonal characteristics of the areas covered in the analysis confirmed that the ACS estimates would differ from the Census 2000 figures based on the different definitions of residence (usual residence in the census versus actual residence for two months in the ACS). Finally, this research concluded that most statistically significant differences between the ACS and Census 2000 estimates would have limited implications for program development or targeting efforts by local officials.

In addition to the test sites, nationwide surveys similar to the C2SS were fielded in 2001–2004 and provided information for states, counties, and cities of 250,000 or more population. The ACS was fully implemented for housing units beginning in January 2005. Because of budget constraints, residents of group quarters (college dormitories, nursing homes, prisons, and other group living situations) were not added until a year later in January 2006. The ACS is now in continuous operation with questionnaires being mailed to about 240,000 households each month in every county of the 50 states, the District of Columbia, and every municipio of Puerto Rico, with samples of group quarters being visited by interviewers.

Throughout the ACS development process, the Census Bureau worked closely with data users, particularly federal agencies that use census data to administer programs and distribute funds, to address their concerns about the new approach of continuous measurement. This outreach program included: colloquia in 1995 and 1997 with federal statistical experts to explore the issues of the new design; a Rural Data Users Conference in 1998 to discuss specific concerns for small areas and populations outside cities; support for detailed studies at the Department of Housing and Urban Development (HUD) and the Department of Transportation (DOT) on issues related to their data applications; and, through the U.S. Office of Management and Budget (OMB), the creation of an interagency committee on the ACS to share information about plans with federal agencies. In addition, the Census Bureau solicited advice from Census Advisory Committees, informal town-hall-style meetings with data users, and a National Research Council Panel on the functionality and usability of ACS estimates.

Preston J. Waite

See also Long Form; Sampling for Content; Sampling in the Census