Saturday, December 11, 2010

American Community Survey: Implementation from 2005

The year 2005 is significant in the history of the American Community Survey (ACS) because it marks the end of more than a decade of testing and the beginning of what is generally regarded as full nationwide implementation. The 2005 implementation expanded earlier test samples to include all counties in the 50 states and the District of Columbia, as well as all municipios in Puerto Rico, for a total initial sample of about 2.9 million housing units. The 2005 sample did have one important limitation regarding coverage of the population—budget constraints restricted the sample to persons in housing units only, thus excluding persons in group quarters (GQs). Starting in 2006, the sample was expanded to include residents in most types of GQs, which include facilities such as nursing homes, correctional institutions, military barracks, group homes, and college dormitories.

Beginning with the 2005 ACS, and continuing every year thereafter, one-year estimates of demographic, social, economic, and housing characteristics are available for geographic areas with a population of 65,000 or more. These areas include the nation, all states and the District of Columbia, all congressional districts, approximately 800 counties, and 500 metropolitan and micropolitan statistical areas, among others—about 7,000 geographic areas in total. Starting in 2008, multi-year data products (for the period 2005–2007) are available for all areas of 20,000 or more—roughly an additional 7,000 areas. In late 2010, the Census Bureau began the release of the first five-year multi-year estimates (for the period 2005–2009) for all geographic areas down to census tract and block group levels. Since about 90 percent of all local governments in the nation have fewer than 20,000 people, 2010 was the first time these areas received any ACS data, all in the form of five-year period estimates.

Since its inception, the Census Bureau and data users have argued that one of the strengths of the ACS is the ability to make changes to the survey over time that will keep it in tune with national priorities and better meet the needs of data users. Changes have already been made to improve the content and data quality of the ACS and provide new tools and resources for data users.

Content Changes

The year 2008 was an important demarcation point in the ACS program because this would be the first year of a five-year estimate (for 2008–2012) centered on 2010, the date of the decennial census enumeration. Thus, any content changes that would be reflected in ACS estimates centered on 2010 for the smallest geographic levels would first have to appear in the 2008 data collection year. In turn, proposed 2008 content changes would have to be solicited, developed, and tested well before 2008. Therefore, preparations for new ACS data products for 2008–2012 actually began in late 2004, as part of meetings among members of the ACS Federal Interagency Committee, and culminated in the analysis of results from a large national content test held in 2006.

More than thirty federal agencies participated in the review of the ACS questionnaire and planning for the ACS 2006 Content Test. The goal was to improve questions and response categories and to evaluate alternatives for questions that showed some indication of a problem, for example, high missing data rates, estimates that differed systematically from other sources of the same information, or historically low reliability of reporting. The 2006 test also compared the ACS grid response format, in which a question such as age was asked of each household member, followed by another question, compared to the sequential questionnaire format planned for the 2010 Census, in which all of the questions were asked of one household member before proceeding to the next person.

As a result of the input from federal agencies, subsequent Census Bureau cognitive testing, and the 2006 Content Test, the 2008 ACS data collection cycle saw the addition of new questions on marital history and health insurance coverage. Major conceptual changes also were made to the questions on disability, resulting in a break from earlier years of the ACS. In addition to new questions and questions with major modifications, about 30 minor modifications were implemented in 2008. Another change, designed to meet data needs of the National Science Foundation, was the addition of a question on field of study of bachelor’s degree, but it could not be fully evaluated for inclusion in 2008. This new item was added in the 2009 data collection year.

The multi-year data product cycles of the ACS pose a unique problem for content and product changes. The new data collected in 2008 were available in the production year of 2009—but only for single-year data products for geographic areas of population 65,000 or greater. Since five years of ACS data are required for small geographic units with fewer than 20,000 residents, a new or modified question does not reach its full implementation until five full years of data are collected.

Given the logistical hurdles posed by content modifications, the Census Bureau decided to implement major changes only once every five years. Thus, the next major content redesign is focused on the year 2013. The ACS Federal Interagency Committee held meetings to determine needed changes that required testing. Topics considered for the ACS Content Test conducted in fall 2010 included Internet access and birthplace of parents. If all goes well, these questions will be included in the 2013 ACS, and users will see the first data products from these changes in late 2014.

Changes in Methodology/Data Collection

The major methodological change in the ACS since 2005 was the expansion of the survey to its “full implementation” level. Up until 2005, the survey was conducted as a large demonstration test. While data and associated products were processed and released to the public, these activities and products were part of the demonstration phase of the project.

This change to full implementation not only meant a sizable expansion of the initial sample, from 800,000 addresses to about 2.9 million, but also sample coverage of all 3,141 counties in the United States, as well as the 78 municipios in Puerto Rico. The workload for all ACS operations increased by more than 300 percent. Monthly mailouts from the National Processing Center (NPC) at Jeffersonville, Indiana, went from approximately 67,000 to 240,000 addresses per month. Telephone nonresponse follow-up (NRFU) workloads, conducted from three telephone call centers, expanded from 25,000 calls per month to approximately 85,000. More than 3,500 field representatives (FRs) across the country conducted follow-up visits at 40,000 addresses a month, up from 1,200 FRs conducting follow-ups at 11,000 addresses each month in 2004. In addition, full implementation saw the annual sample for Puerto Rico rise to approximately 36,000 addresses.

While virtually all continuing surveys of the Census Bureau limit their sample to the civilian non-institutional household population, in 2006, the ACS program was expanded to include sampling for residents in GQs. Concerns about privacy and the operational feasibility of repeated interviewing for an ongoing survey led to the decision to exclude some types of GQs, such as domestic violence shelters, soup kitchens, persons in targeted non-sheltered outdoor locations, and crews of commercial maritime vessels. Starting in 2006, the ACS sampled 2.5 percent of the population living in GQs for a total of approximately 20,000 GQ facilities and 195,000 people in GQs in the United States and Puerto Rico each year.

By 2008, the National Processing Center had switched data entry of the ACS mail form from a “key from paper” (KFP) approach to a “key from image” (KFI) method. Previously, clerks had keyed the data from mailed-back forms by reading the actual paper form and keying the information into a computer file. With the key from image method, the entire form was optically scanned and captured, which involves creating an image of the questionnaire, interpreting the check box entries with optical mark recognition (OMR) technology, and keying write-in responses from the images using a computerized system. The advantages of KFI include the potential for reduced costs and increased data capture accuracy.

While the initial yearly ACS sample is in the range of 2.9 million housing units, the actual sample upon which estimates are based is much smaller. One major reason for this attrition has to do with the fact that nonresponse follow-up is conducted on a sample of addresses that fail to respond by mail or telephone. Moreover, over the course of its implementation, the ACS has witnessed a general decline in mail and telephone response, further increasing the number of cases that require personal interviews and are therefore subject to subsampling. At present, the final sample that is used for the creation of estimates is in the range of about two million addresses per year or ten million cases over five years. This compares with approximately 16 million addresses in the 2000 long-form sample, which is one big reason why ACS estimates are less precise than those from the Census 2000 long-form sample.

In response, the Census Bureau has made several modifications to the survey sampling rate to improve statistical reliability of estimates for small geographic units as well to better optimize the sample overall. These modifications included increasing the sample for nonresponse follow-up in census tracts with low mail-back and telephone response. Future sampling approaches being considered include increasing the overall size of the initial ACS sample to 3.4 million addresses and a plan to better distribute the sample by increasing the number of cases in small census tracts, while decreasing it in large tracts. In addition, the Census Bureau is reviewing the cost and efficiency of its sampling plan for persons in GQs.

Changes in Data Products

As the ACS program moved into full implementation mode, the product set correspondingly began to evolve. Initially envisioned as a set of fairly basic two- and three-way tables (similar to those in the decennial census Summary Files 1 and 3), demands from both outside users and Census Bureau analysts led to an expansion of the detailed (or base) tables over a period of several years. In 2008, the first major increase in ACS data products was implemented. The number of Base Tables (also known as detailed tables) expanded to approximately 879. Ranking Tables increased to cover 64 additional population and housing characteristics; each includes a table, graphic representation, and chart of statistical significance. Thematic maps also became widely available for the first time. Subject Tables were also introduced in 2008. They are similar to the Census 2000 Quick Tables but contain much more detail than Quick Tables. Subject Tables display percent distributions rather than the estimates, except that the universe for each distribution is displayed as a numeric estimate (for example, a universe might be people aged 16 and older). Subject Tables allow for other measures such as medians and means where appropriate, and include the imputation rates for relevant measures. The Census Bureau issued Subject Tables in 42 subject areas, including education, employment, poverty, income, language, and housing.

Standard reports using ACS data also started to develop over time. Initially, a companion report to the annual poverty report based on Current Population Survey (CPS) data was issued. Since then, a series of reports have been issued on such topics as educational attainment, school enrollment, and fertility, using both the new ACS data and the historical data from the CPS. In 2009, the Census Bureau released a set of 16 “Briefs,” short topical reports on specific data items, with the release of the one-year data products.

Quality measures have always been an important part of the ACS releases. The Census Bureau has updated the Quality Measures section of the ACS website to include national and state level quality measures for the 2007 and 2008 one-year estimates, joining the existing quality measures for the 2000 through 2006 ACS. The quality measures include sample sizes, coverage rates, response rates, and item allocation rates.

In an effort to better educate data users, the Census Bureau developed a series of 12 handbooks aimed at showing the appropriate use of ACS data. These ACS Compass Products include user-specific handbooks, fully scripted PowerPoint presentations, and an e-learning tutorial. Each handbook is designed to instruct and provide guidance to a particular audience. A web-based tutorial was released in 2010 and provides user-friendly training on the ACS, including “how to” demonstrations of accessing ACS data products in American FactFinder, the Census Bureau main portal to ACS data.

While standard data and information products constitute a large part of the continuing program of the ACS, the program also creates the potential for large amounts of specialized products. Historically, since only the decennial census could provide small-scale geographic information, “special tabs” were once-a-decade activities. With the ACS in place, these special products are likely to become more commonplace. Already, unique products looking at post-September 11, 2001 conditions in Lower Manhattan, as well as the states along the Gulf of Mexico after Hurricanes Katrina and Rita, have been provided to help planners and developers assess the impact of these major events.

On a more routine basis, various federal agencies, including the Office of Equal Employment Opportunity, the Department of Housing and Urban Development (HUD), the Justice Department, the Department of Education, and the Department of Transportation (DOT), have developed or are working to implement recurring special tabulations to support laws or legislative requirements of those agencies for particular data on small geographic areas. In fiscal year (FY) 2010 alone, 15 major special tabulation projects were under way. These products help to fulfill ACS’s programmatic goal to provide timely data for decision-making and program applications. As the ACS data systems become more mature, it is likely that the demands for special products will continue to increase.

Robert A. Kominski

Joseph J. Salvo

See also Long Form; Data Products: Evolution