Thursday, March 11, 2010

American Community Survey: Questionnaire Content

© CQ Press 2011

Table 1. Subjects on the American Community Survey, 2005–2011

The determination of the particular questions on the American Community Survey (ACS), what is called “questionnaire content,” is driven by mandates from Congress and the needs of federal agencies. In the process, the ACS also provides state and local government agencies, universities, private businesses, and other data users with essential information for geographic areas and population groups. There are four broad categories of questions covering the demographic, social, economic, and housing characteristics of the American population.

The U.S. Census Bureau coordinates the ACS content development and determination process for the ACS with the U.S. Office of Management and Budget (OMB) through an interagency committee comprised of more than 30 federal agencies. All requests for content changes are managed by the Census Bureau’s ACS Content Council, which provides guide­lines and oversight for pretesting, field testing, and implementing new content and changes to existing ACS con­tent.

History of Content Development

The U.S Constitution mandates a decennial census, but it does not spell out a process for determining content. Early in the history of the census, members of Congress, statisticians, and other scholars urged that at the same time the population was enumerated, other important information should be collected. Thus, decennial censuses have always included both a “count” of persons and questions about persons in households and group quarters (GQs). Topics that were included and the kinds of questions asked have varied from decade to decade and often reflected policy-relevant issues of the time.

For Census 2000, the federal government classified all long-form content into three broad classes: (1) mandatory—a federal law required the use of decennial census data for a particular fed­eral program; (2) required—a federal law or implementing regulation required the use of specific data, and the decennial census was the historical or only source, or the data were needed for case law require­ments imposed by the U.S. federal court system; and (3) programmatic—the data were necessary for Census Bureau operational needs, and there was no explicit require­ment for the use of the data under the mandatory or required categories. In general, questions that were classified as mandatory and required were approved for inclusion on the form; only a few programmatic questions were approved (for example, administrative questions such as telephone number were needed for operational purposes).

Shift from the Long Form to the American Community Survey

Each decennial census long form served as the foundation or starting point for the next decade’s version. Thus, the initial content of the ACS at full implementation was based on the Census 2000 long form and, in general, was very similar. (New questions on the ACS included receipt of food stamps and whether any children were born in the last 12 months, asked of women between the ages of 15 to 50). Even though the collection moved from a once-a-decade census to an annual survey, constraining the ACS content was deemed just as critical as it was for previous decennial census long-form questionnaires, due to the mandatory reporting requirement and the relatively high respondent burden.

To manage and provide oversight for the ACS content, the Census Bureau worked closely with OMB in July 2000 to establish a Federal Interagency Committee for the ACS, comprised of representatives from federal departments and agencies that use decennial census data. Working from the justification for the Census 2000 long form, federal agency representatives were asked to examine each question and pro­vide the Census Bureau with the legal basis for requiring the data, the lowest geographic level required for the variables essential for cross-tabulation, and the frequency with which the data were needed.

This process continued throughout the decade and will continue in the future, in large part because the Census Bureau must justify ACS content as part of a regular cycle of evaluation by OMB. The Paperwork Reduction Act of 1995 (Public Law 104-13) requires that proposed information collections, such as surveys, be submitted to OMB for review and approval every three years.

Policy and Process Governing American Community Survey Content

The ACS is designed to produce detailed demographic, social, economic, and housing data every year. Because it accumulates data over time to obtain sufficient levels of reliability for dissemination for small geo­graphic areas, the Census Bureau must minimize content changes. If a question changes significantly or has not been asked for long enough to accumulate three or five years’ worth of data, then the data will not be available for dissemination for small areas.

In recent decades, statutory language requiring specific data was the primary justification for questions on the decennial census long form. In 2006, OMB, in consultation with Congress and the Census Bureau, adopted a more flexible approach to content determination for the ACS that focused more on OMB’s role as the approving authority and less on using legislation to justify new or revised content. OMB’s responsibil­ity under the Paperwork Reduction Act requires that the practical utility of the data be demon­strated and that respondent burden be minimized, especially for the mandatory collection of information. In making ACS content determinations, OMB considers whether the data are needed frequently and at low levels of geography. They also consider whether there are other sources of the same data available that could meet a requestor’s need. OMB approves new content on the basis of an agency’s justification and program requirements. Legislation that specifies adding content to the ACS may also serve as justification.

The Census Bureau’s ACS content policy is used as a basic guideline for all new question propos­als from federal agencies and the Congress. The bureau does not solicit proposals for new or revised questions from the private sector, from state or local governments, or from the public at large. However, the process for obtaining OMB approval to continue ACS data collection requires that the Census Bureau notify the public of its proposed collection and provide the public with an opportunity to comment on content, burden, methodology, or other features of the survey. Thus, the Census Bureau often receives suggestions and input from internal and external groups, and the Federal Interagency Com­mittee for the ACS obtains broad input from all federal agencies. Stemming from that input, the Census Bureau then coordi­nates the creation of subject area groups that include representatives from the Interagency Committee and the Census Bureau; these groups provide expertise in designing sets of questions and response categories so that the questions tested for possible inclusion or revision will meet the needs of the data users.

American Community Survey Content Changes—Past and Future

The ACS content change process provides guidance for Census Bureau pretesting, including a field test, for all new or modified questions prior to incorporating them into the ACS questionnaire. As with most large surveys conducted by the federal government, changes in ACS content are always field tested prior to implementation. The Census Bureau submits the proposed questions to OMB for approval prior to testing. OMB approves testing for a new or revised question. Once the Census Bureau completes testing, OMB reviews and approves all changes to the instrument, including newly designed questions or revisions to existing questions.

In 2004, planning began for a major 2006 field test so that content changes in the ACS could be field tested before the 2008 ACS questionnaire was finalized. The 2006 ACS Content Test was the first opportunity to test revisions to the Census 2000 long-form-sample questions that provided the basis for the initial ACS questionnaire. The test included new ques­tions on the subjects of marital history, health insurance coverage, and veterans’ service-connected disability ratings. It also included many proposed revisions to topics on both the housing and population sections of the questionnaire.

The test methodology was designed to be similar to ACS data col­lection in the production phase. To measure response error, a computer assisted telephone interview content reinterview also was conducted. Simple response variance and gross difference rates, along with other data quality measures, such as item nonresponse rates and measures of distributional changes, served as indicators of the quality of the test ques­tions relative to current ACS questions. The results of the test led to the implementation of the three new questions as well as a substantial number of revisions. A second, much smaller test was conducted in 2007 to test adding a question on undergraduate field of degree. The test proved successful, and the question was added to the ACS in January 2009. Table 1 lists the subjects on the ACS, since full implementation in 2005.

In September 2010, a new content test was conducted by the Census Bureau. Two new questions—computer and Internet access and parental place of birth—and several revisions to existing questions were tested. If testing demonstrates that these questions are viable, their earliest implementation would be in January of 2013.

Susan Schechter

Chet Bowie

See also Content; Content Determination; Long Form

Did this article tell you what you need to know? Let us know...

1 comment:

  1. This was some thing special to know off! thanks for sharing this with me!!
    questionnaire

    ReplyDelete

Please let us (CQ Press Reference) know if you've found the site helpful or if you have questions that you thought would be answered but aren't.