Summary of the March 8-9, 2012 NCVHS Subcommittee on Population Health Hearing

All official NCVHS documents including meeting transcripts are posted on the NCVHS website (http://ncvhs.hhs.gov)

Department of Health and Human Services

NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS

Subcommittee on Population Health

March 8 – 9, 2012

National Center for Health Statistics
3311 Toledo Road
Hyattsville, MD 20782

Meeting Summary

Hearing on Minimum Data Standards for the Measurement of Socioeconomic Status in Federal Health Surveys

Present:

Subcommittee members:

Larry A. Green, M.D., Co-Chair
John J. Burke, MBA, MSPharm. (via phone)
Vickie M. Mays, Ph.D., M.S.P.H.
Sallie Milam, J.D., Co-Chair (via phone)
Walter G. Suarez, M.D., M.P.H. (via phone)

Absent:

Bruce B. Cohen, Ph.D.
Leslie P. Francis, J.D., Ph.D.
Mark C. Hornbrook, Ph.D.
Blackford Middleton, M.D., M.P.H.
Len Nichols, Ph.D.

Staff and Liaisons:

Douglas Boenning, M.D., ASPE
Nancy Breen, Ph.D., NCI, NIH
Virginia Cain, Ph.D., NCHS/CDC
Nicole Cooper, NCHS
Margorie Greensberg, NCHS, Executive Secretary
Debbie M. Jackson, NCHS
Ketherine Jones, NCHS
Robert Kaplan , Ph.D., NIH, Liaison
Cille Kennedy, Ph.D., ASPE
Jacqueline Lucas, NCHS, co-lead staff for hearing
Susan G. Queen, Ph.D., ASPE, co-lead staff for hearing
James Scanlon, ASPE, Executive Staff Director
Edward Sondik, Ph.D., Director, NCHS, Liaison

Presenters:

March 8, 2012

Sherry Baron, M.D., M.P.H., NIOSH
Kurt Bauman, Ph.D., US Census Bureau
Melissa Chiu, Ph.D., US Census Bureau
Connie Citro, Ph.D., CNSTAT/NRC
James M. Dahlhamer, Ph.D., NCHS
Alfred Gottschalck, Ph.D., US Census Bureau
Michael Hout, Ph.D., UC Berkeley
Ernest Moy, M.D., M.P.H., AHRQ
Charles Nelson, Ph.D., CPS/ACS, US Census Bureau
Thomas J. Plewes, M.A., CNSTAT/NRC
James Scanlon, ASPE
Tom Selden, Ph.D., AHRQ
Mitchell Wong, M.D., Ph.D., UCLA

March 9, 2012

John Czajka, Ph.D., Mathematica
Linda Giannarelli, M.A., Urban Institute
Jennifer Madans, Ph.D., NCHS
Amy O’Hara, Ph.D., US Census Bureau
Jennifer Parker, Ph.D., NCHS
Susan G. Queen, Ph.D., ASPE
Fritz Scheuren, Ph.D., NORC

Others:

Lara Akinbami, NCHS
Dacia Beard, NCI
Suzie Bebee, ASPE
Leslie Cooper, NIH
Llewellyn J. Cornelius, U. of MD
Marcie Cynamon, NCHS
Rashida Dorsey, ASPE
Anne Friscoll, NCHS
Janet Gingold, NCHS
Dale Hitchcock
Julia Holmes, NCHS
Jo Jones, NCHS
Lisa Mirel
Keyona King-Tsikata, APA
Kathy O’Connor, NCHS
Patricia Pastor, NCHS
Deborah Rose, NCHS
Lauren Rossen, NCHS
Nathaniel Schenker, NCHS
Makram Talih, NCHS

EXECUTIVE SUMMARY

NOTE: Please refer to Detailed Summary below, Transcript, and PowerPoint presentations for further inforamtion.

Thursday, March 8, 2012

Call to Order and Welcome

Larry A. Green, M.D., Co-Chair and Vickie M. Mays, Ph.D., MSPH, Hearing Chair

Overview: Sec. 4302, Affordable Care Act

James Scanlon, ASPE
HHS Data Standards, Background, Process, Adoption

The Affordable Care Act aims to reduce disparities in health and health care by improving data collection. The first set of standards for major federal surveys included sex, race and ethnicity, primary language and disability status. Socioeconomic status (SES) follows as the next most recommended standard to pursue, including income, occupation, education and other possible considerations (e.g., “class” is considered in other countries). The Hearing aims to identify state-of-the-art practices among the current portfolio of federal surveys and to consider the variables and potential for standardization. Standardization would enable more uniform data collection and comparisons across surveys. Minimum data collection provides a minimum baseline for standards that everyone adheres to.

Purpose and Use of Socioeconomic Status (SES) Survey Measures Policy

Ernest Moy, M.D., M.P.H., AHRQ
Poverty, Program Eligibility, Health Disparities, Research Evaluation

Congress’ charge to National Healthcare Quality Report (NHQR) and National Health Disparities Report (NHDR) is to produce a report that focuses on racial, ethnic and socioeconomic disparities in health care. IOM guidance was described as were NHQR/NHDR choices and constraints relative to quality of care. Examples were given of how SES is typically shown in reports. The effect of education typically varies across race and ethnicity, which highlights the importance of stratifying by SES. The intersection of these factors identifies populations at greatest risk. Multivariate analyses highlight the importance of independent effects. The advantage of standardization is the ability to summarize over large numbers of measures or measure sets. Typically, income and SES have the largest impact of many different contrasts. Only when data are standardized can contrasts acrossmeasures from different measure sets be compared.

AHRQ, which receives many requests to examine state disparity issues, has re-examined racial and income contrasts. Workforce diversity tracking highlights “natural” social hierarchies. An inherited CMS research project has identified a composite SES index for Medicare beneficiaries based on block group level, which is useful for research purposes, including when individual level SES information is not available. In summary, most federal surveys collect income, education and insurance in ways that can be standardized although they are not exactly the same. Variations exist and not all relevant information is collected (e.g., wealth, childhood SES). Declining federal survey response rates were noted. A recommendation was made to keep financial assets and barriers separate from social assets such as education.

Discussion: State disparities data represent a growth area. Data currently collected for policy and programmatic interventions are inadequate. Policy and program data collection at the local level may need support at the federal level. It would be helpful to develop standards that can be used by states and localities. Social assets, while viewed as important, are not clearly defined beyond education. The National Children’s Study was noted as a rare example of a longitudinal study from birth to adulthood that captures SES. Research suggests that adult healthcare patterns are determined by childhood circumstances. In the occupation category, employer-provided health benefits and sick leave are not examined. It is difficult to analyze occupation in the general household category. A multivariate approach is challenging due to sample size and the need for transparency.

Defining Socioeconomic Status

Michael Hout, Ph.D., UC Berkeley

Specific SES concepts were defined relative to advantages, abilities and privileges tied to occupation. Some advantages and abilities lead to better outcomes while others are dysfunctional. Relative to occupation, different working conditions, income, education and lifestyle provide information about advantages, abilities and privileges although specific qualitative occupational distinctions were noted. Desirable and undesirable occupations were discussed. A criterion for rank ordering occupations to the indices of greatest interest was recommended as was consideration of the temporal dimension of SES. Past a certain age, there is little change in educational level but much change in income fluctuation. Situated between educational and income levels, occupation may be a more robust SES indicator. For older retired persons, wealth measures must be part of the mix. While common usage places one occupation score into a regression with other factors, there is some demand to combine income, occupation and education into a socioeconomic index for inferential work. With no academic consensus, the charge is to generate standards for the measurement of education, occupation and income.

Scoring of occupations was described. Some untested but promising approaches to assessing the SES of children are underway and general agreement exists about the importance of neighborhood context to SES. Although thereis no consensus, specific neighborhood SES variables might include: poverty; crime; educational performance; vacancy/turnover; density; air quality; weather; proximity to hazards; and access to hospitals, clinics and food. The research community must access the entire data array. Social class was further discussed. A suggestion was made that the U.S. consider the English approach; or the assignment of socioeconomic quintiles. “Micro-classes”denote the cutting edge of class research and sociology.

Discussion: It is better to consider three risk factor variables or the weighted average of the three (income, education and occupation). Occupation distinctions add predictive power. NCHS will likely want to combine the three and construct quintiles or some manageable number of groups from those kinds of data.

Occupations coding was discussed as was the SES relative to the perceived security of a person or family, an emerging research area. Job security worries have increased in the U.S. since the 1970s. The General Social Survey (GSS) model stipulates that “if you want to measure change, don’t change the measure.” Occupation questions were further delineated.

The Department of Education has always used SES as an ad hoc measure. Using occupational scores within SES can be powerful, especially within a coding system.

Panel: Education: Measuring Educational Attainment in Census Bureau Surveys

Kurt Bauman, Ph.D., Chief, Education & Social Stratification Branch, US Census Bureau
Mitchell Wong, M.D., Ph.D., UCLA Center for Health Services, School of Medicine
Comments from NHIS/MEPS/Census

Dr. Bauman presented background about the Census Bureau’s history of measuring education. The current ACS system focuses on degrees. Specific paper survey questions were reviewed. Reliability of educational responses (Census 2000) was presented, noting an index of inconsistency. A change in the question improved the GED response rate and decreased the index of inconsistency. Challenges with tracking foreign-bornpopulations were identified. A PowerPoint table demonstrated how education lines up with occupation. A strong correlation exists between professional occupation and advanced degrees. Another chart compared earnings of different educational levels. Different educational levels impact unemployment rates.

Dr. Wong discussed the components of education measurement. Variations in current measures were reiterated. Several slides exhibited the strong relationship between traditional measures of years of education and health outcomes. A wide variation exists in the meaning of “school” and how it impacts SES. Educational attainment was differentiated from achievement. Alternative measures and standardized testing (which differ across states) were described. Educational achievement was reviewed in relationship to other SES measures.Parental and child achievement were compared, noting that most of the educational gap is established by kindergarten. An NIMHD-funded study that examines the relationship between educational achievement and health within a selected sample was described. Personal and neighborhood SES impact educational achievement. Limitations of standardized testing were enumerated. Educational achievement is a predictor of health as well as an SES component. Future studies should link health studies to state-level standardized test scores.

Discussion: Discussion ensued about what education means for future socioeconomic success; how it relates to other occupations; and whether it is better to invest in education than in medical care. Selection bias was raised. Preliminary studies show that being in a high-performance school coupled with higher educational achievement leads to better health behaviors. The notion that educational achievement is better than attainment may have been overstated as both are important. A big step forward would be to learn about what schools people attend. Data on school performance are more difficult to obtain and it is important to consider individuals who excel in school despite deprived conditions.

A minimum education standard was again mentioned with respect to the ACS question; and GSS experimentation with a reporting requirement was noted. With regard to whether the variability across federal surveys is justifiedand helpful, a standard question would be better. Equivalency between states is a contentious issue. It is hard to separate educational attainment, school quality, student achievement and ability or IQ level. A literature review was suggested. Field of study makes a big difference to outcomes and earnings as does the question of how early to invest in education (noting Head Start and other programs). Occupation is not the same measure as education. Measurements take the neighborhood and family into account. Should there be a more adaptive/flexible standard for more general federal surveys? A Hauser/Paloni study asserted that High School grades were the best longevity predictor. Data quality was further discussed relative to the challenges of missing data.

Panel: Income

Connie Citro, Ph.D., CNSTAT/NRC, James M. Dahlhamer, Ph.D., NHIS/CDC, Tom Selden, Ph.D., MEPS/AHRQ, and Charles Nelson, Ph.D., CPS/ACS, US

Dr. Citro noted that it is critical but challenging to include good income measures in federal health and healthcare surveys. Collaboration between federal agencies is needed to lessen survey question variation. Background on the CNS was presented. Uses of income statistics were outlined that describe populations, evaluate programs and understand relationships. Such information is important to disparities research in healthcare access, the economic effects of illness episodes and the economic impact of ACA although income data are difficult to collect and often underreported. CPS and SIPP are the best income surveys. Imputation was discussed, noting that one third of the CPS, SIPP and HIS income data are imputed as are more than 40% of MEPS data. The 2011 Supplemental Poverty Measure was delineated. Income questions within health-related surveys should not burden respondents with the detail of major income surveys. The goal is a minimum question set that produces reasonable estimates. The same (tested) format and wording (developed principally by the Census and BLP) should be used. Income information should be obtained for the entire family.

It is important to approximate the SPM, which provides key information about family size and composition; parental employment status; major in-kind program participation; and MOOP. It was suggested that the Census Bureau develop a calculator to estimate the SPM relative to key income and other SPM components. Shortfalls of the current official measure were mentioned. The Census Bureau, CPS and SIPP (not healthcare surveys) should resolve the income quality issue. From this, calibrations can be developed for healthcare surveys to draw upon. Obtaining neighborhood estimates of median income and poverty for small geographic areas from ACS to append to health survey data records would be valuable. Potentially useful non-survey data was also noted (e.g., food desert indictors).

Dr. Dalhamer focused on the collection of income data in the NHIS as well as on recent design and data changes. He presented an overview of the NHIS and its SES measures (in the family socio-demographic (FSD) and family income (FIN) sections. Key questions about the collection of income data in the NHIS had to do with methodological issues facing federal surveys; best ways to collect SES measures; and how to improve data quality and for whom. Also addressed were changes to the collection/release of NHIS income data to address quality and usability issues, particularly design challenges.

Dr. Selden provided background and addressed income and income types in the MEPS. On SES, education, industry and occupation are covered. Questionnaire design and questions were reviewed. Imputation was discussed relative to missing income; income elements hot-decked separately; correlations; bracket responses; wage-salary data; detailed imputation flags provided on PUF; hot-deck matching of donors and recipients; and HDCOLIMP macro for automated cell collapsing.

Imputation frequencies tables for ages 19+ and 65+ were presented (2009 MEPS –HC). MEPS employs bracketed responses if respondents cannot provide exact income amounts, and earnings data from employment-related questions is available for some respondents who did not report either exact or bracketed responses to income questions. Among persons age 19+ the allocation rates in 2009 MEPS, if income sources by type are weighted by their share in aggregate income, are: 2 percent (no information is available), 3 percent (amount known to be positive), 13.3 percent (based on employment data), 14 percent (based on bracket responses), and 68 percent (exact dollar amount reported). These statistics contrast with the 42.7 percent summary allocation statistic cited by Connie Citro in that they differentiate among partial versus completely missing data and avoid implicit dollar-weighting that gives greater weight to allocations affecting high-income persons. The process of top-coding was discussed and comparisons were made between MEPS and CPS (2009 & 2010 March CPS, excluding military).

Dr. Nelson discussed the Census Bureau’s efforts to collect income data,particularly from the ACS; CPS’s Annual Social and Economic Supplement (ASEP); and SIPP. Surveys were discussed relative to sample size/design; process; timeframes; and questions. ACS and CPS survey income questions were compared relative to wages/salary; self-employment; property income; social security income; supplemental security income; cash publicassistance; retirement income; and an “other” category. CPS and ACS poverty rates (2000-2010) were also compared. Generally, the trends are similar and ACS produces reasonable estimates for income and poverty. Income measurement issues include imputation rates; net underreporting; earnings; transfer payments/benefits; property income and pensions; and fairly consistent underreporting results across surveys with some exceptions. Income questions tend to have the highest survey imputation rates although it was noted that ACS rates are much lower than those of CPS. Use of broad income and poverty measures are recommended.

Discussion: When variables are the same on all household surveys, users and the policy community are better served with a consistent approach to establishing a minimum (and maybe a medium-sized) dataset for basic important variables. The complexities of coordination were pointed out (e.g., different structures and reference periods, etc.). Progress had been made in the disabilities survey areas. More information is needed about the importance of public assistance for MEPS. Comparability questions were raised. Also discussed were the Cisco Systems’ Coordinating Center; the federal Committee on Statistical Methodology; HHS’s Data Council; progress on insurance issues that is occurring across HHS and Census surveys; the impact of resistance to change on surveys; and the CNS.

Definitions of “household,” “poverty,” and “family” were further refined. An improved data collection design is needed but currently tabled. There is value in tying net worth to SES, especially later in life. Additionalmissing data were noted. Survey design changes were further discussed as were timing challenges.

Panel: Occupation

Sherry Baron, M.D., M.P.H., NIOSH, Thomas J. Plewes, M.A., CNSTAT/NRC, Alfred Gottschalck, Ph.D., and Melissa Chiu, Ph.D., Bureau of the Census
Comments from NHIS/MEPS/Census

Dr. Gottschalck evaluated the 2008 ACS employment status question change (presentation co-authored by Braedyn K. Kromer, David J. Howard, and David Hedengren) and comparisons to BLS data (i.e., CPS and Local Area Unemployment Statistics [LAUS] for 2007-2009). The 2008 ACS questionnaire changes were outlined. Background materials included mention of research revealing an underestimation of past employment levels; and an underestimation of unemployment levels relative to CPS and LAUS benchmark data, necessitating modification of employment status questions. The impact of test questions was described, noting implementation of revised questionsin the 2008 ACS. Characteristics of many responding to revised questions were identified, minus some workers with marginal Labor Force attachment. The goal was to compare ACS Labor Force data to BLS data nationally and at the state level. The 2008 question change aligned ACS data more closely with CPS and LAUS benchmark data; and unemployment rate differences were cut in half. A revised “worked last week” question captures additional workers,which data ACS intends to link to administrative records.

Dr. Chiu presented an overview of industry and occupation (I&O) data using the ACS, to include context and history; collection of I&O information; coding and classification; data editing; data quality studies; and considerations and recommendations. Specific questionnaire items were described, including class of worker, industry, industry type and occupation.There is a rigorous change process to standardized questions. Respondent instructions by mail and by well-trained field representatives (FRs) were outlined. All ACS coding is computer-assisted clerical coding (200,000 cases/month) but auto-coding will begin mid-March 2012. The coding process and post-processing of data edits were delineated as were Census I&O codes, industry codes, industry code crosswalks, occupation codes, occupation code crosswalks and coding indexes.

A summary of I&O data standards was presented. Surveys using I&O coding include ACS; CPS; SIPP; American Time Use Survey; National Crime Victimization Survey; NHIS; NYC Housing Vacancy Survey; and the National Survey of College Graduates. Data source considerations from ACS, SIPP and Occupation Employment Statistics (OES) were outlined as were health survey considerations that link occupation and industry to health outcomes and address health-related occupation, job and industry issues. More detailed information might be needed in areas such as specific occupational activities; workplace conditions; exposure to toxic chemicals; and organizational structureor dynamics.

Dr. Baron addressed three key areas: major approaches to utilizing occupational codes as an element of SES and SEP; differences by gender, education, race/ethnicity or income in the quality of occupation survey measures; and important survey items to consider when measuring occupation as part of SES and SEP. Specific approaches were delineated. Social class in U.S. statistics is an underexplored area. Self-employment and supervisory status are underutilized in health studies. A longitudinal NIH-funded study (REGARDS) at the University of Alabama examines supervisory status. The O*NET, a new system created by the Department of Labor, was described with examples of measures relevant to health. How demographic variables impact occupation measurement relative to coding quality was discussed (MESA example given) as was the importance of capturing the longest-held job. The NIOSH Industry and Occupation Computerized Coding System (NIOCCS) is a new artificial intelligence system that automatically codes text data to create Census industry and occupation codes (first version expected December 2012). What should be measured was also addressed. Standard Census I&O and other questions are important to consider for inclusion or linkages. Supervisory status deserves further evaluation.

Mr. Plewes further described O*NET; a relevant National Research Council Study; O*NET’s strengths and weaknesses as a source of socioeconomic data; and selected uses of O*NET data for socioeconomic analysis. O*NET’s data collection process and database quality were outlined; and sample questions were provided. A recommendation was made to focus resources on core database activities, leaving development of most new applications and tools to others.

Discussion: Maintaining survey detail (especially for occupation) and consideration of data use in analysis were emphasized (NHIS example given). Coding challenges were further discussed. Theoccupation measurement of SES not captured by education and income has to do with workplace stress; perhaps physical burden on the job; and linkage of occupation and family. O*NET records job strain. The point of measuring SES for health surveys is to illustrate a gradient across society that appears in income and education data. Granularity of occupation data can help determine policy, interventions or programs that decrease SES burden.

How can occupation measure SES in a standardized way given the changing workplace and new job categories? O*NET is an imperfect way to identify new occupations because such information now comes from secondary sources. Special reviews (e.g., “green” jobs; health jobs) are effective but don’t occur regularly. The function of OMB’s SOC Panel was outlined as was the growing field of home healthcare workers. The trade-off with quick change isa loss of opportunity to measure the churn rate. Protecting respondent confidentiality sometimes means aggregating small occupation categories. Loss of granularity due to aggregation into fairly large occupational groupingsis a big problem. The ability to impute to other databases for relevant exposures could improve information use. O*NET helps by making its microdata available.

Creating worker profiles (e.g., job descriptions, risk factors and where they live) would be useful for developing targeted intervention programs. ACS was further discussed relative to workplace geography, health insurance; use in smaller geographies; and enterprise zones. People should be able to link administrative to survey data on the internet, once disclosure and security issues are addressed. State Data Centers focus on dynamiclocal employer/household dynamic data that are linked to confidential restricted data in the RDCs.

The meeting was adjourned at 5:40 p.m.

Friday, March 9, 2012

Call to Order and Welcome — Larry A. Green, M.D., Co-Chair

Summary of Previous Day — Vickie M. Mays, Ph.D., MSPH, Hearing Chair

The Hearing began with definitions of SES, examining income, education and occupation issues. Social class will not be considered as an area of focus although attention will be paid to the health implications of social status and prestige. The Education Panel noted that education is a proxy for other issues. Is the focus on educational achievement or attainment? Quality of education topics included questions about methods of determining equivalency. Care in choosing productive standards was recommended. Education must be measured as part of the SES construct. The Income Panel acknowledged the complexities of income relative to SES. Users must be able to modify data to fit their needs. Due to different uses of the income variable, there are many ways that such data are collected. The new poverty measure should be considered. Specific recommendations were made for a minimumstandard. The Occupation Panel noted such changes as the development of O*NET and auto coding.

Discussion: The challenges of implementing new standards were noted. Suggestions were made to improve the poverty measure; make survey questions more consistent; and encourage ongoing innovation. Three major healthcare information sources were mentioned (i.e., surveys; patient medical records; and administrative data). It is important to map SES standards in population-based surveys; and then to map thatinformation with standards in EHRs and administrative data to ensure harmonization. Coordination with NCI’s efforts to harmonize psychosocial components of EHRs was suggested.

How should education be considered within SES relative to different state High School degree requirements? A continuing collaborative process is expected such as the Data Council’s work on the development of recently adopted standards. The HIT Policy and Standards Committee of ONC’s HITECH Act is deferring to the Data Council’s SES investigations. A NIOSH-commissioned IOM Report entitled, “Incorporating Occupational Information in the Electronic Health Record” (September 2011), concluded thatoccupation code, industry code and work-relatedness should be incorporated into EHR capabilities. From this Hearing and in light of the current review of state meaningful use requirements, a recommendation could be made to the Secretary, CMS and ONC about incorporating this type of information into EHR capabilities.

Panel: Linkages

Jennifer Madans, Ph.D., NCHS, Jennifer Parker, Ph.D., NCHS, Fritz Scheuren, Ph.D., NORC, and Amy O-Hara, Ph.D., US Census Bureau

Dr. Madans spoke about coordination within the statistical system, noting advantages of data linkage. The role of the federal Committee on Statistical Methodology and the Interagency Council on Statistical Policy were described. Despite many challenges, NCHS is a key participant in many current efforts to improve data linkage across federal agencies. The preference is to link to national databases. Due to few good SES linkages, population data collection systems currently ask about basic SES variables. Gathering income data is very time-intensive and costly. Unique features of the U.S. statistical system were outlined. The Bureau is trying to better understand what the public thinks it does. Tax data are highly sensitive, noting that legislation and individuals can make access to that information difficult.

Dr. Parker noted that survey information is high quality and more easily controlled than linked data. NHIS has developed a controllable imputation model. Some linkages provide SES information and, in certain pilot studies, a better picture of program participant SES status. Analysis challenges are continually present. The RDCs attach contextual information to surveys. Some geocodes are coordinated in-house. Tosummarize, the surveys have good SES information for record and geographic linkages that administrative records do not improve upon.

Dr. O’Hara reported on the Census Bureau’s record linkage capabilities. The Census Bureau has conducted record linkage projects to evaluate data quality. SES information is gathered indirectly, relying on survey data points.

Dr. Scheuren discussed linkage context and the complexities of the linkage paradigm in addition to the purpose of linkages and administrative data. The focus should be on information (rather than data) obtained from weak linkage because it allows for a decent point estimate and a measurable variance. Both content and coverage yield important information. Editing and data quality problems were discussed relative to cost and confidentiality. “Explainability” and comparability were noted as issues. A concept from the 1967 Fellegi-Sunter paper of an upper and lower found limit was described as were the role and collection of paradata. Imputation was further discussed relative to its three bounds (true links; the upper bound (of greatest interest); and the non-links or lower bound); as was linkage. Work with the CPS using paradata was described. Validating results calls for small samples that ground-check the data. A study of Canada’s statistical systems was recommended.

Discussion: The degree of error tolerable for social science research on public health depends upon what is done with the data (“fit-for-use”). Within the healthcare system, the monitoring function and small changes are important. There are different quality requirements for different survey items, noting that accountability, transparency and credibility must be considered. The Census Bureau wants to measure and monitor what is going on, using the “fit-for-use” model. Paradata were further discussed as were proxy respondents; informed consent; information-sharing from the IRS; and survey incentives.

Data collection is the primary function of the twelve main statistical federal agencies (headed by NCHS). In addition to the Interagency Council on Statistical Policy connected to OIRA, a federal Committee on Statistical Methodology resides in OMB. Unlike Canada’s one authorizing legislation, the U.S. has twelve that are not often in sync. Departmental mandates can conflict with the development of a cohesive federal statistical system. Title 13 gives authority to request data from various entities but regulation statutes may prevent data-sharing. CPS poverty and income measurements were further discussed. NCHS is concerned with misclassification. Becausethere is no one official set of U.S. statistics, the statistical community must figure out how to minimize confusion.

While NCHS links with contextual Census data, its primary linkage is with CMS. Setting up links with the Department of Education would be costly and complex. An agreement between the Census Bureau and the Department of Education has been shelved for exceeding statute boundaries. The Census Bureau would like a national database with school-level information. Its Longitudinal Employer Household Dynamics (LEHD) Program has state-by-state employment insurance wage information agreements. An assessment is needed to determine whether the cost, risks and timeline are worth the effort to create a national database. The Census Bureau’s School Staffing Survey was noted as a good resource.

Other topics included adjusting education quality using standardized test scores; RDC data merging; location of SES data collection; collaboration between NIOSH and NCHS; cross-agency methodological work, from which comesthe idea of minimum standards; and the Census Bureau’s push to standardize demographics and core survey questions. Discussion ensued about state and local links (including those of NCHS and AHRQ); lack of infrastructure of community data; and data-sharing.

Panel: Methodology

Measuring SES: Methodology — John Czajka, Ph.D., Mathematica
Survey Objectives and Sample Design — Linda Giannarelli, M.A., Urban Institute
Question Wording and Reference Period — Susan G. Queen, Ph.D., ASPE

Dr. Czajka spoke of measuring income with observations from a comparative analysis of survey income data (drawing on research under contract to ASPE, HHS and the Census Bureau). Key points included: there is no survey that “gets it right” in all respects; income is hardest to measure well in the bottom third of distribution; the distinction between current (monthly) and annual income matters most in the bottom third; non-response to survey income questions is high and even higher to asset questions. In addition, a substantial rounding out of income reporting was noted as were differences in the concept of household or family when measuring poverty; the importance of measuring income sources proportionately; and the growing challenge of retirement income data collection. Aggregate income was delineated and poverty rate differences between surveys werediscussed. An allocation analysis was presented; and the rounding of reported income, which impacts poverty ratings, was outlined. Retirement plan changes were identified, noting that the surveys have not adjusted to the changes. As savings withdrawals are not counted as income, more retiree income will not be counted as support in the surveys.

Ms. Giannarelli presented her thoughts as a user of federal surveys collecting income data, with a focus on low-income populations and on using SES data to assess benefits program eligibility. Two big challenges are misreporting and underreporting; and missing data and how they are addressed through allocation. Differences were noted between “captured” vs. “reported” incomes in the data and additional challenges to ACS income collection were identified. The SNAP Exact Match Study (Julie Parker, Census Bureau working paper) and the “SNACC”Project or Medicaid Undercount Project were described. Possible reasons why people don’t correctlyreport benefits were articulated. While aggregate impacts of misreporting and allocation may not be significant, the impact for specific studies or subgroups can be significant, noting that allocated data differ from truly reported data at the micro-level. Wish list items were identified.

Dr. Queen spoke about the implementation of HHS data standards. The Affordable Care Act Section 4302 has special provisions related to health disparities. The new standards were adopted in October 2011. Survey questions were presented relative to the variables of race, ethnicity, sex, primary language and the disability standard. The idea was to come up with a minimum standard that surveys could more easily comply with that would not limit data collection. Applicability of data standards was outlined as were implementation scheduling and monitoring. The adoption of standards provides a common language; promotes uniformity and consistency; and permits greater comparability across surveys. Practical considerations were noted as were next steps.

Discussion: Clarifications about CPS-ASEC underreporting and the SNAP Exact Match Study were provided. There was further discussion about inconsistencies relative to the seeming eligibility or ineligibility of allocated or truly reported persons. A striking difference between CPS and SIPP was noted in the imputation of welfare, food stamps and benefits. Variables are more important to what’s going on than imputation methodology. Increasing state variations in benefit programs administration were noted. To summarize, a good job is being done with the collection of earnings data (80% of income data). SIPP is the best option to gather non-income data (except for dividends and interest). Questions were posed about whether checks could be incorporated into the question process to improve reports; whether interviewers could be trained to ask income data questions more comfortably; and about how to collect retiree information moving forward. With regard to the latter, a suggestion was made to examine the Survey of Consumer Finances with its focus on assets. Issues to consider include the definitions of income, savings, and savings withdrawals.

Committee Discussion and Next Steps

A suggestion was made to compile a letter focused on recommendations about minimum standards for each of the three variables and to decide whether to include social class in the surveys for consideration at the June 2012 NCVHS meeting. Issues to consider include future survey designs; to what extent agencies, providers and users are harmonizing; and whether the right research agenda is in place to move forward. The Hearing focused on three questions: what are today’s state-of-the-art standards for collecting data to measure SES in federal surveys; what are the variables; and what opportunities exist to standardize these variables. More than one product may come from the work.

A clear recommendation about minimum standards was made in the education arena. The occupation area also gathers data consistently although data analysis is problematic. The more complex income area should be further examined for consistency, paying special attention to the poverty indicator and how to measure non-earnings income. Are individual minimum standards needed for education, income and occupation; and is there a hierarchy? TheSubcommittee’s charge is to determine what surveys best capture information in ways that lead to greater harmonization and coordination. Ms. Greenberg clarified that the goal was to assist the Department in identifying a minimum standard to collect SES in federal surveys. To make a recommendation, the need to examine the HHS surveys was identified but the task’s scope has yet to be determined.

A “great” organizing conclusion is that SES is a latent variable predicted by education, income and occupation, modified by race and gender. The Subcommittee should be careful about making extremely explicit recommendations and should maintain modest expectations for its June letter. One suggestion was to develop mid-level recommendations along with a process recommendation about where to locate follow-up work, noting that NCVHSis not the best group to take this on. How does the U.S. develop a system that provides a coherent overview? The cross-agency OSTP’s involvement was suggested. Necessary components include measurement, methodology and harmonization. Recommendation implications may not be limited to just surveys. What is the impact of social stratification on measurement of the three variables? Could a minimum standard even be found for the income category other than to collect it in a certain way? The letter should include Subcommittee findings, priorities and an opinion about whether minimum standards can be created. A suggestion was made to focus only on federal health or HHS health surveys.

The letter, to be accompanied by a background document, should mention issues for future SES consideration. To summarize, the Subcommittee should complete a letter by June 2012 about its findings, rendering an opinion about the possibility of minimum standards for income, education and occupation (without specifying specific wording of standards); and perhaps suggest a ‘how-to’ process for developing SES minimum standards (with caution about what gets excluded). A good format is to present a finding with an associated recommendation. The range of surveys affected by minimum standards should be considered.

Discussion ensued about the best structure for the letter. A suggestion was made to illustrate whether the intent of the survey data is met by what are collected; and to consider methodological issues. An April 2012 Subcommittee teleconference is needed to consider draft findings and recommendations about minimum standards for HHS population-based surveys (same as for the first set of standards). Because no one is yet in a position to recommend specific standards, the focus should be on the strengths, limitations, variations and best practices for data currently collected. An environmental scan or baseline assessment of how the variables are being collected in federal surveys would be helpful. Any standards should be based on proven methods and measurements with a concept of the core and the minimum, noting constraints.

Next steps include possible participation in a Webinar to fill information gaps and a mid-April 2012 conference call with the entire Populations Subcommittee. More specifically, gaps should be determined by the end of March. Any Webinar and the full Subcommittee call will occur in mid-April or before. The letter should be submitted two to three weeks in advance of the Committee meeting on June 21-22, 2012.

DETAILED SUMMARY

NOTE: Please refer to Transcript and PowerPoint presentations for further information.

Thursday, March 8, 2012

Call to Order and Welcome

Larry A. Green, M.D., Co-Chair and Vickie M. Mays, Ph.D., MSPH, Hearing Chair

Overview: Sec. 4302, Affordable Care Act

James Scanlon, Ph.D., ASPE
HHS Data Standards, Background, Process, Adoption

The Affordable Care Act aims to reduce disparities in health and health care by improving data collection. The Secretary of HHS was asked to adopt a set of data collection standards for use in major federal surveys. The first set of standards included sex, race and ethnicity, primary language and disability status. The statute also gave the Secretary authority to adopt additional standards. HHS Data Council standards recommendations were published for public comment, scrutinized by HHS, Census Bureau partners and OMB, and adopted in October 2011 (available on HHS website). The first set of standards is currently being implemented in the major surveys. Socioeconomic status (SES) followed as the next most recommended standard to pursue, including income, measurement of income, educational level and other possible considerations (e.g., “class” is considered in other countries). Measures like income are important relative to epidemiology, research, disparities and policy.

The Hearing aims to identify state-of-the-art practices among the current portfolio of federal surveys and to consider the variables and potential for standardization. Adopting standards carries a fairly high level of proof and burden, which requires all federal surveys to ask the same specified questions. Standardization enables more uniform data collection and comparisons across different surveys. Minimum data collection provides a minimum baseline for standards that everyone adheres to. It is useful to consider whether any variables serve as barriers.

Purpose and Use of Socioeconomic Status (SES) Survey Measures Policy

Ernest Moy, M.D., M.P.H., AHRQ
Poverty, Program Eligibility, Health Disparities, Research Evaluation

Congress’ charge to NHQR and NHDR is to produce a report that focuses on racial, ethnic and socioeconomic disparities in health care. IOM guidance was described (e.g., 2010 recommendations related to race, ethnicity and language standardization), noting that race and ethnic disparities are not understandable or actionable without SES stratification. NHQR/NHDR choices and constraints were outlinedrelative to quality of care (i.e., use of extant data; need to standardize; creation of an artificial SES hierarchy; and a decision not to do a composite).

Examples were givenof how SES is typicallyshown in reports, including stratified analyses by race, ethnicity and SES. The effect of education typically varies across race and ethnicity, which highlights the importance of stratifying by SES. The intersection of these factors identifies populations at greatest risk. Multivariate analyses highlight the importance of independent effects. These are typically different categories that can help target policy initiatives. The advantage of standardization is the ability to summarize over large numbers of measures or measure sets. Typically, SES or income has the largest impact of many different contrasts. Only whendata are standardized can contrasts across measures from different measure sets be compared.

AHRQ, which also examines state performance (“state snapshots”) relative to quality of care, has received many requests to examine state disparity issues. In addition, racial and income contrasts have been re-examined, allowing for performance comparisons and a better understanding of how state performance gaps between low and high income populations compare to the national gap. Tracking workforce diversity highlights “natural” social hierarchies (e.g., Whites and Asians disproportionately represented among RNS while Blacksare disproportionately represented among LPNs and LVNs; other examples given).

An inherited CMS research project has identified a composite SES index for Medicare beneficiaries based on block group level, which is useful for research purposes.In summary, most federal surveys collect income, educationand insurance in ways that can be standardized although they are not exactly the same. Variations exist (examples given) and not all relevant information is collected (e.g., wealth, childhood SES). Declining federal survey response rates were noted. A recommendation was made to keep financial assets and barriers separate from social assets such as education. What to do with the occupation category remains a question.

Discussion: State disparities data, which includehospital claims data and BRFSS, represent a growth area. Data currently collected for policy and programmatic interventions are inadequate: AHRQ is more of a “primer and motivator.” The state quality and disparities snapshot demonstrates variation, noting even greater variation at the local level. Policy and program local level data collection may need support at the federal level. It would be helpful to develop standards that can be used by states and localities. While “social assets” are viewed as important, they are not clearly defined beyond education. Knowledge of international data collection is limited (although AHRQ works with OECD) and its value questioned.

The National Children’s Study (an NIH longitudinal study from birth to adulthood) most likely considers children’s SES relative to family SES(although it is not published as a separate category). Research suggests that adult healthcare patterns are determined by childhood circumstances. A federal report on the well-being of children has key national indicators and measures that are not necessarily directly linked to SES. In the occupationcategory, employer-provided health benefits and sick leave are not examined. It is difficult to analyze occupation in the general household category. A multivariate approach is challenging due to sample size (example given)and the need for transparency.

Defining Socioeconomic Status — Michael Hout, Ph.D., UC Berkeley

Specific SES concepts were defined relative to advantages, abilities and privileges tied to occupation. While some advantages (e.g., education, affluence) and abilities (e.g., doing well in school) lead to better outcomes in other spheres, other privileges are dysfunctional (e.g., real estate agent showing different apartments to Black and White people). Relative to occupation, different working conditions, income, education and lifestyle provide information about advantages, abilities and privileges, although specific qualitative occupational distinctions were noted.

The mandate of a current NSF-funded project is to redo “occupation” in the General Social Survey (GSS) by coding responses according to the new 2010 Census Bureau scheme.Occupations will also be scored on measures within the GSS while using the American Community Survey (ACS) and the Current Population Survey (CPS) to attach some scores, which provides ways to rank and characterize detailed occupations. A notion that goes back to the 1920s is that some occupations are better than others; and that rank orderings reproduce themselves with a high degree of robustness and regularity. Ratings correlate strongly with attributes of “occupational intelligence” or workcomplexity. Treiman’s 1977 research was cited. A widely-recognized scheme indicates the same rank ordering of occupations, which correlates to occupational education and pay. The Duncan score (average of occupation credentials and pay) predicts outcomes better than public preference rankings. The occupation of farmers and clergy, often romanticized, are examples of why the socioeconomic occupation ranking (predictive score for regression on pay and credentials) performs better empirically than public preference rankings.

One recommendation was to develop a criterion for rank ordering occupations to the indices of greatest interest. Another was to consider the temporal dimension of SES. Past a certain age, there is little change in educational level but much income fluctuation. Situated between educational and income levels, occupation may be a more robust SES indicator. For older retired persons, wealth measures must be part of the mix. While common usage throws one occupation score into a regression with other factors, there is some demand to combine income, occupation and education into a socioeconomic index for inferential work. With no academic consensus, the chargeis to generate standards for the measurement of education, occupation and income, noting that relevant weights might have to be project-specific. Differences between self-administered and interviewer-driven surveys were noted.

Scoring of occupations was described, noting a three-digit post-interview code for different occupations. Children’s SES is complex as some have two co-resident parents while others don’t and some live with extended family members, etc. Some untested but promising approaches to assessing the SES of children are underway, particularly with advances in the handling of missing data and imputed methods. With the introduction of exposure measures, there is now general agreement about the importance of neighborhood context to SES. Although there is no consensus, specific neighborhood SES variables to be considered include: poverty; crime; educational performance; vacancy/turnover; density; air quality, weather, proximity to hazards; and access to hospitals, clinics and food. Data must be anonymized, noting that identification of particular features makes it easier to locate specific neighborhoods.

The research community must access the entire data array. NORC now has its data enclave. Restricted Data Centers (RDC) allow researchers to access geographical detail suppressed from public release by the Census Bureau.NCHS must also arrange for researcher access to neighborhood data. Social class was then discussed. In England, the term “social class”has been re-coded to include seven occupation categorizes. A suggestion was made that the U.S. consider such an approach; or the assignment of socioeconomic quintiles. The cutting edge in class research and sociology are “micro classes.”

Discussion:

When considering risk factors, it is better to consider three variables or the weighted average of the three (income, education and occupation)rather than one or two individually. Occupation distinctions add predictive power. NCHS will likely want to combine the three and construct quintiles or some manageable number of groups from those kinds of data. NORC’s rank ordering of occupations from the 2010 Census will beready for distribution in March 2013.

Occupations coding is a big task, noting the challenge of how to make use of 580 highly differentiated qualitative distinctions. Scores should be assigned, starting with an average of pay and credentials (socioeconomic index) or by using one or the other or both. Additional scores can be generated and used (e.g., probability of health insurance with certain occupational titles) instead of the socioeconomic score. For inspiration on how tomake use of these data, reference was made to a 1997 paper by Bob Hauser and Rob Warren in the journal, Sociological Methodology. The need for standardization was reiterated.

Discussion ensued about SES and perceived economic security of a person or family, an emerging research area. Job security worries have increased in the U.S. since the 1970s. The GSS model stipulates that “if you want tomeasure change, don’t change the measure.” Questions have been added to the education section; use of a show card and factoring in inflation are the only changes to the income section since the early 1970s; and several wealth modules have been developed, although no core wealth measure yet exists. Occupation questions have never been changed but at present, an upgrade to the 2010 protocol is occurring that includes recoding all questions since 1972. A question was posed about whether other models exist for creating public use and restrictive use datasets that release data or make them available for analysis. Dr. Bauman mentioned that the Department of Education has always used SES as an ad hoc measure (although not what sociologists call SES). Using occupational scores within SES can bepowerful, especially with a coding system.

Panel: Education: Measuring Educational Attainment in Census Bureau Surveys

Kurt Bauman, Ph.D., Chief, Education & Social Stratification Branch, US Census Bureau
Mitchell Wong, M.D., Ph.D., UCLA Center for Health Services, School of Medicine
Comments from NHIS/MEPS/Census

Dr. Bauman presented background about the Census Bureau’s history of measuring education. The current system, seen in ACS, focuses on degrees. Specific paper survey questions were reviewed. Reliability of educational responses (Census 2000) was presented, noting an index of inconsistency (specifics given). A change in the question improved the GED response rate and decreased the index of inconsistency. Within ethnic groups, a subset of the Hispanic population and certain foreign-born populationsare most reluctant to indicate “race” on the questionnaire. Other challenges with tracking foreign-born populations were identified (e.g., capturing educational performance and language problems). A PowerPoint table demonstrated how education lines up with occupation. A strong correlation exists between professional occupation and advanced degrees. A PowerPoint chart compared earnings of different educational levels. Currently, 30% of the adult U.S. population have bachelor’s degrees or higher. Educational levels also impact unemployment rates.

Dr. Wong discussed the components of education measurement. Variations in current measures were reiterated. At present, education measurement focuses on years of education (diploma and degree) but not as much on achievement or preparation for future schooling or occupation. Several slides exhibited the relationship between traditional measures of years of education and health outcomes, noting a strong relationship between the two and significant changes between 0 -12 years of education and post-High School and beyond. More gradient was noted in the receipt of health care.

A wide variation exists in the meaning of “school” (public; private; charter; home schooling; independent study; vocational;blended and distance learning) and how it impacts SES. Educational attainment was differentiated from achievement. Alternative measures and standardized testing (which differ across states) were described, noting that the National Assessment of Education Progress allows for state comparisons. Educational achievement was reviewed in relationship to other socioeconomic measures such as income gap (which has risen dramatically in the past 40 years), noting its increase at a greater rate than income inequality.Parental and child achievement were compared, noting that most of the educational gap is established by kindergarten.

A study funded by NIMHD examines the relationship between educational achievement and health within a selected sample (of children applying to high performing charter schools in Los Angeles), focusing on risky health behaviors (smoking; alcohol; marijuana; sex) and SES. The students in the lowest achievement tertile are at greatest risk. In addition, personal as well as neighborhood SES impact educational achievement. Limitations of standardized testing were enumerated (specifics given). Educational attainment has poor correlation with achievement. Educational achievement is a predictor of health as well as an SES component. Future studies should linkhealth studies to state-level standardized test scores.

Discussion: Discussion ensued about what education means for future socioeconomic success; how it relates to other occupations; and whether it is better to invest in education than in medical care. Selection bias was discussed relative to the above-mentioned NIMDH study and a new NIDA-funded study. Preliminary studies show that being in a high-performance school and higher educational achievement lead to better health behaviors. The notion that educational achievement is better than attainment may have been overstated as both are important, although environment matters a lot. Less significant distinctions such as vocational certificates were noted. A big step forward would be to learn about what schools people attend although data on school performance are more difficult to obtain. It is also important to consider individuals who excel in school despite deprived conditions.

A minimum education standard was again mentioned with respect to the ACS question. The GSS is experimenting with a reporting requirement that obtains the name of post-secondary institutions, noting the lack of a national High School database. One good distinction is whether a child attends a private or parochial school but beyond that, study results are unreliable. With regard to whether the variability across federal surveys is justified and helpful, a standard question would be better. Equivalency between states is a contentious issue. The Department of Education is working to lessen the need for college remediation by improving standardized test scores.

It is hard to separate educational attainment, school quality, student achievement and ability or IQ level. A literature review was suggested. Field of study makes a big difference to outcomes and earnings as does the question of how early to invest in education. Head Start has proven that early educational intervention makes a difference. Boston’s ABC Program, the Perry Pre-School Study and the National Children’s Study merit consideration. While related, occupation is not the same measure as education. Measurements take the neighborhood and family into account. Should there be a more adaptive/flexible standard for the more general federal surveys? A Hauser/Paloni study indicated that High School grades were the best longevity predictor (BMI and premature death).

The easiest way to measure education is via attainment, years of education and degrees but gathering more information on quality of education and refining such measures as occupation, income and wealth would give educational data greaterpolicy relevance. Data quality was further discussed relative to the challenges of missing data.

Panel: Income

Connie Citro, Ph.D., CNSTAT/NRC, James M. Dahlhamer, Ph.D., NHIS/CDC, Tom Selden, Ph.D., MEPS/AHRQ, and Charles Nelson, Ph.D., CPS/ACS, US

Dr. Citro noted that it is critical to include good income measures in federal health and healthcare surveys but challenging (in terms of respondent burden and data quality). Collaboration between federal agencies is needed to lessen survey question variation.Background on the Committee on National Statistics (CNSTAT)was presented. The CNSTAT report, Measuring Poverty: A New Approachlays groundwork for the supplemental poverty measure. CNSTATworks with small-area income and poverty estimates; the ACS; THE Census Bureau Survey of Income and Program Participation (SIPP); and the National Children’s Study.

Uses of income statistics that describe populations, evaluate programs and understand relationships were outlined. All measures need to collect continuous income data (exact amounts rather than pre-categorized). Many federal programs are tied to the poverty threshold, with detailed income and eligibility provisions. While income is less useful than education for some research (such as causal pathways between SES and health), it is key to other areas such as disparities research in healthcare access, the economic effects of illness episodesand the economic impact of ACA. Although income data are difficult to collect and often underreported, such information from federal health surveys is needed for descriptive and program evaluation. CPS and SIPP collect the best income data. One third of the CPS, SIPP and NHIS income data are missing and must be imputed as are more than 40% of MEPS data.

The 2011 Supplemental Poverty Measure (SPM) is a better policy analysis and research tool than the official measure in that more information is gathered about families, employment status, in-kind benefits, taxes, medical out-of-pocket expenses (MOOP), child care and child support payments. Income questions within health-related surveys should not burden respondents with the detail of major income surveys. The goal is a minimum question set that produces reasonable estimates (and perhaps a longer set for more analysis). The same (tested) format and wording(developed principally by the Census and BLP) should be used. Income information should be obtained for the entire family.

It is important to approximate the SPM because the current official measure understates how much children are helped and overstates how much the elderly are helped.In contrast to the current measure, the SPM provides key information about family size and composition; parental employment status; major in-kind program participation; and MOOP. It was suggested that the Census Bureau develop a calculator to estimate the SPM relative to key income variables (such as those mentioned above and cash income)and otherSPM components (e.g., child care, other work expenses; and net taxes).The Census Bureau, CPS and SIPP (not healthcare surveys) should collaborate to resolve the income quality issue. From this, calibrations can be developed for healthcare surveys to draw upon. Obtaining neighborhood estimates of median income and poverty for small geographic areas from ACS to append to health survey data records would be valuable. Potentially useful non-survey dataitems were also noted (e.g., food desertindictors).

Dr. Dalhamer focused on the collection of income data in the National Health Interview Survey (NHIS) as well as on recent design and data changes. He presented an overview of the NHIS, especially its SES measures (located in the family socio-demographic (FSD) and the family income (FIN) sections. A sample adult interview section gathers occupation and industry information. Key questions about the collection of income data in the NHIS had to do with methodological issues facing federal surveys; best ways to collect SES measures; and how to improve data quality and for what purpose.

The collection of total family income data from 1997-2006 and its challenges were described, some of which are dealt with through follow-up questions that have not yielded adequate or accurate information. Follow-up questions of 2007-present were changed to allow forcategorization of above or below the poverty threshold. More partial income information is being collected although approximately 12-15% families cannot be categorized into thethree basic poverty categories. Continuous income data are desirable but a 25% overall non-response rate exists. Multiple imputations are used for missing values on total family income and personal earnings (i.e., imputing more than one substitute data value for each missing value; noting that NHIS data have five imputed replacement values annually). The trade-off with multiple imputations is increased analyst burden.

Other recent changes to the collection and release of NHIS income data were noted, to include the release of continuous, top-coded total family income and personal earning amounts on public use imputed income files; and the addition of two unfolding bracket questions as follow-ups to the exact amount question ($150,000 income and 200% of poverty threshold). Collection of personal earnings was discussed relative to total family income. To address discrepancies between earnings and total family income in a small percentage of cases, verification checks are undergoing experimentation and analysis. Experimental work (field-tested questions) on wealth collection indicates that wealth may be a better measure of SES for older adults; and that there is value in gathering such information (despite non-response rates as high as 50%). Follow-up wealth questions have been tabled. Design considerations were raised relative to detail in non-income surveys; reference periods; defining family/household composition; mode of data collection; consistency checks and imputation.

Dr. Selden addressed the topic of income in the Medical Expenditure Panel Survey (MEPS). The MEPS follows a subset of people in the NHIS for two years via a CAPI-designed in-person interview. The main focus is on medical use and expenditures by type of payer as well as on insurance coverage. On MEPS, education, industry and occupation are covered. MEPS collects data at the individual level for a wide range of income types (specifics given), which support total family income and percentage of the poverty line as well as asset and employment data. Alignment with CPS is good due to detailed questions; and final weights are post-stratified to match CPS.

Questionnaire design and questions were reviewed. Changes over time have primarily involved removal of a skip pattern based on income tax filing status that reduced the need for imputation. Income was discussed relative to missing data and imputation; income elements are hot-decked separately; correlations; bracket responses; wage-salary data; detailed imputation flags provided on PUF; hot-deck matching of donors and recipients; and HDCOLIMPmacro for automated cell collapsing. Tables of imputation frequencies for ages 19+ and 65+ were presented (2009 MEPS –HC). The process of top-coding was discussed, noting that the best way to top-code is to find a threshold. Comparisons were made between MEPS and CPS (2009 & 2010 March CPS, excluding military).

Dr. Nelson discussed the Census Bureau’s efforts to collect income data, particularly fromthe ACS; CPS’s Annual Social and Economic Supplement (ASES); and SIPP. Surveys were discussed relative to sample size/design; process; timeframes; and questions. The SIPP, a panel survey that follows poverty and income changes over several years, is the only Census Bureau survey with detailed dataon disability, assets and net worth. A major SIPP redesign will occur in 2013. The CPS ASES, collected annually in February and April, can gather over fifty income sources. The ACS is a large sample mail-out/mail-back survey (three million addresses; two million interviewed households/year) that is conducted throughout the year. Annual data for all population areas of 65,000+ and multi-year data for smaller areas are gathered by the ACS although income data aremuch less detailed than CPS ASES.

ACS and CPS Survey income questions were compared relative to wages/salary; self-employment; property income; social security income; supplemental security income; cash public assistance; retirement income; and an “other” category. This latter category holds the biggest difference in that ACS asks one question about “anything not reported elsewhere,” while CPS asks separate questions for the additional categories. When considering what difference the detail makes, it was noted that the aggregate income generated by CPS was about 4% higher (@$7.8 trillion) than the ACS aggregate (@$7.5 trillion). CPS had higher earnings aggregates and ACS had higher aggregates for self-employment, public assistance, SSI and retirement incomes.

CPS and ACS poverty rates (2000-2010) were compared. More recently, results have converged, with fewer differences than earlier in the decade. Generally, the trends are similar and ACS produces reasonable estimates for income and poverty. The same trends are seen for household income although survey differences exist.The ACS is the only federal source of neighborhood characteristics data. A description of how the ACS is published was presented in the social, economic and housing spheres, noting that estimates are based on five-year averages. The Census Bureau recommends that CPSbe used to examine income and poverty nationally in addition to long-term trends. States should use the ACS because sampling is much more granular than CPS and in addition, poverty data are very good.

Income measurement issues include high missing/imputation rates; net underreporting; earnings; transfer payments/benefits; property income and pensions; and fairly consistent underreporting results across surveys with someexceptions (specifics given). Income questions tend to have the highest survey imputation rates although it was noted that ACS imputation are much lower than CPS rates, primarily because ACS completion is mandatory. A chartof imputation rates (1977-2010) showed that the percent of dollars imputed has increased from 21-35%. Item imputations provide more information about work hours and occupations. Income imputation observations noted recent joint earnings research by ACPS/Census Bureau on matched data; initial research focusing on the impact of earnings imputations on poverty status, with preliminary findings that poverty impacts are small. Use of broad income and poverty measures are recommended.

Discussion: Budget issues aside, Dr. Citro recommended flexibility when looking for ways to reduce non-response.If certain issues with the same basic mechanisms (e.g., health insurance coverage) are the same on all household surveys, users and the policy community are better served with a consistent approach to establishing a minimum (and maybe a medium-sized) dataset for basic important variables.The complexities of coordination were pointed out (e.g., different structures and reference periods, etc.). Progress has been made in the disabilities survey areas, which demonstrates that coordination is possible. More information is needed about the importance of public assistance items on MEPS relative to other surveys. Comparability questions were raised.

Regarding collaboration among agencies, the Cisco Systems’ Coordinating Center is in OMB’s Chief Statistician’s Office. Its six staff members are highly stretched. A federal Committee on Statistical Methodology is related to the Chief Statistician’s Office with subgroups that address key issues in the statistical community. Within HHS, a Data Council addresses such issues as insurance. Progress on insurance issues is occurring across HHS and Census surveys. Resistance to change means that surveys are often added to rather than altered. The Committee on National Statistics is an independent non-profit with a congressional charter to make suggestions to or requests of the government.

A question arose about the changing definitions of “household” and its impact on the income category. The term “poverty” isbeing redefined (i.e., the official definition uses the Census“family” (i.e., two unrelated peopleliving together are two separate units) while the MEPS treats cohabiters as one family, including children. With regard to net worth, there is value in gathering such information but an improved data collection design is needed. This is currently tabled due to the amount of detail that would need to be added to the income section. There is value in tying net worth to SES, especially later in life. It was also noted that other income sources from outside the household sometimes get missed in household surveys (e.g., from grandparents).

Would asking MEPS questions at the couples-level be useful?With the ACA taking full effect in 2013, it would be unfortunate for a survey design change to interruptan examination of issues pre- and post-healthcare reform. All major surveys need funds for ongoing methods panels that experiment, bridge series and allow for overlaps. In response to a question about the best approach to timing the ACA survey income measurement, it was noted that asking about previous 12-month income is an operational rather than data-driven decision. Building an annual dataset from shorter questions would be even better but costly. Other timing challenges were discussed relative tooccupation and measuring SES. The question of how and when “family” is defined further complicates matters. For example, the NHIS collects income for the previous calendar year but defines the family at the time of the interview.

Panel: Occupation

Sherry Baron, M.D., M.P.H., NIOSH, Thomas J. Plewes, M.A., CNSTAT/NRC, Alfred Gottschalck, Ph.D., and Melissa Chiu, Ph.D., Bureau of the Census
Comments from NHIS/MEPS/Census

Dr. Gottschalck evaluated the 2008 ACS employment status question change (co-authored by Braedyn K. Kromer, David J. Howard, and David Hedengren) and comparisons to Bureau of Labor Statistics (BLS) data (i.e., CPS and Local Area Unemployment Statistics [LAUS] for 2007-2009). The 2008 ACS questionnaire improved labor force questions to better capture employment status data. With an increased number of employed persons captured in the 2008 ACS, that data are now more consistent with CPS and LAUS data. Background materials included mention of research revealing an underestimation of past employment levels; and an underestimation of unemployment levels relative to CPS and LAUS benchmark data, necessitating modification of employment status questions.

ACS test questions produced a higher estimate of employed people compared to the control but not a lower estimate of unemployed people (although overall unemployment rate was lower). Given the positive findings, the revised questions (e.g., “worked last week” question) were implemented in the 2008 ACS. Characteristics of those responding torevised questions were identified, noting that ACS was not completely capturing workers with marginalor irregular attachment to the Labor Force. In 2008, approximately 1.2 million people were considered “marginal” workers. Demographic and economic differences between marginal and non-marginal workers were consistent with characteristics of people working temporarily or with changing work schedules (e.g., enrolled in school; self-employed). The goal was to compare ACS Labor Force data to BLS data nationally and at the state level.The 2008 question change aligned ACS data more closely with CPS and LAUS benchmark data; and unemployment rate differences were cut in half. A revised “worked last week” question captures additional workers. In the future, ACS intends to link the “worked last week”data to administrative records to better capture that segment of the population.

Specific questionnaire items were described, including class of worker, industry, industry type and occupation. There is a rigorous process for changing questions, even to standardize them with other federal surveys.Respondent instructions by mail and by well-trained field representatives (FRs) were outlined. A data capture file is developed with the gathered data and keyed from a scanned image of the questionnaire (including misspellings and foreign languages). All ACS coding is computer-assisted clerical coding (200,000 cases/month) but auto-coding will begin mid-March 2012. The coding process was delineated as were Census I&O codes, industry codes, industry code crosswalks, occupation codes, occupation code crosswalks and coding indexes. When the data are coded, a post-processing of data edits is completed at Headquarters (example given). Assignment from a donors’ hot-deck is used for incomplete data. A consistency check evaluates consistency among the Labor Force, education and income data.

A summary of standards for I&O data was presented. Surveys using I&O coding include ACS; CPS; SIPP; American Time Use Survey; National Crime Victimization Survey; NHIS; NYC Housing Vacancy Survey; and the National Survey of College Graduates. Data source considerations from ACS, SIPP and Occupation Employment Statistics (OES) were outlined as were health survey considerations that link occupation and industry to health outcomes and address health-related occupation, job and industry issues. More detailed information might be needed in areas such as specific occupational activities;workplace conditions; exposure to toxic chemicals; and organizational structure or dynamics.

Dr. Baron addressed three key areas: major approaches to utilizing occupational codes as an element of SES and Socioeconomic Position (SEP); differences by gender, education, race/ethnicity or income in the quality of occupation survey measures; and important survey items to consider when measuring occupation as part of SES and SEP. Specific approaches include: 1) use of SES as a relative rank in social hierarchies (examples givenof occupation as an indicator of income and education [Nam-Powers score]; and occupation prestige scores in which occupation is an indicator ofsocial status); 2) occupational class, which considers differences in employment workplace conditions and relations, cutting across education, income and social standing. An example of a system that uses occupational class is theU.K.’s National Statistics Socioeconomic Classification System [NS-SEC]), and this system requires three additional variables aside from the occupation code (employer, self-employed or employee; organization size; and supervisory status).

Social class in U.S. statistics is an underexplored area. Self-employment and supervisory status are underutilized in health studies. Workers with the highest rates of unincorporated self-employment are those with the lowest education level. A longitudinal NIH-funded study (REGARDS) at the University of Alabama examines supervisory status, which information is not commonly collected in U.S. surveys. The study has also added an occupational module to the annual telephone survey.

The third major approach to measuring occupation as part of SES and SEP is work content. Research has been done on the impact of work exposures on health and some studies have examined the contribution of work exposure toexplaining health disparities by SES and SEP (note white papers from a NOISH-sponsored conference on Eliminating Health and Safety Disparities at Work – Sept. 2011). Workplace exposures are modifiable and therefore, a pathway to reduced health inequities.

The O*NET, a new system created by the Department of Labor, was described, which provides linkages, based on occupation codes, to occupation-specific measures relevant to health. A potentially important element of work content, which is not captured in systems like O*NET, is whether an individual worker’s job is temporary. Minority and low income people are more likely to hold temporary jobs.

How demographic variables impact occupation measurement relative to coding quality was discussed (MESA example given) as was the importance of capturing the longest-held job.The NIOSH I&O Computerized Coding System (NIOCCS) is a new artificial intelligence system that automatically codes text data to create Census industry and occupation codes, thus decreasing the burden of generating codes (first version expected December 2012) [example and results of preliminary testing given]. What should be measured was also addressed (specifics given).Standard Census I&O and other questions are important to consider for inclusion or linkages (self-employment; work hours/shift; temporary work; employer health insurance and sick leave). Supervisory status deserves further evaluation.

Mr. Plewes further described O*NET; a relevantNational Research Council Study; O*NET’s strengths and weaknesses as a source of socioeconomic data; and selected uses of O*NET data for socioeconomic analysis. O*NET provides information for career guidance, re-employment, counseling, workforce development and research. Its two components are content model (framework for occupational data) and searchable electronic database. The data collection process was outlined and sample questions were provided.

The National Research Council Study Panel charge includes documenting and evaluating current and potential O*NET uses; exploring linkage to SOC and other datasets; and identifying improvements in currency; efficiency; cost-effectiveness; use of new technology; and content model. Aggregation challenges were delineated and a recommendation was made to assess benefits and costs of changing the occupational classification system. Data collection issues were discussed as was improving database quality. A recommendation was made to focus resources on core database activities, leaving development of most new applications and tools to others.

Discussion: Maintaining survey detail (especially with occupation) and consideration of the purpose ofdata use in analysis were emphasized (NHIS example given).Coding challenges may have less to do with identifying job titles and more to do with lack of job steadiness. New occupations identified within the informal economy (i.e., how a person gets paid) are entered into the coding index.Getting the code as precise as possible is desirable, noting that a slightly wrong code might not make that much difference. How to account for status and prestige differences within the same job code has no easy answer.

The occupation measurement of SES not captured by education and income has to do with workplace stress; perhaps physical burden on the job (which may be more health- than SES-related); and linkage of occupation and family.Job stress is a composite of different components. O*NET records job strain. The point of measuring SES for health surveys is to illustrate a gradient across society that appears in income and education data. Granularity ofoccupation data can help determine policy, interventions or programs that decreaseSES burden.

How can occupation measure SES in a standardized way given the changing workplace and new job categories?O*NET is an imperfect way to identify new occupations because such information now comes from secondary sources. Special reviews (e.g., “green” jobs; health jobs) are effective but don’t occur regularly. OMB’s SOC Panel reviews occupations every ten (soon to be eight) years and industries are reviewed every five years. Home healthcareworkers are predicted to have the highest growth percentage and number of new job holders over the next ten years. The trade-off with quick change is a loss of opportunity to measure the churn rate. Protecting respondent confidentiality sometimes means aggregating small occupation categories.

The biggest problem with using occupational data is the loss of granularity from aggregation into fairly large occupational groupings. The ability to impute to other databases for relevant exposures(e.g., health studies) could improve information use. The ability of researchers toimpute between standard occupational groups and job tasks not collected in standard surveys is impressive. O*NET helps by making its microdata available. The Census Bureau believes in using and publicizing datasets to demonstrate usability. The O*NET database may become tied to the autocoder.

In response to what community leaders should know about occupation to make their communities healthier, it was suggested that creating worker profiles (e.g., job descriptions, risk factors and where they live) would be useful for developing targeted intervention programs such as child care at certain hours or job training. ACS is so broad that it can be linked to workplace geography and health insurance. Using ACS in smaller geographies, while not “perfect,” can still be useful (example given). The power of ACS in relation to development enterprise zones was also mentioned as communities can use ACS at very low geographic levels.

People should be able to link administrative to survey data on the internet, once disclosure and security issues are addressed. State Data Centers (typically used by the State Department of Labor) focus on dynamic local employer/household dynamic data that are linked to confidential data restricted to the RDCs.

The meeting was adjourned at 5:40 p.m.

Friday, March 9, 2012

Call to Order and Welcome

Larry A. Green, M.D., Co-Chair

The focus of the Hearing has been on state-of-the-art and current data collection standards for measuring SES in federal surveys; what variables are being collected and what opportunities exist to standardize these variables.

Summary of Previous Day

Vickie M. Mays, Ph.D., MSPH, Hearing Chair

The Hearing began with definitions of SES, looking at income, education and occupation issues. Social class will not be considered as an area of focus although attention will be paid to health implications of social status and prestige.The Education Panel noted that education is a proxy for other issues. Is the focus on educational achievement or attainment? Quality of education topics included questions about methods of determining equivalency. Care in choosing productive standards was recommended. Education must be measured as part ofSES construct.

The Income Panel acknowledged the complexities of income relative to SES. Users must be able to modify data to fit their needs. Due to different uses of the income variable, there are many ways that such data are collected. The new poverty measure should be considered. Specific recommendations were made for a minimum standard. The Occupation Panel noted changes that include the development of O*NET and auto coding.

Discussion: Challenges of implementing new standards were noted. Suggestions were made to consider improving the poverty measure; to make survey questions more consistent; and to encourage ongoing innovation. Three major healthcare information sources were mentioned (i.e., surveys; patient medical records; and administrative data). It is important to map SES standards in population-based surveys; and then to map that information with standards in EHRs and administrative data to ensure harmonization (examples given). Coordinationwith NCI’s efforts to harmonize psychosocial components of EHR (PROMIS, see http://www.nihpromis.org/) was suggested.

How should education be considered within SES relative to different state High School degree requirements? A continuing collaborative process is expected such as the Data Council’s work on the development of recently adopted standards. The HIT Policy and Standards Committee of ONC’s HITECH Act is deferring to the Data Council’s SES investigations. A NIOSH-commissioned IOM Report entitled, “Incorporating Occupational Information in the Electronic Health Record” (September 2011),concluded that three important data elements mature enough to be incorporated into EHR capabilities requirements were occupation code, industry code and work-relatedness. From this Hearing and in light of the current review of state meaningful use requirements, a recommendation can be made to the Secretary, CMS and ONC about incorporating this type of information into EHR capabilities. When informationsources are not controlled, there is less control over information quality.

Panel: Linkages

Jennifer Madans, Ph.D., NCHS Jennifer Parker, Ph.D., NCHS
Fritz Scheuren, Ph.D., NORC Amy O-Hara, Ph.D., US Census Bureau

Dr. Madans spoke about coordination in the statistical system, noting advantages of data linkage for said system relative to quality and cost. Working together ensures that joint issues are addressed more effectively. The federal Committee on Statistical Methodology deals with how best to use administrative records; and the Interagency Council on Statistical Policy brings together statistical agency directors. Despite many challenges, NCHS is a key participant in many current efforts to improve data linkage across federal agencies. Preference is to link to national databases that are consistent across geography and that contain whole populations of interest (e.g., mortality). Due tofew good SES linkages, population data collection systemscurrentlyask about basic SES variables. Gathering income data is very time-intensive and costly. Unique features of the U.S. statistical system were outlined. The Bureau is trying to better understand what the public thinks it does. Tax data are highly sensitive, noting that legislation and individuals can make access to that information difficult.

Dr. Parker noted that survey information is high quality and more easily controlled than linked data. NHIS has developed a controllable imputation model that uses survey information (mortality example given). Some linkagesprovide SES information and, in certain pilot studies, a better picture of program participant SES status (example given). Analysis challenges are continually present. The Research Data Centers (RDCs) attach contextual information to surveys. Some geocodes are coordinated in-house. To summarize, the surveys have good SES information for record and geographic linkages that administrative records do not improve upon.

Dr. O’Hara reported on the Census Bureau’s record linkage capabilities. Without consent to link all survey records, data quality issues arise. This is challenging because the best information on income, education and occupation is attached to those survey data. Incomplete linkages often result from missing data (example given). If a person match is not needed, an address match can be assembled from administrative sources. The Census Bureau has conducted record linkage projects to evaluate data quality. Direct match of survey to administrative records data is done although the Bureau also uses administrative records in indirect applications (example given). SES information is gathered indirectly, relying on survey data points. In terms of data linkages, Census can obtain and analyze tax data that have a Census benefit, although it will not meet the NCHS standard. The Census conducts linkages for incoming survey data with various identifiers (e.g., SSI; commercial). Through the Bureau’s RDCs, individuals can access those data, although the lack of obvious dataset candidates to link to the various surveys was noted.

Dr. Scheuren discussed linkage context and the complexities of the linkage paradigm (how linkages are done) in addition to the purpose of linkages and administrative data. The focus should be on information (rather than data) obtained from weak linkage in that it allows for a decent point estimate and a measurable variance. Both content and coverage yield important information (example given). Editing and data quality problems were discussed relative to cost and confidentiality. “Explainability” and comparability were noted as issues.

A concept from the 1967 Fellegi-Sunter paper of an upper and lower found limit was described as was the role of paradata (e.g., who the interviewer is makes a difference). Imputation, the process of validating results andcompleting the inference, was noted as having three bounds (true links; the upper bound (of greatest interest); and the non-links or lower bound). At the data level, the true link cannot be separated from the true non-link. Bounds depend on the match variables. Linkage probabilities are model-based. All file variables can be used for the inference. Within a total system, the linkage must be brought into the analysis. Multiple systems are now being linked. The characteristics of upper and lower bound were further delineated. Work with the CPS using paradata was described. For the middle period between the true links and true non-links, a recommendation was made to model the uncertainty and use multiple imputations, moving away from counts and toward estimates. At present, paradata are not being collected in the same way across surveys.

Validating results calls for small samples that ground-check the data. A study of Canada’s statistical systems was recommended. Multiple imputations will not yield a perfect match and attention must be paid to calculating the variances. Imputation rates, which have changed dramatically since 1962, were further discussedrelative to matching CPS to Detailed Earnings Record (DER). Poverty estimates of the past 50 years have had manysurvey data problems.

Discussion: The degree of error tolerable for social science research on public health depends upon what is done with the data (“fit-for-use”), noting, however, that the same collection system is used for analyses that could and could not tolerate more error. Within the healthcare system, the monitoring function and small changes are important. There are different quality requirements for different survey items, noting that accountability, transparency and credibility must be considered (which translates into less tolerance for error). The Census Bureau wantsto measure and monitor what is going on, using the “fit-for-use” model (examples given).

Paradata are collected from information the Census Bureau already asks (interviewer, regional and organizational differences noted). Proxy respondents provide information about relationships – they are imputation by people in the household although the Census Bureau defines this by identifying self-reported and non-self-reported persons. Surveys have an informed consent process that does not cover IRS and income data. Information-sharing from the IRS requires written consent for a dedicated time period. Some surveys mandating consent offer incentives (approved by OMB & IRB) but incentives have not been used for linkage approval.

Does the statistical community come together adequately to tackle difficult problems?Data collection is the primary function of the twelve main statistical federal agencies (represented by NCHS at HHS). International principles are adhered to as are OMB directives (specifics given). In addition to the Interagency Council on Statistical Policy connected to the Office of Information and Regulatory Affairs (OIRA), a federal Committee on Statistical Methodology resides in OMB. Unlike Canada’s one authorizing legislation, the U.S. has twelve that are often not in sync. The agencies are embedded in departments in order to provide them with needed information.Departmental mandates can conflict with the development of a cohesive federal statistical system. Title 13 gives authority to request data from various entities but their regulation statutes may prevent data-sharing.

The CPS measures poverty in the U.S. by making a poverty rate estimate. Today about one-third of income measured in the CPS is imputed. NCHS, which examines health characteristics by poverty status, is far more concerned with misclassification. Because there is no one official set of U.S. statistics, the statistical community must figure out how to minimize confusion.

While NCHS links with contextual data from the Census, its primary linkage is with Centers for Medicare and Medicaid Services (CMS). Setting up links with the Department of Education would be costly and complex and it is not clear that the cost of gathering school information would be worth the benefit. An agreement between the Census Bureau and the Department of Education has been shelved for exceeding statute boundaries. For the Census Bureau, a national database with school-level information would be great. Its Longitudinal Employer Household Dynamics (LEHD) Programhas state-by-state employment insurance wage information agreements. An assessment is needed to determine whether the cost, risks and timeline are worth the effort to create a national database. The Census Bureau’s School Staffing Survey was noted as a good resource.

There is no clear way to make an adjustment to the quality of education, except possibly by usingstandardized test scores. Generally, RDCs do not give out data that are brought to them although data merging could be done.SES data are gathered relative to where people live (example given). Data that are released to the public must first be evaluated for fit-for-use. The strongest collaboration between NIOSH and NCHS is NHIS’s occupational health supplement and with the National Occupational Mortality System (NOMS). Cross-agency methodological work is occurring, from which comes the idea of minimum standards. The Census Bureau continues to push for standardizing demographics and core survey questions.

What state and local links exist? Community data are powerful for public health changes but have no infrastructure; and every agency manages these contacts differently. NCHS’s biggest state and local connection is through the Vital Registration System, noting its help in the past to the development of state Centers for Health Statistics. AHRQ works with the states on their Health Cost and Utilization Project (HCUP). The BFRSS distinguishes between fit-for-use and performance requirements. No national database exists for Supplemental Nutrition Assistance Program (SNAP) participants. Data-sharing with research institutions helps the gathering of state data.

Panel: Methodology

Dr. Czajka addressed the topic of measuring income with observations from a comparative analysis of survey income data (drawing on research under contract to ASPE, HHS and the Census Bureau). Key points included: there is no survey that “gets it right” in all respects; income is hardest to measure well in the bottom third of distribution; the distinction between current (monthly) and annual income matters most in the bottom third; non-response to survey income questions is high and even higher to asset questions. In addition, a substantial rounding out of income reporting was noted as were differences in the concept of household or family when measuring poverty; the importance of measuring income sources proportionately; and the growing challenge of retirement income data collection.

Aggregate income was delineated, noting a 5% difference in total income despite huge measurement differences in CPS, ACS, MEPS and NHIS. Despite a weighted population much lower than the total of other surveys, the Panel Study of Income Dynamics (PSID) has the highest aggregate income. The bigger interest is how income breaks out across population segments relative to the aggregates, noting striking similarities between the ACS and the CPS.SIPP captures more at the bottom than other surveys. Population totals are not the same across surveys even though they refer to the same point in time. Earnings account for a little more than 80% of income, noting that ACSand SIPP get many more earnings than the CPS. Big differences also exist in unearned income, which present classification challenges.

Poverty rate differences between surveys were discussed, paying special attention to the impact of family definition. One consistency across surveys is how unwilling many people are to respond to income questions (noting the many income sources). A simple approach to gathering income data does not serve the low-income population well. An allocation analysis was presented; and the rounding of reported income, which impacts poverty ratings, was discussed. Clarifying definitions in terms such as self-employment makes a difference to data results. Retirement plan changes were identified, noting that the surveys have not adjusted to the changes. As savings withdrawals are not counted as income, more retiree income will not be counted as support in the surveys.

Ms. Giannarelli presented her thoughts as a user of income data on federal surveys, with a focus on low-income populations and on using SES data to assess benefits program eligibility. Two big challenges are misreporting and underreporting; and missing data and how it is addressed through allocation (examples given). A table showing CPS-ASES underreporting compared reported enrollment to program data. Ms. Giannarelli noted that even with allocated responses, an incomplete picture of people enrolled in benefits programs is portrayed. Differences between “captured” vs. “reported” income data were noted. Additional challenges to ACS income collection include people filling out forms independently; few income sources collected individually; and combined amounts, to include public assistance and welfare income; asset income; retirement income; and “all other” income. Differences in likely TANF income, employment compensation and SSI income verses program data were outlined (examples given)[ACS 2008]. In general, it was noted that many recipients fail to report enrollment while some reporters are not actually enrolled.

The SNAP Exact Match Study (Julie Parker, Census Bureau working paper) and the ”SNACC” Project or Medicaid Undercount Project were described (for which the adjusted undercount estimate was 32%). Possible reasons why people don’t report their benefits correctly were articulated. Allocated data are a substantial portion of what public-file users analyze, which present challenges when considering low-income families. While aggregate impacts of misreporting and allocation may not be significant, the impact for specific studies or subgroups can be significant, noting that allocated data differ from truly reported data at the micro-level. Wish list items include: for ACS, to ask more individual income questions, reinstate the question on work-related disability and ask if a household lives in public or subsidized housing; for CPS/ASEC, to identify individuals combining schooland employment at any age; and in general, to continue cognitive testing as well as refinements to CATI/CAPI systems and allocation methods.

Dr. Queen spoke about HHS data standards implementation. The Affordable Care Act Section 4302 has special provisions related to health disparities. The new standards were adopted in October 2011. Survey questions were presented relative to the variables of race, ethnicity, sex, primary language and the disability standard. The idea was to come up with a minimum standard that surveys could more easily comply with that would not limit data collection. Applicability of data standards was outlined as were implementation scheduling and monitoring. The adoption of standards provides a common language; promotes uniformity and consistency; and permits greater comparability across surveys. The need to use ACS wording and response categories for disabilities questions may pose a challenge to other surveys. Practical considerations such as requirements of instrument revisions; codebooks and documentation; interviewer training; and crosswalks and bridging techniques were raised. Next steps include a review of current SES-related HHS and Census Bureau survey questions; continued interagency collaboration; and identification of opportunities tocoordinate and/or harmonize data collection.

Discussion: Clarifications about CPS-ASEC underreporting and the SNAP Exact Match Study were provided, pointing to the magnitude of false negatives and even some false positives and the value of cognitive testing. There was further discussion about inconsistencies relative to the seeming eligibility or ineligibility of allocated or truly reported persons. A striking difference between CPS and SIPP was noted in the imputation of welfare, food stamps and benefits. Increasing state variations in benefit programs administration were noted. To summarize, a good job is being done with the collection of earnings data (80% of income data). SIPP is the best option to gather non-income data (except for dividends and interest, for which the best source is the Federal Reserve Survey of Consumer Finances). The Census Bureau uses the terms “allocation” and “editing,” which combined might be a kind of imputation.

Questions were posed about whether checks could be incorporated into the question process to improve reports; whether interviewers could be trained to improve their comfort in asking income questions; and about how to collect retiree information moving forward. With regard to the latter, a suggestion was made to examine the Survey of Consumer Finances with itsfocus on assets. Issues to consider include the definitions of income, savings, and savings withdrawals, noting that the ACS excludes withdrawals from savings.

Committee Discussion and Next Steps

Dr. Mays suggested compiling a letter focused on recommendations about minimum standards for each of the three variables and todecidewhether to include social class in the surveys for the June 2012 Committee meeting. Issues to consider include future survey designs; to what extent agencies, providers and users are harmonizing; and whether the right research agenda is in place to move forward. The Hearing focused on three questions: what aretoday’s state-of-the-art standards for collecting data to measure SES in federal surveys; what are the variables; and what opportunities exist to standardize these variables. More than one product may come from the work.

Dr. Breen heard a clear recommendation about minimum standards in the education arena. The occupation area is also gathering data consistently although data analysis is problematic. The more complex income area should befurther examined for consistency, paying special attention to the poverty indicator and how to measure non-earnings income (e.g., transfer benefits payments; dividends interest; and retirement income).

Are individual minimum standards needed for education, income and occupation; and is there a hierarchy? Dr. Queen thought that the Subcommittee was charged with examining the surveys to determine which ones best capture information in ways that lead to greater harmonization and coordination (rather than recommending use of a specific survey). Ms. Greenberg clarified that the goal was to assist the Department in identifying a minimum standard to collect SES in federal surveys. To make a recommendation, Dr. Mays cited a need to examine the HHS surveys but the scope of task has yet to be determined.

How does the U.S. develop a system that provides a coherent overview? The cross-agency Office of Science and Technology Policy’s (OSTP) involvement was suggested. Necessary components include measurement, methodology andharmonization. Recommendation implications may not be limited to just surveys. What is the impact of social stratification on measurement of the three variables? Dr. Mays wondered if a minimum standard could even be found for the income category other than to collect it in a certain way. The letter should include Subcommittee findings, priorities and an opinion about whether minimum standards can be created.A suggestion was made to focus onlyon federal health or HHS health surveys (but be consistent with Census surveys).

The level of precision in minimum standards depends upon how the data are used (example given). The letter, to be accompanied by a background document,should mention issues for future consideration that are critical to understanding SES. To summarize, the Subcommittee should complete a letter by June 2012 about its findings, rendering an opinion about the possibility of minimum standards for income, education and occupation (without specifying specific wording of standards); and perhaps suggest a ‘how-to’ process for developing SES minimum standards (with caution about what gets excluded).A good format is to present a finding with an associated recommendation.The range of surveysaffected by minimum standards should be considered. The Data Council would probably establish a working group to review Subcommittee recommendations. The OMB has a history of calling groups together to tackle particular issues.

Discussion ensued about the best structure for the letter. A suggestion was made to illustrate whether the intent of the survey data is met by what are collected; and to consider methodological issues. An April 2012 Subcommittee teleconference is needed to draft findings and recommendations about minimum standards for HHS population-based surveys (same as for the first set of standards). Mr. Scanlon stated that because no one is yet in a position to recommend specific standards, the focus should be on the strengths, limitations, variations and best practices fordata currently collected. An environmental scan or baseline assessment of how the variables are being collected in federal surveys would be helpful. Any standards should be based on proven methods and measurements with a concept of the core and the minimum, noting constraints.

The meeting was adjourned at 3:20 p.m.

To the best of my knowledge, the foregoing summary of minutes is accurate and complete.

/s/ June 21, 2012

____________________________________________________

Co-Chair Date

/s/ June 21, 2012

____________________________________________________

Co-Chair Date