[This Transcript is Unedited]

DEPARTMENT OF HEALTH AND HUMAN SERVICES

NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS

SUBCOMMITTEE ON POPULATIONS

WORKSHOP ON DATA LINKAGES TO IMPROVE HEALTH OUTCOMES

September 18, 2006

Renaissance Hotel
999 9th Street, NW
Washington, D.C.

Proceedings by:
CASET Associates, Ltd.
10201 Lee Highway
Fairfax, Virginia 22030
(703)352-0091

List of Participants:

  • Donald M. Steinwachs, Ph.D, Chair
  • A. Russell Localio, Esq.
  • William J. Scanlon, Ph.D
  • C. Eugene Steuerle, Ph.D
  • Nancy Breen, Ph.D
  • James Scanlon, Ph.D
  • Joan Turek
  • Ed Sondik

TABLE OF CONTENTS

P R O C E E D I N G S (9:10 a.m.)

Agenda Item: Call to Order, Welcome and Introductions

DR. STEINWACHS: I am Don Steinwachs. I have the pleasure of chairing the
Populations Subcommittee of the National Committee on Vital and Health
Statistics. We welcome you today to a two-day workshop on data linkages to
improve health outcomes.

I think everyone has a copy of the agenda that spans two days. After saying
a couple of words about NCVHS and what the Populations Subcommittee does, I
will be turning to other members of the committee to talk about the motivation
and the direction that this program has been shaped to try and answer some key
questions for us about best practices and successes in data linkages, and
identifying where there are barriers, and overcoming those barriers to improve
data linkages, to provide information necessary for improving the health of the
public.

As many of you may know, the National Committee on Vital and Health
Statistics is an advisory body to the Secretary of Health and Human Services on
health information and different data policy. The Populations Subcommittee has
a particular focus within that, and that focus is on population health
measurement. In looking at population health, we tend to emphasize the
distribution of health characteristics in the population, trying to identify
disparities in subpopulations.

The program today is shaped in many ways trying to look at those sources of
data that provide potentially important information on risk factors for health,
socioeconomic status, education, income and other factors, as well as
potentially open doors to understanding the health of subpopulations, because
the data are more comprehensive and may exist in national surveys and
interviews that are done.

There is a picture that all of us have on the committee for what health
statistics might be. About five years ago, a report came out of the
subcommittee before any of us who are here today were members, but Marjorie was
there and some others, so can talk about it authoritatively.

It is a report on the vision for health statistics in the 21st century.
That vision talked of bringing together the variety of information that is
needed to look at health risks and health outcomes, to look at disparities in
health and the provision of health services, a very ambitious agenda, one I
don’t think that we can attain without exploiting fully the capacity for data
linkages for existing data, as well as trying to fill the data gaps.

Another piece of work this Populations Subcommittee has taken on is trying
to look at health in minority populations. We found very clearly that in many
minorities, we have insufficient data to be able to say something about the
health status and disparities in health, whether you are talking about Native
Hawaiians, Pacific Islanders, you are talking about the tribes in the American
Indians, and other groups. So when we take on the Secretary’s goal of reducing
disparities in health and health services, we find in many areas we just don’t
have the information. Again, there is hope that possibly through linking data
sets and identifying areas for strengthening data collection, we can improve
that.

Before turning to the introductions specifically for the two-day program, I
would like to go around and have members of the committee introduce themselves,
so you know who is on the Populations Subcommittee and on the staff.

I am Don Steinwachs. I am at Johns Hopkins University, Bloomberg School of
Public Health and Chair of the subcommittee.

(Whereupon, introductions were performed.)

DR. STEINWACHS: I should also let you know that this two-day workshop is
being broadcast on the Internet, so I ask for you to speak into the microphone.
I know that some of the people who are attending around the table do not have a
microphone. In that case, if there is not a microphone easily accessible to the
individual to ask a question, I would ask that the person who is receiving the
question restate the question. Oh, we have a portable mike. I should have never
doubted that technology would be here.

Why don’t we take advantage and also ask the audience here to introduce
themselves for the benefit of people on the Internet.

(Whereupon, introductions were performed.)

DR. STEINWACHS: I would like to turn now to Gene Steuerle and Nancy Breen,
who along with Joan Turek shaped the agenda, and many of you got to talk to as
the agenda came into place for the workshop and have been trying to think
through these issues.

Gene, I don’t know whether you are going to start or Nancy, to make some
comments about the specific expectations for the workshop.

Agenda Item: The Importance of Data Linkages –
C. Eugene Steuerle

DR. STEUERLE: I would like to also extend my thanks to all of you for
coming. I realize that any activity like this is always time consuming. I can
see peoples’ brains now wandering back to what they have got sitting on their
desks that is not being done right now, so I really appreciate that.

I know we have got a couple of speakers that are still in the audience.
This is really meant to be a roundtable. The configuration of the room is what
it is, but please come up to the table. We are not just asking the speakers
around this time, but anybody who is a speaker in a future session, please come
up, because among the things we want to have is a dialogue. We want someone
from say Census to turn to somebody from say IRS and say, if we could only have
merged these data sets, we could have achieved the following. Or, we have a
problem just like that with respect to privacy, or something like that. So we
really do want it to be a dialogue.

I want to be clear that the National Committee on Vital and Health
Statistics, of which I am a member and Don is the Chair, our role is mainly
advisory. The main person we advise, although it doesn’t have to be the only
person, but the main person we advise is the Secretary of HHS. That indeed is
one of our final products as we write these letters to the Secretary and say
the following things should be done.

The purpose of this meeting is not just to talk about what data linkages
are possible or are being done, but we really want the data advisory group to
say, here are things that maybe aren’t being done, or here are things that with
a little bit of additional resources could be done, or here are some real
constraints blocking the following linkage.

I ask all of you to think about it in the following sense. Our ultimate
goal of course is to improve the health of the U.S. population. We are not
linking data sets or we are not raising the issue because we are researchers
and we like bigger data sets, which maybe we do, but hopefully we are doing it
because we have a very specific reason.

I ask you to think about that also in terms of the power of the advice. If
the advice says here are some data sets that are linked, it says one thing. If
we could link these data sets, it is likely we could understand better the
following. That would lead to an improvement in health for the U.S. population.
It carries a lot more weight.

I realize that a number of you, particularly from the agencies, are
resource constrained. That is one of the issues you will probably raise. You
will say, we could do the following were it not for additional resources. There
is probably a certain extent to which politically you are constrained from
saying my boss or someone else in the Administration or Congress was too dumb
to realize I needed more resources, but you can say here are the benefits if we
could do the following, and we can let other people worry about whether those
costs can be met.

We already know and anticipate that two of the biggest constraints that
people will raise once they come up with ideas are with respect to resources,
but also privacy and confidentiality. If you get to that issue, we would like
you to help us also tease out whether there are alternative ways around the
problem. Maybe there are ideal ways we want to share data sets, which I think
is crucial to the extent we can, but even if we can’t, to what extent are we
able to bring in outsiders into agencies, to what extent are we blocked.

I know in dealing with groups that I have advised, at times they are
blocked from bringing people inside the agencies for a lot of reasons that have
little to do with the law or anything else, but just — I don’t want to say
narrow, but almost Catch-22 types of rules. We would like to identify those. So
really, to ask you again to think about the ultimate goal; it is to improve the
health of the population and to help us sort out what could be done and not
done.

In all the sessions we have tried to leave a fair amount of time for
discussion and dialogue. So again, I would like to ask all of you who are not
the formal speakers to jump in. People back here who are too modest to come up
to the table, we really want your advice and input.

I think what we find as we go through is that all of us probably know —
because we are talking about information and data, something in which we can
have an infinite demand, if we could have an infinite supply barring all costs.
So a lot of us know an example here and an example there. The hope of this
hearing is, we can go beyond an example we might gather one by one and put them
together to be able to weave from the examples a broader story. So that is
again the reason we encourage and hope that you will participate. I myself, and
I know I speak for other members of the committee, are deeply appreciative of
the time you have given to us.

Nancy?

Agenda Item: Nancy Breen

DR. BREEN: I would also like to welcome everybody, and thank you for taking
the time to come here to this workshop on data linkages.

We feel it is really important. Gene and I didn’t really now each other
until we both came up with this idea that we should start to investigate what
was going on in the federal government more systematically on data linkages,
and try to determine what are the best practices, and also see if we could do
more along those lines. The examples that both of us knew for data linkages
were so fruitful and were bearing such great research, that it seemed like the
is was something that we should try to move forward more systematically. So
once again, I am delighted that people have taken the time to come.

In the course of developing this, I want to take a minute to thank Joan
Turek, who is sitting next to me. As we were talking about this at the
committee, Gene and I had some ideas, and we had a list, but we didn’t know
everybody in the federal government that was doing this. Joan, it turned out,
did.

DR. STEUERLE: And does.

DR. BREEN: So she filled the gaps for us. I would really like to thank Joan
for what she has given to this. Thank you, Joan.

MS. TUREK: You see what happens when you work for Uncle Sam for 37 years.

DR. BREEN: And so productively, thank you. One of the things that she did
was to make us recognize that the Census Bureau is doing a whole lot in this
area, which I had no idea. I worked at the Census Bureau years ago, and they
were collecting survey data and they were collecting the census. They were
collecting a lot of different data, but as far as I knew, they weren’t doing
much in administrative records.

At the National Cancer Institute where I work, we started working with them
a few years ago on the national mortality longitudinal survey, so I knew that
they were starting to work with administrative records, but Joan made us
realize that there is a whole group of people who were systematically working
through how they could use survey and administrative data to improve not only
the quantity but even more important perhaps, the quality of the data that we
use.

In conversations — can I take this as a transition to start to introduce
the first speakers?

DR. STEINWACHS: Please do.

DR. BREEN: Okay. I guess I will start by introducing the speakers and then
say a little bit about our conversations and what I think they are going to be
talking about, just to bring out some of the high points, and then they will
provide more detail on what they are actually doing.

I would like to introduce Sally Obenski. I think you have been reorganized
recently. The title that she had about a month ago, and also that Ron Prevost
who is sitting next to her, had about a month ago has been changed. This is
such a timely and hot topic that Census has reorganized so that there is now a
division which is the Data Integration Division.

David Johnson, are you chief of that division.

MR. JOHNSON: No, no.

DR. BREEN: Would you like to be promoted to chief of that division today?
You are in that division?

MR. JOHNSON: No, I am the director in my division, the new division, the
demographic director.

DR. BREEN: So you are here. One thing I might want to mention because
people may be interested in this, it wasn’t the impetus for this, but an issue
that is related to access and a number of researchers have been concerned about
was the defunding of SIPP, the Survey of Income and Program Participation, a
really important panel survey that the Census Bureau was doing, which gave us
right up to the date information on employment, on who was in and out of the
labor force. We learned a lot about patterns and long term trends that we could
never figure out from any of the other data sets that we had.

David Johnson — I hope I have this part right, David — is the program
manager on the dynamics of economic well-being system, the DEWS, which is going
to be engineered in order to provide the information that SIPP was providing to
us. So he is here and available to answer questions related to that, so I want
to thank him for that.

Then the third speaker, the formal speaker from the Census here today is
Gerry Gates, who is the Chief Privacy Officer. He is here at the end of the
table.

Of course, we want to know here today how we can improve data systems by
improving data linkages, and how we can improve interagency collaboration to do
that. As you will see, Census is working with all kinds of agencies within the
federal government in order to try to do that, and quite effectively.

Another question that Gene brought up is, how can we insure adequate
access. This is a big issue. As Sally said, we don’t have all the answers, but
we are very committed to working with stakeholders to figure out how we can
best do this. So it is really important that people around this table, at home,
on the Internet and subsequently work with Census in order to try to understand
how we can best get access, because that is a big question that is remaining.

This information is confidential. We don’t want to violate anybody’s
confidentiality, of course. They won’t give us additional data if we do. But we
do want the information to be available to people, because otherwise it is not
helpful in improving population health and reducing and eliminating health
disparities.

Sally is going to give us an overview. Let me just give you Sally’s new
title. Both of them are now in the Data Integration Division, and they are both
assistant division chiefs. Sally is the assistant chief for administrative
records applications and Ron is the assistant division chief for data
management. Sally will provide an overview as I said, and then Ron will talk
about — he has done some interesting research, in which he has found by
looking at survey data and administrative data there is a big mismatch. So
coming from that point, how can we use this finding to improve the data sets
through the linkage mechanism. It is a real quality improvement question, which
when Gene and I started thinking about this wasn’t really where we were
thinking it would go. We were thinking along the lines of just getting more
information.

So I want without further ado introduce the speakers and let them talk
about these issues. We will let all three of the speakers speak, Sally, Ron and
Gerry, and then we will have some time for questions after that. So I want to
turn the mike over to Sally. Thank you.

Agenda Item: Census Bureau

Sally Obenski, Assistant Chief for Administrative
Records Research

MS. OBENSKI: Thank you. As Nancy mentioned, what I would like to do is
provide an overview of the technical infrastructure that is enabling the
expanded use of administrative records and record linkage at the Census Bureau.
It also will provide the 30,000-foot view of projects that are using linked
data sets. We are also going to talk a little bit about operational and
technical constraints, whereas the policy constraints are going to be discussed
both by Ron and Jerry.

As many of you know and others may not, Title 13 which is our guiding
mandate, states that we are to use administrative records extensively as
possible. Also, our strategic plan calls for the use of administrative records
to reduce both reporting burden and minimize costs, and also to come up with
innovative data sources. We are also governed under a plethora of legal
guidance and protections that include Title 13 and Title 26, Privacy Act, and
so on and so forth.

Whenever any of us discuss using administrative records, we always make
sure that the audience is aware that we have commitments to our data providers,
to our data users, and to the public. In order to use other parties’ data we
take it very seriously. We have a stringent infrastructure known as the data
stewardship program that Gerry is going to talk about, in which the use of
administrative records nest. We insure that there is a consistent application
of policies, and we have numerous administrative controls including, before a
project is approved it undergoes quite a bit of scrutiny. We have checklists.
We have to make sure that we are protecting the privacy and always the
confidentiality of the respondents.

Another point that is important, I will make it later, is that although we
acquire data sets that have personally identifiable information on it, those
identifiable information are stripped from the data sets before they are used,
and they are replaced with a protected identification key which is an anonymous
key that allows us to do record linkage while maintaining privacy.

What this gives you is a snapshot of the program evolution, the
administrative records program evolution. The Census Bureau has used
administrative records since the 1940 census, where we used it to identify the
first differential under count. However, in terms of a formal program, it
really came at the aftermath of the 1990 census, in which we showed a severe
and serious differential under count. Out of that came 17 program designs, of
which one was, could we supplant direct enumeration using administrative data.

So a small group of people began researching. There was a privacy
conference that was held in ’93. There were numerous surveys to look at the
feasibility of such an undertaking. About the mid-90s we began developing a
prototype of what is known as a statistical administrative record system, or
STARS.

Now I would like to give you some of what I consider to be the enabling
technology and methods that have allowed us to expand the use of administrative
records. Early on in the program, it became obvious that in order to do
anything seriously with administrative data, we would have to have a database
made up of national files.

So STARS is such a database. It was first prototyped in 1999 to be tested
in the Census 2000 evaluation experimentation program. STARS is comprised of
seven national fields, including IRS files, HUD, Indian Health Service,
Medicare and Selective Service. As you see in terms of the record count, it is
quite a large file. The way that STARS was designed was to conform to the short
form census, in that it has short from demographic characteristics, age, sex,
race, Hispanic origin. Its address part conforms to our master address file. So
this database was created as a prototype in 1999 and tested in 2000.

It was tested in what we call the administrative record experiment or AREX.
What came out of this experiment even surprised the developers of STARS. It was
not so much that it could ever supplant a census, but it validated the
conformance of STARS to census, in the fact that we had demonstrated that we
had captured 85 percent of the census addresses and 95 percent of the persons.
So STARS has been recreated every year ever since, and improvements continue to
be made on it.

Let me talk a little bit and digress a tiny bit about the Numident. The
Numident is an incredibly important file that we acquired in the late ’90s from
the Social Security Administration, and it is the transaction file that has
every SSN that has ever been requested. This is about 803 million records.

What we have done is to collapse this file into the best, into unique
records. We use it for two things. Initially, the file did provide our
demographic data for STARS. Secondly and incredibly importantly is that it
provides our verification and validation system.

Before I talk about the validation system I do want to swerve back to the
demographic data. As I said, initially we used in STARS demographic data from
the Numident. The problem with the race data in particular was that it only
captured white, black and other, and had no ethnicity. Furthermore, race data
on children is no longer being captured on social security records when
children are born.

So to build on this, we in fact built on work that was done by Barry Biden
in the late 1990s, in which he linked the current population survey to the
Numident to start building a race model. We went the next step. We linked the
Census 2000 to the Numident. Where we had a match we brought over race and
Hispanicity and where we didn’t have a match we modeled the differences. What
this has allowed us now is that we have a hybrid system that we have Census
2000 race and Hispanicity and we have the very, very excellent high quality
Numident age and sex.

So the result of this fixed a very substantial weight in STARS, which was
the race data, and it is currently being used by a number of Census Bureau
programs, including the intercensal estimate.

Person identification validation system that we call the PVS. The Social
Security Administration who we have worked very closely with over the years in
a number of venues requires that any file that is going to be linked to their
data must be validated. Originally we worked with SSA on their validation
system to do such activities. Over the last few years we have been working with
them in order to develop our own completely automated system, which has been
approved by them. That is what this PVS is.

The importance of this system can’t be understated, because it is in fact
our record linkage infrastructure, if you will. We use the Numident, the SSA
file as our reference file. When we have a file come in the door, regardless of
whether it is going to be linked to SSA data or not, we run it as part of a
huge quality control check against their reference file. We link addresses, we
match on name, address and date of birth. We search within the address, and
then if we don’t find the person in the address, we go ahead and search by
name. Then we append the record with a unique protected identification key. It
is this pick that is used by the Census Bureau record linkers in order to do
their work throughout the Census Bureau. There is no identifiable information
passed to anyone.

Finally, the other major enabler was the implementation of the American
Community Survey. As you all know, the ACS provides a large timely sample of
essentially decennial long form data. It is essential in order to start getting
data at smaller levels of geography.

What we have been working on with some of our researchers is the idea of
using the ACS to model the model. We have gotten the rules from deeper surveys
such as the SIPP or the CPS, but then in order to push it down to lower levels
of geography we used the ACS.

Now I am going to talk very, very quickly in general about the work that we
do in processing our files and anonymizing them, are used across the Census
Bureau by numerous important programs, including the intercensal estimates and
the small area income and poverty estimates. I would like to remind folks that
these systems are administrative records based and are responsible for the
allocation of billions of dollars of federal funds. Also, as some of you are
familiar with the national longitudinal mortality study, we support them with
our processing, and also the LEHD program.

Now I would like to talk a little bit about some research that we are
involved with that is looking at uses in the decennial census. We have three
major programs underway that are being evaluated in the 2006 census test. One
of them is to see if we can assist, not replace but assist, the hot deck
imputation method by using administrative records to assign age, race, sex,
Hispanic origin when we can match a record.

What this does is, it reduces the work load that falls to the hot deck,
which improves its standard error. This looked very promising, and we are
checking to see if it is operationally feasible in a production environment as
we speak.

The second use was to use administrative records to identify households
with coverage problems. What this is, is a systemic problem where a number of
misses, omissions, come at the within-household. In a given household we tend
to miss people. This is for a whole lot of reasons.

So to ameliorate this, we send out a major field operation, which is the
coverage followup operation, and it is very expensive. We have a research
project underway in which our STAR system was remodeled. We used some modeling
to develop probabilities that certain types of housing units in certain types
of areas would be under covered. This is also currently being evaluated.

Finally, we have been involved for several years now looking to see if we
can enhance the group quarters frame, which was a very challenging endeavor in
Census 2000. We are looking at using Info USA, which is the yellow pages, and
also the ES 202, which is the business register for states. We also have
evaluated seven states and their co-op list, and that program proved so
successful that it is being expanded nationally.

What are some other kinds of survey improvements? Ron is going to talk
about one very important case study, but just a few others that are worth
mentioning. Bob Faye has been working in some very exciting work in seeing if
he can use our STARS database to develop survey controls for reducing ICS small
area variance. This is looking highly promising.

A second body of research we have had underway was to take a look at a
STARS to CPS match and to look at non-respondents and to see if there is any
difference between them and the responders. The linkages occurred, but the
analysis is still underway.

Another thing that was very exciting was our response to the aftermath of
Katrina. As you all know, the effect on the federal statistical system
highlighted an inability to react in a real-time basis. As luck would have it,
we were in the process of acquiring the national change of address file from
the U.S. Postal Service.

So we got together with the data linkage experts. Everybody said this is
our single best chance to get something out there. So they graciously agreed to
give us an abstract, and we used that file to come up with some alternative
survey controls and later on we produced county level tallies using NCOA. We
also were very interested in getting FEMA’s files, but we didn’t have them in
time for the immediately response.

So as an outcome of this, we are looking into the feasibility of developing
the next generation of STARS, which maybe could produce some real-time
measurements.

Here I have given you a very brief snapshot of record linkage and some
exciting projects. What are the constraints? In order to use third party data,
this requires an extraordinarily complex memorandum of understanding in order
to insure that all parties are protected.

Just to give you an example, after Katrina we had immediate discussions
with FEMA. OMB wanted us to have the FEMA files, we wanted them. It should have
been slam dunk; it took nine months. So the other problem is that when you deal
with some of the federal poverty data, they tend to be state based. Dealing
state by state as I think Julia could attest is quite an undertaking.

Also, there are differences in content definition, quality and program
rules. We have come up against this in some of our projects when we are trying
to build eligibility models, and they sometimes differ at the county level, let
alone the state.

Also, lag time. This is a big, big problem. Most of our files lag by about
a year. For example, one of our more important health ones lags by about four
years before we get the national file. As we all are seeing, more applications
require more near-time, real-time response.

Technical constraints, getting the right data in the right format. Even if
we write these very detailed specifications, it many times takes our analyst
many conversations and the data going back and forth to get it right, because
it is coming from two different views, the administrative view versus the
survey integration view.

Also, we do have varying rates of validation, for example, Medicare very
high, Medicaid lower. Just to note, if the record does not validate, we do not
use that record. If we can’t put a pick on it, we do not use that record in our
projects.

Also, something we came up with in terms of looking at SIP administrative
data is the coarseness of administrative data compared to the nuances of
teasing out what we want from the survey. Finally, measuring error, what does
it mean, how do we put a confidence interval around an integrated data set. It
is very challenging.

What are we doing to overcome the constraints? Revolving file acquisition
issues especially among state data may require OMB or Congressional assistance.
The lag time for general demographics is we believe largely addressed by the
national change of address file and possibly moving to this enhanced STARS.

We have under the new Data Integration Division completely standardized and
centralized file acquisition. The next bullet is speaking to continual
improvements in our person validation system, including converting it to SAS,
which has made it much more accessible to our analysts. We have identified a
data quality standards team that is going to be looking at measuring error in
integrated data sets.

What are our conclusions? New files and innovations clearly leading to this
expansion of administrative records uses, but new challenges continue to arise.
The idea of having to regularly update a file like STARS that has got hundreds
of millions of records, and then it is going to be updated on a quarterly
basis, is very, very difficult. Also, just understanding what integrated data
sets are. But I believe, everyone here would believe from the Census Bureau,
that we are at the incipience of a new generation of products and services.

That concludes my talk.

Agenda Item: Ronald C. Prevost, Special Assistant,
Demographic Surveys Division

MR. PREVOST: I guess I will go into my part of the presentation now, which
is talking about health related administrative records research at the Census
Bureau.

What I am going to do is, I will start with a brief commercial announcement
about the small area health insurance estimates program. We will also talk
about the Medicaid undercount study description, its preliminary results, what
our next steps are. We will discuss a little bit about related research we are
doing, the benefits of integrated data sets, and the policy challenges and
conclusions.

The U.S. Census Bureau has embarked upon a small area health insurance
estimates program that produces a consistent set of estimates of health
insurance for all counties in the United States. The intent of the program is
to have published estimates for these counties and states by age, under 18 and
total, with confidence intervals. Right now, we are investigating model
improvements and expanding the age categories for which we can estimate.

Here is just a couple of examples. This is just a brief display. Policy is
implemented locally, and there are significant differences in local area
ability to have insurance coverage. This slide here is a quick snapshot, I
don’t expect you to read it or understand it all, but it is just a blurb to see
what the differences are in the U.S. for the total population without health
insurance. We have a similar situation for children without health insurance.

These health insurance coverage estimates are created by combining survey
data with population estimates and administrative records. Currently we are
working on race, ethnicity, age, sex and income categories that are being
investigated for counties and states. The state level estimates such as the
uninsured black or African-American population under age 18 and less than or
equal to 200 percent of poverty. We are also looking at county level estimates
such as the uninsured children under the age of 18, again under that poverty
constraint. This project is partially funded by the Centers for Disease Control
and Prevention’s national breast and cervical cancer early detection program.

What is forthcoming is health insurance coverage estimates in 2007. We are
going to be providing updated county and state level estimates by age, state
level estimates by race, ethnicity, age, sex and income categories. Then
depending on future funding, the SAHIE program plans to produce county and
state level model based estimates as an annual series.

This is just one way in which we have used administrative records at the
Census Bureau. We use them as Sally showed earlier for inter-censal estimates,
for the smaller poverty estimates, et cetera, and so this is an example of
modeling.

What we are coming up with now for the next part of the presentation will
be the Medicaid under count project. I see that there are a couple of the
collaborators in the audience, Mike Davern and Dave Baugh. They are the experts
here, I am merely the rapporteur. We have had a great collaboration I believe
between our agencies, between the Centers for Medicare and Medicaid Services,
the states who have helped provide their data. We have the Assistant Secretary
for Program Evaluation, and those of us at the Census Bureau. Can’t forget our
sponsors. Our sponsors are the Robert Wood Johnson Foundation and also ASPE.

I think this is a really great project because of the type of collaboration
that we have. When you are integrating data sets, you need to bring in
expertise from both sides, who is expert in the survey data, who is expert in
the administrative data and the use of those data.

What is the Medicaid under count? I’m sure you all are familiar with this.
Survey estimates of Medicaid enrollment are well below the administrative data
enrollment figures. Why is that? In preliminary numbers for calendar year 2000,
the current population survey estimated — and this is using the more
conservative estimates, not the published numbers — 25 million persons that
were in the system. The Medicaid statistical information system or what Sally
referred to earlier as MSIS estimated that there were 38.8 million persons
enrolled. I’ll show you how we get to these numbers a little bit later on.
There is a substantial under count therefore in the CPS relative to the MSIS.
In this case it is 64 percent.

Why do we care? We care because we want to better serve our customers. We
want to improve our surveys, and we want to enhance performance indicators and
provide feedback loops. These numbers are used for policy simulations by
federal and state governments. They are the only source for the number of
uninsured. They are also the only source for the Medicaid eligible but
uninsured population, et cetera. So this under count calls the validity of
survey estimates into question, and this study is intended to understand the
causes.

What could explain the undercount? There are universe differences between
the administrative records that are collected at the states and the survey data
collected by the Census Bureau. There is measurement error. There are
administrative and survey data processing, editing and imputation errors, and
there are survey sample coverage areas and survey non-response biases.

So we came up with a bunch of hypotheses. These hypotheses were drilling
down into these sources of error. Why would there be persons included in the
MSIS but not in the CPS? There are persons living in group quarters, and group
quarters are defined differently by the two agencies CMS and Census Bureau.
There are persons who don’t have a usual residence, but still receive services.
There are persons who receive Medicaid in two or more states. This occurs
obviously because if you move within a given month and you apply in your new
state, you are going to show up in both states, and that is what we have to
work with.

We have to look at what is meaningful health insurance coverage. There are
persons with restricted Medicaid benefits. There are persons that have only
Medicaid coverage for one or a few months. So what does it mean to be insured?

On the CPS side we have respondent knowledge. Because of plan names,
enrollees and Medicaid and prepaid plans may not know they have Medicaid
coverage. We also are hypothesizing that Medicaid enrollees who didn’t use
Medicaid services may not consider themselves covered by Medicaid. We also have
the issue of proxy responses for other members of the household that may be
incorrect, especially for non-family members or when there are households that
have multiple families, and the respondent for the survey is answering for
both.

We developed this study. The steps of the study were several phases. The
first phase was to develop a validated national CMS enrollment file and to
determine the coverage and validation differences between the MSIS and the
MEDB, which is the Medicare enrollment database, to determine the
characteristics of these databases and to look at dual eligibles which we
thought might have a factor in here, and also to conduct a national Medicaid to
CPS person match, and this way we would determine why the Medicaid and CPS
differ so widely on enrollment status, and then we would build a suite of
tables detailing explanatory factors and characteristics.

I mean to say that this study is a longitudinal data file that we are
building. We are collecting data from calendar year 2000 through 2002. These
are the first results from 2001.

In later phases, in order to enhance the study, we determined that there
were some variables that we would like to receive that weren’t in the national
files. So we are currently working with several states to get their local
information to see how we can improve the information we have on the national
files, and how that can be used to conduct a variety of researches on both our
master address file, CPS, the American Community Survey, otherwise known as the
SS-01 in this case for the 2001, and to also look at Medicaid addresses to see
if there was any specific frame bias.

Finally, we want to take a look at the impact that the state data has on
the national data to see if it provides any further explanatory factors that
could potentially apply in the national environment.

The later phase of the study, the fourth phase, and we have about a year
left in the study, give or take a year, we will be matching the national
Medicaid system to the national health interview survey as well to look at
person coverage, and then each way along the way we will be documenting the
results in papers.

Preliminary explanations. We have enforced the CPS group quarters
definitions on the MSIS data where we have administrative data address
information. That is, we were able to locate a person at a specific address and
in the Census master address file. If that address was defined as being a group
quarters we eliminated these people because in the CPS it is a household-based
survey that has only — it is the civilian non-institutionalized population, so
there are components of the population that are not included by definition in
the universe. Then also looking at duplicative persons in different states, and
understanding the covariates for the mid-reporting.

Here is a brief graphic that shows that there is a major overlap in the
universe, but there are folks on both sides that are not included, particularly
those who are under group quarters, those who were deceased during the time
period, and obviously we couldn’t survey them in the CPS, those that did not
have valid records on either side, and those that were in two states. So we had
to build a common universe out of that in order to conduct the study.

We removed the dual eligible cases defined as a group quarters by Census,
and then we ran the data through the Census Bureau’s personal I.D. validation
system that Sally had discussed earlier. We removed these duplicative valid
records and then we removed the MSIS enrollees that were not enrolled in full
benefits.

So how does this break down? We started with 44.3 million MSIS records in
the year 2000. We had one and a half million of those records that were more
than one state or were in a group quarters, and we had four million that had
partial benefits. Partial benefits, this is a situation where you only had
received Medicaid for a day, or where you only had use of family planning
services, et cetera. I know there are a whole variety of these things, and Dave
Voss, he is the master here, he can tell us if you have more questions on this.

On the sample loss side, nine percent of all MSIS records did not have a
valid record, and were not eligible to be linked into the CPS. On the current
population survey side, 6.1 percent of the respondents’ records were not
validated, but more importantly, roughly 22 percent refused to have their data
linked. What this meant was, they refused to provide a social security number.
Therefore, the way that we interpreted it was, if you do not provide an SSN, we
will not link your data.

However, the effectiveness of the ID validation system that has been
developed by Census Bureau has allowed us to change the method of collection of
information from respondents, so that we are not asking for social security
number anymore. There really is a two-part question. One is, what is your
social security number and the other is, can I link your data. We really should
separate those two out. And because we can link data without a social security
number, and we know that is particularly sensitive, we have eliminated that
from our demographic surveys.

So in the future, hopefully we will be able to link more data that way. I’m
sure Gerry will be talking about the privacy implications and how Census Bureau
is addressing that.

Here is an example of the validation differences that you see across the
United States in the records coming from the Medicaid system. Those areas in
red and those areas in black, you can’t see the black too well, but there is a
number of them in California and also up in Montana, are areas of the country
where we had the worst validation rates. So if one was conducting a study in
the state of Montana or the state of California and attempting to apply that
study to the United States, you would get a very different view.

In California in some cases, I believe they are up to almost one third of
the records that were not validated. California has different rules for their
Medicaid systems. They serve a slightly different population than the rest of
the United States, so we are hoping that the state data that they provide us
will assist us in being able to do further linkages and improve it so that we
don’t have these validation issues.

In the state of Montana, the reason we had issues linking the data was that
in some cases, I think it was particularly children, they used state IDs, case
numbers, in the social security number field, at least in this year. I don’t
know how long that continued.

We matched the respondents together with the reported data only. We have
12,341 CPS person records that matched into the MSIS, 1906 had imputed or
edited CPS data, which was about 15 percent of the total.

If you are looking now at why the dysjuncture between what we saw in the
administrative records and what we saw in the survey, 60 percent of the
respondents in CPS responded that they had Medicaid, nine percent responded
that they had some other public type of coverage, but not Medicaid, even though
we had on the administrative record showed that they were in Medicaid.
Seventeen percent responded that they had some type of private coverage but not
Medicaid, and 15 percent responded that they were uninsured.

So basically what you can gather from this is that people really don’t know
the source of their health insurance coverage.

The factors that were associated with this error was the length of time
that they were enrolled in the system and how recently they were enrolled. For
example, and we have in other studies as well, when you asked the question,
have you been enrolled in X for the last year or over the last calendar year,
if they are currently enrolled in that month and you ask them the question, you
get a very good response. We had a similar study in food stamps, and if it was
within one or two months of the survey month that the persons had been enrolled
and participated in the program, they were showing only a ten to 20 percent
response error. If it had been six months since they had received benefits from
the program, we were showing response errors in the range of 60 to 80 percent.

Poverty status impacts Medicaid reporting, but it does not impact the
percent reporting that they are uninsured. This gets back to that stigma issue.
Stigma does not seem to be a factor here. As a matter of fact, it is the folks
that are at the higher levels of income who still qualify for the program who
are most likely to misreport. They think that they are getting private health
insurance and not Medicaid.

Adults 18 to 44 are less likely to report dis-enrollments, and adults 18 to
44 are more likely to report being uninsured. Overall, the CPS rate of those
with Medicaid reporting that are uninsured is higher than in other studies, and
the CPS rate of those with Medicaid reporting Medicaid is lower than with other
studies.

The work that is remaining. I already discussed briefly the other phases of
the project, where we will be bringing in the state files. We will be using the
MSIS data to enhance the study. We will also be bringing in the analytical
extract, the MAX file, to take a look at differences between those who are
enrolled and those who are enrolled and receiving benefits, which may have a
big difference in the way that the responses are occurring.

We hope to soon be working with the national health interview survey, and
then we will be doing a comparison measure of error in the CPS to the state
survey experiments, and then also looking at how well the NHIS does, because
the NHIS asks the question very different than the CPS does. The question we
have is, if the NHIS does better, is that telling us that we need to change the
way in which we are asking the question.

We will also be evaluating how well the CPS edits and imputations work both
at the micro level and overall macro level. We will be evaluating additional
state level Medicaid, and then we will be looking at the coverage area and
survey non-response bias.

As I said, these are preliminary results. They are subject to change after
further investigation. At the moment, we conclude that the survey measurement
error is playing the most significant role in producing the under count. Some
Medicaid enrollees answer that they have other types of coverage, and some
answer that they are uninsured. The overall goal of this project though is to
improve the CPS for supporting health policy analysis, especially refining the
estimates of the uninsured.

The use of integrated data sets is really an important growth that we are
seeing here. You couldn’t get the sort of analysis that we are doing by looking
at aggregate data from administrative records or aggregate data from the
current population survey, or any other survey for that matter. You have to
look at the unit level to determine what the causes are of why misreporting or
dysjuncture is occurring between the two systems.

The administrative data, while they may provide the experience, it is
really two sets of truth. The administrative data show you the information and
the experience that federal agencies have and state agencies have working with
a given set of individuals. They only collect that administrative data, they
don’t talk about all the demographics and social and economic information that
you really want to have in order to have a really complete picture for use of a
multitude of areas, including the development of policy, the implementation of
policy and the evaluation of your activities.

There are some related research examples I just wanted to share with you
briefly. We had a similar experience, where we started down this path three
years ago with the Maryland food stamp study, where we worked with some folks
at the Jacob Branz Institute in Maryland. We matched the American Community
Survey data to the food stamp recipient data, and we found that we had 50
percent response error. That is, the Maryland food stamp recipients were 50
percent higher than we were showing in our estimates. I understand there are
other surveys out there that have come up with similar results.

We were able to explain in our linkage study 85 percent of the discrepancy
by looking at these individual records. In fact, the misreporting was 63
percent of that discrepancy, much of it due to the temporal biases that we were
seeing that I mentioned earlier, and also to the fact that there seemed to be a
serious dysjuncture when there was a person who was not receiving the benefit
who was the survey respondent, or who had not applied for the benefit.

Other related research that we are doing in integrated data. We are working
with the University of Chicago Chapin Hall Center for Children on a child care
subsidy study. The other thing that this integrated data set does that you
would never get from administrative data alone is that it allows you to develop
eligibility models, those people who are eligible for a program but are not
necessarily participating. So we are looking at developing eligibility models
in the study, and then the researchers will examine the effects of this on
employment and self support, that is, the outcome measures at our research data
centers.

If we were to add data, we were asked earlier what data would you add to
the study if you could to improve research on health outcomes. We have
identified a few data sets that we think are of particular importance that if
we could bring it together with this three-year longitudinal file we already
have, that would be really important. It would be the WIC files, files from the
NHANES survey, the MEPS survey. We have not yet linked in the CPS food security
supplement, but that would be great, to bring in relationship data from the
SS-5 information that Social Security Administration collects. The SS-5 is your
application for a Social Security card; this currently is not on our extract.

There are other federal health insurance data sets that would be important
to look at. I think there are future things that we could look at like
co-insurance and its effect on things to bring in the VA medical health data,
to bring in tri-care information, to bring in the Indian Health Service
information. I think if you could get a good picture of what was happening in
the federal sphere, then we could take it to the next step perhaps and see if
there were any data out there for private health care coverage.

So the value of integrated data sets is that they really provide a more
robust and accurate picture of what is going on. It builds on both views of the
world, what the agency is seeing and what the persons are experiencing, but it
controls for both their weaknesses. It provides better statistics for input
into simulations, for predictions and for funds distribution, and as the demand
for data increases, and frankly we are all experiencing a budget decrease, data
reuse may be the only cost effective option for moving forward in the future.

So in doing this, we have a bunch of policy challenges, that is,
communicating the benefits of integrating these data sets versus the privacy
concerns, the need for interagency teams to insure accurate results. I think
this team that we had on the Medicaid study was a great picture of that,
because it wasn’t until you had all the expertise from every side to bear on
one specific problem that you could really address it.

To look at interagency agreements, as Sally said, what ends up happening
with many of these research activities is that you set a research plan for two
years, and you spend a year and a half trying to get the data. There needs to
be a more effective way of moving forward.

Then once the data are put together, who owns them? Big question. Everybody
owns them. Then there is the potential growth of possible disclosure risks. If
you were to try to blend these data sets together and then release them as a
data file, how do you do so without the administrative agency who had provided
you that data being able to back out the survey respondents’ information? This
is certainly that we can’t allow to happen. I’m sure Gerry will talk about
that.

Then there is the need for these longitudinal databases to find an
anonymized person at an address at a specific point in time. That really should
be our vision in the future. But how do you do that and balance it with
privacy?

In conclusion, integrated data architectures are the future of American
statistics. There was recently a paper released at the UN ECE conference. The
Norwegians, who have been working in similar systems — well, a number of
European countries have, but the Norwegians presented a paper that said they
are changing their approach to the way they are doing work, and it is not going
to be register based anymore. They believe that the blending and the
integration of data is the way that they want to work. So I think we are in
good company there.

As I said earlier, as the demand for data increases and budgets decrease,
data reuse may be the only cost effective option for us. We have to overcome
technical and policy related challenges, and this approach will support
evidence based public policy research on decisions.

Thank you.

Agenda Item: Jerry Gates, Chief Privacy Officer

MR. GATES: Good morning. As you heard, my name is Gerry Gates. I am the
chief privacy officer at the Census Bureau. It is a position I have held for a
little over a year now. Prior to that I was chief of the policy office, and for
over ten years prior to that I was the administrative records program officer
at the Census Bureau, responsible for coordinating administrative records
access and use. I have had some fairly close relationships with several people
in this audience over the years, acquiring administrative data for our
programs.

What I want to talk to you today about is some of the policy issues
associated with acquiring and using administrative data for statistical
purposes. I think the key policy issues related to these uses revolve around
trust. I have identified four trust relationships which I think are critical in
determining how we will be allowed to use administrative data for statistical
programs.

The first is between the administrative data provider and the statistical
data collector. In reaching agreements on using administrative data there is a
consideration of several key issues. First of whether the proving agency trusts
the statistical agency to protect the data and use it according to legal and
formal agreements. The second issue relates to whether the providing agency
believes that they have the legal authority to provide that data.

Another issue involves how the providing agency will be protected from any
public backlash associated with the sharing of that information. Another issue
involves what are the risks that the uses of these data may reflect negatively
on the data provider, in terms of whether or not the quality of that data is
sufficient, and whether the program goals are being met.

Another issue involves whether the provider has any say in how the
published data are protected, and what role they will play in that. Finally,
there is the issue associated with whether the provider is able to make use of
the results. In addition to cost reimbursement, will there be any quid pro quo,
will any data be returned to the data provider.

The second trust relationship is between the statistical data collector and
the respondents to our surveys and censuses. This raises issues about what
consent was reached in terms of permitting the data collected in surveys and
censuses to be linked with administrative data, was that agreement a specific
consent or was it a notification, was it an opt in or an opt out agreement,
were there any conditions associated with that agreement and finally, how is
the process transparent to the public at large; are these linkages being done
in such a way that they are well known, or are they not known.

The third relationship is between the administrative data provider and the
program recipient. This relationship is very similar to the relationship
between the statistical agency and the survey respondent. It is an agreement as
to how that information they provide will be used.

Finally, there is a relationship between the statistical agency and the
users of the data. These data are only valuable if they are being made publicly
available or being made available for research purposes. This involves an
agreement as to the quality of the data for the intended use, as well as how
accessible those data are.

So that frames where I want to go with this discussion today, to talk a
little bit about how a statistical agency makes policy decisions in reflection
of these trust relationships that they have to accommodate.

The Census Bureau’s mission includes a prominent statement about our
responsibility to honor privacy and protect confidentiality, as you can see
here. This demonstrates that when we use and collect information, we find it
critically important that we address our relationship of trust with our
respondent.

Our law specifically says that we can only use the information that people
furnish us for no purpose other than the statistical purpose for which it is
supplied, and that we cannot make any publication where the data furnished by
any individual could be identified. So this sets the ground rules. When these
data are collected they must be protected, and they must only be used for
statistical purposes.

So when the data come in, they come in from an administrative agency, and
for our statistical purposes they cannot go back out for administrative
purposes. The law prohibits it.

The Census Bureau has decided to address these trust relationships through
its commitment to what we call data stewardship. It is a formalized program we
have established since 2001 reflecting our management commitment to comply with
our legal requirement and to acknowledge and address our ethical requirements
in terms of professional statisticians.

We have established a formal structure to accomplish this. It is titled our
data stewardship executive policy committee. Through that committee and its
subcommittees, we set policy related to the acquisition of data, the protection
of data and the use of the data. Three formal committees report to our senior
level committee, which is chaired by our deputy director. One of those
committees is our privacy, policy and research committee, one is our
administrative records planning committee, and another one is our disclosure
review board. So as you can see, administrative records are a focal point of
this data stewardship program.

As I said, the program is built around our foundation of values and
principles. Management decisions reflect — I think what is key to this is,
management decisions reflect not only our legal requirements, but consideration
of the ethical obligations to individuals.

We have established controls to formalize and insure that our policies are
met. These are done through the implementation of privacy impact assessments, a
requirement of the Government Act of 2002. What we have done is, we have linked
all our data stewardship policies to the assessment, so that managers have to
acknowledge the compliance with specific policies in addressing risk to privacy
throughout the process, from the initiation, the collection and the processing
of information.

As I said, we have established many new policies that respond to the issues
associated with this, and we continue to develop new policies in response to
our changing environment. Basically, data stewardship is making commitments. It
is a commitment to our data provider to manage and safeguard their information
in accordance with legal and policy requirements. It is a commitment to our
data user community that the administrative records will result in high quality
data products. Also, it is a commitment to the public that we will maintain
confidentiality of personal information, and also make sure that it will only
be used for statistical purposes.

We have legal guidance and protections that are in place. Title 13 of the
United States Code is the basic legal framework for the Census Bureau programs.
As Sally mentioned, Section 6 of Title 13 specifically authorizes us to obtain
records from existing sources, rather than collecting that information again.
So we obtain our legal authority for the collection of administrative records
from that section of the law.

Section 9 of Title 13 is the provision that we must keep the information
confidential. Title 26 becomes very important to the Census Bureau, because
that is the IRS code. The Census Bureau makes use of administrative records
from the Internal Revenue Service, something we have done for over 40 years.
The authority for that is in Section 6103-J of the IRS code that permits the
Census Bureau to obtain tax information for its statistical surveys and
censuses.

We also have specific recognition in the privacy act that statistics are a
routine use for information collected by the federal government. The Privacy
Act specifically singles out the Census Bureau as a routine use, so
administrative agencies can provide information to the Census Bureau under that
provision.

We also are guided by the Paperwork Reduction Act, which instructs that we
collect the information to the minimum extent possible, and use information
that is already available. This is a companion to the Section 6 of the Title
13.

Let me start with some definitions I think it is important to understand.
We talk about confidentiality and we talk about privacy, and they both are
important to the use of administrative data for statistical programs. I think
it is important to understand this distinction.

I get this definition from the IRB Guide Book. Confidentiality pertains to
the treatment of information that an individual has disclosed in a relationship
of trust — again we go with the trust — and with the expectation that it will
not be divulged to others in ways that are inconsistent with the understanding
of the original disclosure without permission.

As I said, there are some legal requirements for confidentiality. The basic
one is Title 13. There are also reflections of confidentiality in the security
guidelines established by the Government Information Security Reform Act and
the Federal Information Security Management Act, which is also part of the
E-Government Act of 2002, and confidentiality is also impacted by the federal
information processing standard, which is FIPS 199. So these all provide a
framework under which we must protect and secure the information that we
collect not only from our respondents, but also data that we obtain from
administrative agencies.

Now, information privacy on the other hand is defined here by Alan Weston
in 1967 as the claim of individuals, groups or institutions to determine for
themselves when, how and to what extent information about themselves is
communicated to others. So this is the individual’s control over their
information and how that information is used. The requirements for insuring
privacy come from Title 13, in the sense that Title 13 says that we can only
use information for statistical purposes. We have to tell people that that is
how we are going to use their information. The Privacy Act instructs us that we
have to tell people about the authority we have to collect the information, the
purpose for the collection, how the information will be used, and whether our
asking for that information is mandatory or voluntary.

We are also guided by the Freedom of Information Act, which says that
private information cannot be disclosed to requesters. Also, HIPAA has a role
to play here as well as the E-Government Act of 2002, which as I said
establishes requirements that agencies must conduct privacy impact assessments
to insure that information is protected.

A little bit about Census Bureau policies as they relate to administrative
record use. These are all policies associated with all our privacy principles.
This is the basis for our data stewardship program. We as an agency have
acknowledged that these are principles that whatever we do must be necessary
for our mission, that we will be open and transparent about what we will do. We
will have respect for the individuals who provide us the information, and we
will provide confidentiality for any information we gather. So these are the
overarching principles of our data stewardship program.

We have established policies to implement those principles or to insure
that we are in compliance with those principles. The highlighted principles as
you can see under mission necessity, linkage of decennial census records and
number five, record linkage. These are policies that we have established
related to our linkage of information. I am going to talk a little bit more
about the record linkage policy in a minute. Then there are some policies
related to multiple principles including collaborative arrangements with
agencies which certainly impact arrangements with administrative agencies.
Finally, the administrative records policies and procedures, which is a
handbook for how we will manage and how we will establish agreements to use
administrative data, how we will manage and handle administrative data that we
gather.

So those are the four guiding policies which impact administrative records.

Now a little bit about policy on record linkage. This policy establishes
six principles for conducting Census Bureau projects that use record linkage
techniques.

The first is mission necessity. What that says is that the linkage must be
necessary and consistent with the Census Bureau’s legal authority and mission.

The second principle is best alternative. That principle says Census Bureau
will examine alternatives for meeting the project objectives and determine that
record linkage is the best alternative, given considerations of cost,
respondent burden, timeliness and data quality.

The third principle is public good determination. The Census Bureau will
weigh the public benefits to be gained from the information resulting from the
record linkage against any risk to the individual privacy that may be created
by the linkage, and determine the benefits clearly outweigh any risks.

Next is sensitivity. The Census Bureau will assess the public perception
that the level of risk to the individual privacy to the particular linkage and
create an appropriate level of review and tracking.

Openness. The Census Bureau will communicate with the public about its
record linkage activities, how they are conducted and the purposes and benefits
derived from them.

Finally, consistent review and tracking. Record linkage activities will
undergo a consistent review process using the criteria set forth in its policy
and a centralized tracking by the Census Bureau.

Now, the policy establishes a checklist of questions that have to be asked
related to each of these principles, and risk points are assigned. If the risk
points are high enough, it is considered a highly sensitive record linkage and
it needs specific approval from our data stewardship executive policy
committee. So there is a thought process that goes into this, and an assessment
is made that yes, this is a high, moderate or low sensitive project.

There are controls that we have established that support our administrative
record uses. Sally has mentioned this, but I think this comes under her
purview, centralized data acquisition and agreements. We have one focal point
within the Census Bureau for establishing the contacts with the administrative
agency to acquire data and to establish agreements. We also have a centralized
review process, so that all projects using administrative records go through a
formal review to insure that they are compliant with the legal requirements and
the agreement between the agencies. We require need to know access. Only those
people who need to have access to this information are permitted access. We
remove identifiable information immediately, and replace it with what we call a
pick.

We have an administrative records tracking system, which is a computerized
system, which basically allows us to control a project from its inception to
completion, to insure that anyone who works with that project understands what
the rules are, how that information can be used.

We have file receipt logs and audit trails to insure that we are complying
with the agreements. When our data have to be used off site, we do independent
site reviews to make sure that the security is required.

Finally, we do security and confidentiality training. All of our employees
on an annual basis must take confidentiality training under Title 13, and
training specific to Title 26, which is tax data, and also IT security
training. So this is a mandatory annual training.

Let me talk just a minute now about some unique privacy and confidentiality
concerns related to our administrative records use. A lot of this involves
perception as much as it does reality, I think. That is why these issues get
very complicated. We have to be sure that what we are doing is perceived to be
the right thing to do.

The first concern is, is the consent needed for the statistical use of
administrative records through the data provider, through the survey collector,
and what are the conditions for that consent. Is it an opt in or opt out, does
it matter whether it is voluntary or mandatory, whether the survey is voluntary
or mandatory. These are important decisions that we have to make, to make sure
that it is transparent as to what we are doing and the permissions that are
being given to do this.

Another unique issue involves, will the public accept a near real-time
system to respond to immediate statistical needs. Past uses of administrative
records are quite different than they are today with the new systems that we
have developed. The data that we have had before gets old very quickly. We have
had matched systems from data that is five years old. Today that becomes much
more current information, so that raises additional concerns for how that
information could be used, not how it will be used, but how it could be used.

The next is, does the public trust the protections around the interagency
data sharing. Today’s climate is different than it was prior to 9/11, in terms
of acceptance of record linkage and data sharing activities among government
agencies.

Next is how can we be more transparent about record linkage activities.
Events in Canada several years ago led us to conclude that these things cannot
be done in secret. They have to be very, very publicly known, because the
public will react very negatively to information they find out about something
that is happening that wasn’t publicly acknowledged. So we have to be very
transparent about what we are doing.

Finally, how can we continue to meet the statistical needs while assuring
confidentiality. That is what I want to talk about next. It is critically
important that we make these links data accessible for research purposes. That
is where the value is. We have to determine our options.

As Ron mentioned, administrative records linked to survey data raises
unique confidentiality concerns because of the fact that that administrative
data exists somewhere else. So we are limited in the amount of public use micro
data sample files that we can publish. We don’t want to discount them, but we
have to understand that they are not going to be of as much value as they would
have been if they were just the survey data. So we have to look at other
options.

We have been assessing many other options for providing access. As Sally
mentioned, we have a network of research data centers, and we provide access to
qualified researchers for work supporting the Census Bureau’s programs through
these research data centers. So this provides an opportunity for those people
who are in geographic proximity to the regional data centers and have projects
that meet these criteria to access some of these linked data. We are looking at
options for streamlining and enhancing those centers.

We also are actively researching techniques like have been used for the
Luxembourg income study for many years, which allows users to develop programs
to run on the matched data. They don’t actually access the matched data, but
they can submit those programs, and they are run on the matched data and
products are released to them. So that is something NCHS has done a lot of work
on too, and the Census Bureau is continuing its research.

We are also researching the development of synthetic data John Abod, who is
currently working with the Census Bureau, has done quite a bit of research on
synthetic data which is model data, so that we may be able to release linked
data that are not the actual data, but maintain much of the properties of the
original data.

Finally and probably most important, we have to continue the dialogue we
have established with researchers and program evaluators and program
implementers about what their needs are and how best we can meet those needs
under the constraints that we have to live with.

So in conclusion, what I wanted to say today is that the Census Bureau is
committed to meeting the needs of its customers, by enhancing the reuse of
statistical data through our use of administrative records and survey and
census data sets. We are also committed to reducing the cost and respondent
burden of developing statistics, and insuring the trust of the public, our data
providers and our data users, and finally, continuing a public dialogue on the
advantages and cautions surrounding our use of administrative data.

With that, I want to thank you.

DR. STEINWACHS: Nancy and I and all of us want to thank all three of you
very, very much. This has been exciting. It opened my eyes to a lot of things I
didn’t know, but probably other people around the table did, and that is part
of the sharing.

Nancy and I discussed, maybe we could take a five-minute break to let
people have a human break, at least stand up and stretch, and come back and
start the dialogue, both questions, comments and interchange.

(Brief recess.)

DR. STEINWACHS: If we get started, we will get a chance to get a discussion
going and answer peoples’ questions.

Let’s get started. Let’s open it up to questions and comments, both from
those around the table and those in the audience. We have a microphone that we
can pass around.

MS. GENTZLER: I am Jenny Gentzler from Food and Nutrition Service at USDA.
I would like to hear some more discussion about the feasibility of getting food
stamp and TANIF data from umpty-ump state agencies, and sometimes even if it is
administered by the county agencies. I know that is a linchpin of the new dues
system. So I would really like to hear some more discussion about whether we
could have all of those matches done and have the data available before 2020.

Thank you.

DR. STEINWACHS: Who would like to take that?

MS. OBENSKI: Even though there are technical challenges with different
state data, our experience to date is that we can overcome the technical
challenges.

By far the single biggest hindrance or obstacle are all the legal
requirements around acquiring the data set. For example, in our child care
subsidy program, which is a pilot project that we are undertaking with
researchers from Texas, Illinois and Maryland, it took a year and a half. This
is working with researchers who have established relationships with the state
entities. This wasn’t the Census Bureau calling up cold.

So although there are technical data challenges, I think I can speak for my
analyst that those they can overcome, but the policy and legal obstacles or
challenges to make sure that all parties are protected is quite formidable.

MR. PREVOST: I just wanted to add, we were involved in a child care subsidy
conference last week, where a similar question came to us. I think if you look
at the way that data are submitted to CMS as an example, an agency would almost
have to have a centralized focus within the federal government that somehow
related to the regulations around a program and perhaps tied to the funding to
those states, to be able to collect or to require the collection of that
information in the standardized format in order for this to occur, because
otherwise it becomes voluntary. I guarantee you, out of the 51 or more entities
out there, somebody will not want to participate for one reason or another.

MR. LOCALIO: You gave us an interesting response. Let me just tell you what
happened last week. The full committee had a meeting. One of the issues had to
do with the liability — I guess it was NCHS — or the decreasing ability of
NCHS to get vital birth and death data from the states and local entities that
produce those data.

We had a speaker from New York City, Steve Schwartz. He called himself Dr.
No, because he tells so many people that they can’t get access to the data.
Then we had a presentation that indicated some of the difficulties that this
poses for people who previously got data but now they cannot on births and
deaths. Obviously this is very important for a researcher, who needs to know
how many people in the study have died. That is your outcome, survival.

So what people didn’t hear is what I asked him after we broke for lunch. I
introduced myself and I said, suppose Health and Human Services told New York
City that their further receipt of any Medicaid funding from the federal
government was contingent upon their cooperating with NCHS in providing the
data as they used to. His reaction was very defensive. He said, oh no, Tenth
Amendment would prohibit that because that has traditionally been a state and
local activity. Then I reminded him that in the early ‘70s, every state
had a 55 mile an hour speed limit because they were told 55 miles an hour or
you don’t get any highway funds.

So I would submit that a lot of the way around some of these obstacles
which are most unfortunate and not in the best interests of the country as a
whole is to do what I think you alluded to. That is, to suggest that more
cooperation will be forthcoming if these data systems are linked to certain
line items that provide appropriate incentives.

Now, the other thing that came up which I think was not nearly as
threatening as what I mentioned is to sponsor model state and local laws that
would be appropriate and consistent with the needs of everyone and have
uniformity, and to say, we have this model, this requires somebody to get out
there and think about it and write it, and that is not always easy. But if you
have model laws, it then becomes a little bit easier to convince state and
local entities, many of which have a lot of difficulty passing a budget every
year, that this is worthwhile if you do some of the work for them.

So I just wanted to relate that to people, that this is my conversation on
Thursday of last week. I am wondering if anything was said about me back in New
York City on Friday.

DR. STEINWACHS: Anyone representing New York City here?

DR. MADANS: Lucky for you, Steve was still in Hyattsville on Friday, and it
hadn’t gotten back to New York.

I just wanted to clarify a little bit about what is happening there. This
is really a data release issue that we are facing with vital statistics. There
are some items we are not getting, but that is a different issue.

NCHS is getting the information. What has changed is what and how we are
allowed to re-release it. It does feed directly into the issues that have been
brought up. I have a feeling some of the presentations you are going to hear
today and tomorrow will get boring, because if you take out Census and put in
NCHS, other than examples, the issues are all the same. But if you nod off, we
will understand.

This whole issue about what is safe to release and what are our
requirements, what are the various parties’ requirements, how do they mesh, is
one of the major issues. To the extent that something like a model law, or some
way that we can ease the way that we work together is really important.

Every time we start a different linkage project, it is as if no one has
ever linked data. We start from scratch, dealing with the same issues over and
over again. Nothing changes, again, substitute the different data sets. So
getting agreement on best practices, on what is reasonable to do, what is
acceptable to do jointly I think would make our lives very easy, because after
awhile this is just work. We are not getting anywhere. We are just redoing the
same thing over and over and over again.

DR. STEINWACHS: Let me ask a question that falls into one of the areas this
subcommittee has been dealing with, and that is trying to improve the reporting
of race and ethnicity. It sounded to me as if I just had to come to the Census
and I had both good reporting and filing in the missing blanks by statistical
means.

Let me just ask the question as part of my education. You have that data
well developed and are updating it and monitoring it. On the flip side of that,
there are people who analyze the Medicare claims data, administrative data, to
look at disparities in health care services. But mainly they have what Sally
was saying, they have white, black, other, because it dates back to the — the
source of the data dates backs through Social Security or original enrollment
in the Medicare program.

What would be involved if CMS came to you and said, could we get from you
an estimate of race and ethnicity that we could link to each of these people
and we could update that. I was trying to get a sense of how does that fall in
privacy and confidentiality. Does that raise issues? In a sense, they have it
but they don’t have the measure that we use today. They don’t have the
categories. But it would help me to get some sense of what the possibilities
are, maybe have a little discussion of something like that kind of reverse
request, where CMS was coming to you.

MR. GATES: Maybe I will start this discussion. I think the issues
associated with assigning race and ethnicity have to do with — if you are
going to do it on an individual level, how good it is. If you are assigning
race and ethnicity based on some models that are a hundred percent accurate,
then you have got confidentiality concerns, because of the fact that that
information may have been — where you are deriving that information from, if
it is derived from the Census, let’s say, that is protected, and you are
creating this model that is a perfect model, then you have got a real issue in
terms of, did you really comply with the confidentiality requirements. So that
would be a major concern.

So I think the issues associated with this have to do with perception
about, have you violated the confidentiality.

DR. STEINWACHS: Let me ask it another way. What if there were some auspices
under which you took Medicare data into your data centers and did a linkage?
What would be the issues if you went the other way?

MR. PREVOST: I think I can speak to that a little bit. Part of it is, that
is what we have done. The auspices under which that would work, we have our
research data centers, which would provide an environment that was protected,
where the individual, if they had a project that had been approved, could come
in, could work on that project and the data that were ensconced in that data
center, and then there would be a confidentiality filter, being people who man
the research data center, who would take a look at the results to see if those
results were presentable outside. By that, I mean that they would not be
disclosing any given individual’s information.

So we have worked under that. We have also worked under situations like in
our current areas, where we are jointly conducting this research with our
partners. We do confidentiality research to make sure we are not disclosing
anything, and then we provide it as a group.

MR. GIBSON: I apologize for interrupting. I’m just going to add some
technical details here. We have had the privilege of trying to work with Sally
and Ron before. My name is Dave Gibson, and I am representing Spike Duser, who
is out this week and he has asked me to speak in his behalf.

DR. STEINWACHS: Thank you.

DR. BREEN: CMS.

MR. GIBSON: I’m sorry, I’m from CMS, I apologize. We have tried to
establish some relationships with Census. It wasn’t for want of trying on Ron
and Sally’s part. We really appreciate their professional demeanor and
everything that they were trying to bring.

We have a lot of problems with administrative data. I’ll be talking about
that some this afternoon. But I would like to correct something, the fact that
you are saying that some of the data for the other race and ethnicity
categories are not broken out.

We have done a better job in terms of especially the Indian Health Service.
We have a relationship with them where they populate our data, and if you are a
member of a recognized tribe, that information will trump anything that is in
our database. We have had mailings out to people in that other category, who
are we think mostly Hispanic, and people who have Spanish and Hispanic
surnames. We have done mailings to them and tried to get information back to
populate the expansion of our race codes to go beyond just white, black and
other.

That was a constraint that as you know was based on the old SS-5, and data
that is coming in currently they are collecting it more, but we are collapsing
it back down in some cases, but in our enrollment databases we are trying to
keep that information there.

So there is some effort going on to try to improve the race and ethnicity
data. I think the thing that you have to worry more about is the fact that with
enumeration of SSNs at birth being done by the states, many of the states now
are refusing to give SSA the race and ethnicity of the child. So whereas most
of the Medicare population are people who are obviously aged for 85 percent,
there are disabled in that category, some of the ESRD especially could be
children. You are going to find a lot more of these unknowns creeping into the
data over time.

So I don’t know if that is helpful or not, but just little technical
details there.

DR. BREEN: Could I ask David a followup to that? I was wondering, the
techniques you said you were using working with the Indian Health Service,
which of course would be a really good source, but wouldn’t it be more cost
effective to work with the Census in order to get information from them rather
than to send out a mailing? That sounds like an expensive way to do it.

MR. GIBSON: I believe we did it a one-shot time. I don’t recall. I would
have to talk to folks in the enrolment area to see if they have had a followup
on that. But my awareness of it was only that it was done one time. Yes, it
probably would be more cost effective to try to do some sort of a — you are
talking about some sort of a statistical match?

DR. BREEN: Yes.

MR. GIBSON: That would probably be much more helpful, I agree.

DR. BREEN: And you would probably end up with better results.

MR. GIBSON: I would suspect we would.

MS. TUREK: Gerry, is this a Title 13 issue? If they were to do a match, can
you give them your race data from the Census to match into their files? An
exact match.

MR. GATES: Are you saying if we gave it to them?

MS. TUREK: To CMS; may you do that?

MR. GATES: No, we can’t. It would have to be done by us.

MS. TUREK: Then you couldn’t share the final data set with them. You
couldn’t give them the micro data, right?

MR. GATES: No, not the micro data. We could give them back tabular data,
but we could not give them back micro data.

DR. IAMS: If I may point out, I am from Social Security. Starting in the
mid-80s, the social security number has been issued at birth. We don’t have
race and ethnicity, so your black-white is disappearing from the data set. At
some point in time people will be on Medicare, and you won’t have anything.

DR. BREEN: Is it not possible to fill in the blanks and return the data?

DR. IAMS: I’m not sure what you are suggesting.

DR. BREEN: If there were missing information, to update that information,
or if there were incorrect information, to update that information at the
Census Bureau with the CMS data, and then return the data corrected.

MR. GATES: No, that would not be possible under our statute. We could not
disclose that personal information to CMS.

MR. PREVOST: While it would be technically possible and we have done it
within the constrains of the Census Bureau, delivering that data back to the
agency, we would not be capable of doing that.

MS. TUREK: Title 13; they don’t give that to anybody.

DR. STEINWACHS: Let’s continue. I want to get everyone in.

DR. STEUERLE: Continue this conversation. I don’t want to interrupt.

DR. STEINWACHS: Others who want to join in on this conversation?

DR. DAVERN: Mike Davern from the University of Minnesota. Would it be
feasible, and I know you guys have talked in the past about building an
imputation model, in which you could give CMS the information it needs to take
a best guess at what the race might be on their file, based solely on Medicare
information?

MR. PREVOST: Yes, it is certainly possible to do that. We would have to
conduct a wide variety of research to make sure that we weren’t in particular
cases re-identifying somebody.

DR. STEINWACHS: Other comments on this?

MR. J. SCANLON: Aside from the data acquisition issue, which will probably
be with us for a long time, you mentioned on the dissemination end you had
research data centers, you had other ways of releasing the data. It seems to me
that is probably an area to focus in as well.

Do you have any indication of to what extent research data centers are
utilized, and when you do talk to customers what kind of ideas do they give
you? Apparently you can’t use remote access for a number of reasons, but
someone can request certain tabulations or regressions or other things. You can
provide the product itself. But how much is it utilized? What sort of models
are you thinking of for the future, and how do you relate to the way other
agencies are approaching this?

DR. DAVERN: I think one of the issues on the research data centers is the
process involved in getting approval for the particular project, especially
with administrative records data. It has to go through an approval process with
the disclosure review board and all those types of things, so it takes a long
time to get approval for the research data centers.

I know there is a large interest in people using them, and there are a lot
of people who are using the research data centers. I don’t know what the
breakout is between those using administrative data and those just using some
of the other Title 13 data. I think there are two different ways to do it. But
the research data centers are used heavily. There are only 12, 13 research data
centers, and they are looking at expanding the access to those.

MR. J. SCANLON: Do you physically have to present yourself?

DR. DAVERN: You physically have to go to the research data center, correct.

DR. SONDIK: I am doing the wrong thing because I am reading ahead here.

DR. STEINWACHS: I think that is against procedural rules, isn’t it?

DR. SONDIK: It probably is, but I noticed this before, and given that you
just asked the question it can come up again. But in the talk that is going to
be given by Julia Lane, there is a slide that says research data centers
drawbacks low in declining utilization. It said, fewer than a hundred active
projects.

DR. STEINWACHS: Let’s call on Julia.

DR. SONDIK: I was struck by this. There is a judgment there that that is
low, but it struck me as — I don’t know what it means across 12.

Agenda Item: Using Linked Micro Data: Julia Lane

DR. LANE: My background is, I was at the Census Bureau for seven years, and
then I was at the National Science Foundation, in the economics program, which
continues to fund the research data centers.

That judgment is not mine, by the way. That is the judgment of Brad Jensen,
who used to be the director of the research data centers, and that is a direct
quote of his testimony to CENSTAT. NSF had very similar concerns, because NSF
funds it. When I was there, there was a lot of concern about the little usage
which last year was under a hundred projects spread over eight, count nine if
you think of the California one.

So the big issues that we have when we put it before review panels was
first of all, there was a delay in the review process, which could take up to a
year, which is for graduate students and so on. Physically having to go on site
to do the work is a big deterrent for research purposes, and certainly for
program agencies. Joan might want to speak to that.

So we at NSF came to the conclusion that it is very important to support it
as a niche approach, but it is certainly not going to be a bill-based response.
That is a little bit what I will be talking about this afternoon at Joan and
Gene’s request.

MS. TUREK: Let me give an example of very intensive data use. I manage the
transfer income model which is used to look at the impact of changing federal
programs. It has been around since the ‘60s, so it has probably got $70,
$80 million in it over all that time.

We use the CPS fortunately and not CEPR. We would be in real trouble. We
run the micro database through that model and assign people to different
buckets, depending upon their characteristics. We also can change the code in
that model daily when we are doing something for the Hill. At one time we had
an open line to one of the subcommittees and were doing runs while the
subcommittee was in process.

When they change the programs, we have to write new code. So for us to use
the matched data we would need to be able to have our model sitting out
somewhere where we could change code, do runs and get the results in real time.

I think that is going to require some major changes in law. I wouldn’t want
you to do that to any of my data sets that I am using until after — I would
want to keep what I have until you have got the rules in place that I could use
the new, or you are going to affect one of the major policy functions of our
office.

Do you want to say something about social security? Because you had SIPP
based models.

DR. IAMS: We continue to have SIPP based models. We are a data center that
follows the rules and regulations of Census and the Internal Revenue Service,
and treat it as if it is gold at Fort Knox. We are highly protective of our
security and privacy, and we do run these models. We have an ongoing
relationship on social security reform with the White House, that is receiving
it on a flow basis. But that is facilitated by having Social Security
Administration being a data center that permits us to use the restricted data.

MS. TUREK: Which we would not be, because we are a policy office.

DR. IAMS: Yes, you are a different agency.

DR. BREEN: Is there any access by the public or researchers to your data
center or that data that you are talking about, Howard?

DR. IAMS: Yes and no. The two main models we use with restricted data, one
is called MINT, modeling income in the near term, and the other is our SSI
financial eligibility model we call FEM. We have had researchers funded through
our retirement research center grant system use MINT data for various
descriptive projects. I think people from the Urban Institute have done that as
grants. But that is really a statistical adjustment to five SIPP panels, and I
am not sure how broadly based the Social Security Administration wants to make
that available. We get into an agency proprietary aspect.

DR. STEUERLE: Howard, just quickly, you and Tom Petska do at times — I’m
not saying it is done as much as you like, but you do also tend to have joint
projects. You have had external research combined with internal research.

DR. IAMS: Yes, we have had joint projects with external researchers, some
of whom we fund. Offhand I can’t think of anyone we don’t fund.

We have researchers with our retirement centers. We fund projects through a
cooperative agreement to three centers. The Social Security Administration
funds projects at National Bureau of Economic Research at the Michigan RRC and
at Boston College. We have had researchers come and use matched data, getting a
grant from us and using matched data at our center, and we have CBO who has
come and used matched data.

They are using the administrative data matched to the Census Bureau data. I
am speaking about that this afternoon. We have an array of administrative files
that are matched to it.

On a slightly different note, I have felt that the Bureau in the past could
have made more use of these matched data to improve the data quality of their
missing data and imputed data. Our data on being paid a social security
benefit, that is what Treasury sent them, and our data on an SSI check, that is
what we sent them. In the CPS or in the SIPP, when you have a matched record
and someone fails to provide an answer, you could provide a statistical
assignment from our administrative record that is going to be better than using
a hot deck from some other person, who may or may not be related.

We found in our studies that SSI gets misreported in CPS and in SIPP.
People call it social security, so you have more social security beneficiaries
in these surveys than we have as beneficiaries for matched people, because some
of those people are getting SSI and say it is social security. Then you have an
under count in SSI.

If you were to do that assignment, you probably could carry it further,
because with SSI usually comes Medicaid. So you could assign Medicaid based on
SSI. But there are a number of things.

I believe in the new economic well-being survey system, they are planning
to make more use of administrative records in improving the information
collected in the survey than has been done in the past. There is not a case
where you have to go get privacy from 51 states. We do have it, and we do have
an exchange and a series of MOUs and exchange data with Census for statistical
uses.

DR. W. SCANLON: I just had a question for clarification. What are the
restrictions, or what are the requirements for an agency to be a data center,
in terms of social security, qualifying for it to be one? Is it conceivable
that CMS could qualify to be one?

DR. IAMS: If I could be colloquial, we basically take your first-born
child. I am joshing, since this may be recorded.

There is a series of background checks that are required. There are a
series of sworn statements that have to be made. There is a training program
that IRS maintains that people have to go through. There is a training program
that Census maintains that has to go through. Then our data center walls off
the data; people can’t print from the data, they have to print to a printer,
and one of our employees goes and reviews the output that has been printed, and
you can’t take the data away.

Then to get to our data center, you have to go through two armed guards on
two different floors to get to it. Then you would have to know what you are
doing when you got to the computer, in terms of getting access to the file and
all the standard protections.

So there is a set of requirements that Treasury has laid out for being a
restricted data center that protects their data. You have to go through these
safeguards, and users have to go through these types of things. I don’t know if
Tom Petska tomorrow is planning on touching on this at all. But it really is a
very tight control, in terms of getting access and use. I think the Census
Bureau maintains the same type of thing at their RBCs.

DR. DAVERN: Let me say a little bit about that. We started the data center
network probably about 12 years ago through our Center for Economic Studies.
The process for identifying new research data centers is handled through an
arrangement between the Census Bureau and the National Science Foundation to
determine where the best place to locate the centers are, based on the research
community associated in that area, and how we best can establish a partnership
with an academic institution or whoever we decide to partner with to establish
a secure environment for accessing these data.

As Howard said, there is a very tight process for determining what projects
will be accepted through the research data centers. You should know that people
who access these data are considered — we call them special sworn status
persons, and they are given special sworn status because they are helping the
Census Bureau conduct its activities under Title 13. So we go through a process
of identifying how that project is going to support our programs by improving
our data, improving our knowledge about how the survey is functioning. So that
is all part of this process.

But there is a formal process for identifying new research data centers.
The arrangement that Howard is talking about between the Social Security
Administration and Census Bureau was established in 1967, way before we started
formally introducing research data centers. It was in recognition of a
collaboration that these two agencies had for better working together on these
projects that were of great interest to the Census Bureau in working on its
income programs and other programs.

So I think it is important to understand how this process has come to be
where it is.

MR. LOCALIO: I just want to make a couple of comments on data centers from
the perspective of a working statistician.

I am probably currently the statistician on 25 projects. I have meetings
every day, even when I am not in the office, sometimes by phone. To do an
analysis and to do it right sometimes takes years. It is impossible to get
funding out of anybody to finance staying in a hotel room for six months and
doing work.

So I would suspect that a lot of work that could be done is not getting
done, and I would suspect that a lot of the work that is being done is not very
good. I can tell you right now, putting on another hat, that is as an associate
editor for a journal where I act as a statistical reviewer, a lot of this stuff
is not very good and I send it back to the federal agency as well as other
academic institutions. I say, I has to go back and you have to redo this
analysis. So if you are working in a data center, you would have to make
another arrangement to go back to the data center and rerun everything and
rearrange your schedule.

So I can understand why there are a hundred current projects and it is
dwindling. That is no surprise to me. I am amazed there are a hundred active
projects.

If you think the data center or dozen of them is an answer to one of the
issues, it is just not. It won’t work. What it does is, it trivializes
statistical science to the fact that people are writing a couple of lines of
code in SASS, and then running it. You may be writing thousands of lines of
code in SASS or RS Plus or some of the other programs, thousands, and you are
rewriting them and rewriting them to get it right. These data sets are complex.
There is misclassified data and missing data.

See all this gray hair? This is not an easy life. It doesn’t pay very well,
either.

So I think we have to consider much more seriously about what types of
projects can be done at data centers and on what terms and conditions. It is
better than nothing, but if it does work, there has to be much more thought to
increased flexibility as well as adding to their numbers.

By the way, my views are not new. Marjorie, remember that e-mail I sent in
response to NCH’s data center request for comments? That was in the spring of
2005. It caused some laughter because I tend to try to kid around a little bit
when I am serious.

So those are the real issues when you are working with data. By the way,
the data sets also tend to be big, and everything is bad.

MS. OBENSKI: I’d like to partially respond. We know that that is a problem,
and we certainly can appreciate the fact that these data sets are
extraordinarily complex. I have watched the very best analysts run and rerun,
et cetera.

One of the models that ameliorates it, does not fix it, is for example in
the Chafin Hall project, which is very complicated, it is TANIF, UI wage, chalk
year subsidy and the American Community Survey. We are working very closely
with the researchers, but we are doing the data analyses and the runs and the
reruns and getting it right. What we are delivering to them in the RDC is a
fairly pristine data set that they should be able once they get access to do
pretty good research on without the runs and reruns. We intend to keep involved
with them to help expedite that.

Again, it is not a fix, but we believe that it should help things in the
RDC.

MR. PREVOST: I just wanted to add a little bit more to it. I think one of
the other things Howard said was important. What could be done is for edits and
imputes of the data to occur to the POMS micro data sets themselves, therefore
affording access to individuals as they would any other POMS file.

But as Gerry I’m sure can say, and I said in my speech earlier, this
certainly raises the disclosure issues from being able to reidentify people. I
think POMS data sets as a whole — as computer technology and everything else
increases, the ability to match back to am individual and to reidentify them is
growing. It is certainly a concern that we all have as statisticians.

So anyway, certainly one model in the short run could be doing this with
edited data in a POMS if it passed disclosure proofing. But then also to
provide the differences that we are seeing between the survey data and what is
being collected in the administrative data to enhance the modeling that is
being done by researchers for policy reasons. I think you need both prongs of
that in order to make that function work.

DR. STEINWACHS: We are going to need to wrap up here, so one comment, Gene,
and then we will close up and go to lunch.

MR. GIBSON: I am afraid in some ways, the changes that CMS is planning may
necessitate something along the lines of what we are talking about with a data
center, and whether we would qualify, I don’t know.

We have been used to giving out our files as flat files for both the claims
as well as the unloaded EDB or else the denominator file. We intend to go to an
integrated data repository, which is basically a relational database with
everything in it. There would be only one source of that data, and you would go
to that data, so there wouldn’t be these multiple copies of flat files that we
would be giving out as either an Epsidec or a SASS file out to our users.

So that has us as researchers inside concerned, because we have found that
when we go against relational databases like the NMUD which is used for our
claims — and even we don’t have access to them, but we have seen when you use
queries against them, it can take a long time to get a response back. That is
typically caused because the records are very wide, they are variable linked
packed decimals.

We would like to entertain the notion of shortening them, trimming them off
on the front and back and coming up with what I would call core research files,
which would be very, very deep, but very, very narrow. Trying to get our data
processing friends to buy into that is another story, but that would be another
possibility.

We could work with the groups who know how to handle data such as Census.
Where we are typically hurt is by the lack of meta data associated with the
instructions from our CWF friends in policy when they implement policy, or our
CWF friends in OIS, that is our data processing group, when they go out to our
standard systems and come up with procedures for physical areas, carriers and
providers. The meta data somehow gets lost, and it is not associated with the
claims.

So if there could be some way of working with agencies who know how to
handle meta data all the way from the policy to the systems to the manuals to
the actual claims and enrollment database, such as with the Census folks who
are SSA or NCHS, and then think about trimming off these humongous files for
the variable length packed decimal stuff that is not really needed, we could do
a lot, lot more in terms of being more responsive. A relational database, we
found, if it is relatively narrow you can get answers out of it fairly quickly.

DR. STEUERLE: I have had the fortune of being associated with a number of
agencies that are very interested in data from SOI to Social Security now to
HHS, and on numbers of occasions have made contact to Census.

It is probably unfair, but I would say that probably the view of the
statistical community, at least around Washington and maybe around the country,
is that Census is king. At times it ends up to be the lion king and other times
it ends up to be King Kong. It is just big, it has some really good people
doing wonderful things, but at times the nature of the agency is such that it
accidentally steps on people that it doesn’t want to, or knocks over buildings
by accident or something.

As an economist, I tend to think of this as very much based on — I ask
myself, what are the incentives of each agency. It was interesting in your
talks, because a couple of things stood out. One was, the word counting came up
numerous times in your analysis. Even when you listed the reasons you would do
things, I think the public good was listed third or fourth on a list. They
weren’t necessarily hierarchical lists, so maybe that is unfair, but it wasn’t
listed first, it was listed third or fourth.

I think the reason for that is, the Constitution gives you authority on
counting, and that gives you great sway if you go to Congress, because
Congressmen care about how many people live in their districts.

If you could justify things on the basis of counting, you can often go out
and do a number of things. The further you get from counting and the closer you
get to research, including research that might show policies as being
ineffective or policies as varying by states, and in some states being
effective and ineffective, the harder it is for you to get the easy
justification.

The other thing I sense, and this is not Census, I think part of my point
about making the king statement is, I think the reason people turn to Census is
just because you are larger than anybody else. They hope you are going to solve
the problem, even if it really is an IRS problem or a Social Security problem
or an HHS problem.

This question about public acquiescence and public buy-in. That is a tough
one. It reminds me of the story of when the private person asks to do something
that might create some harm, so you do something that might at some level
reveal somebody’s privacy. It is called a type one error. One person objects,
and somebody in the press plays it up, and it gets bad news.

The consensus of many agencies in government of course is, you don’t want
bad publicity. It is almost like that is the first incentive. The second
incentive is in some cases to serve the public.

However, there are times when that is overcome. That is when type two error
becomes publicly visible enough. The case that you gave was real time analysis.
All of a sudden there is a hurricane, we have — really, let’s be honest, we
have not done a good job on integrating data sets. We did not adequately serve
the public. Again, not to blame it on FEMA. I don’t even know who would be
responsible. But all of a sudden, the type two error gets large enough that
people say, privacy concern is nothing when we couldn’t serve the population of
New Orleans.

This leads me to asking a last question. What do you see as ways of cutting
through the Catch-22s, the dilemmas that many people have pointed out here with
some specific examples, on where we could use a better integrated data set? By
the way, I don’t agree fully, Russell, that these data centers aren’t useful,
because I think it has been an attempt of these agencies to — it is the one
way they figured they could cut the ice. I can give some examples where I think
it has made a difference, even though for every one example I could give you
where it has worked, you could give me ten where it doesn’t.

But it is a step. So what we are encouraging you to do and what I hope you
will do is give us advice on what are some steps we could do. Do we need more
people working on state protocols? Do we need advocates within agencies who
would be research advocates, not having the advocacy coming from the outside,
but somebody inside saying this would serve the public good, let’s figure if
there is not a way to solve this problem and get around our 47 constraints, or
dealing with Tom Petska at IRS, or dealing with Russell.

I am really asking you, what do you see that we could advise policy makers
on how to deal with these issues? I have been involved with too many drafting
of laws, and I know what happens. It is what former press secretary Jim Brady
used to call the bog set method, a bunch of guys sitting around the table. One
says I think this looks good, now all of a sudden it is the law, but it is not
that it was written in stone or was even well analyzed in the way it was
drafted. Do we need people in agencies who work on how we can redraft the laws
to deal with these things? Do we need advocates for the public good? That is
the one thing we can use to give the type two error its importance.

It does seem to me that there are cases, and I don’t want to use real-time
analysis to be the only example, there are cases where we are dis-serving the
health of the public by not doing the things that as analysts or researchers we
know we can do, but as bureaucrats or law abiding citizens we know we can’t do.

DR. STEINWACHS: He asks an easy question, doesn’t he?

MR. GATES: That gave us a lot to think about. I would like to separate that
into two areas, if I could. One is the issue about whether we can do the
linkages to support this research. The other is, can we provide the data to the
researchers to make it happen.

I think in terms of whether we can do the linkages, there are a lot of
issues we can deal with, but the perception issues are the really critical
issues, about how do we convince people, convince everybody, that this is the
right thing to do. We should be linking these data for these purposes, because
we can do it in a very safe and controlled way.

I think we have to have more of a public discussion about that, about the
fact that these records for statistical purposes under these controlled
conditions are a good thing.

The other issue about whether we can provide these data for the researchers
is also a critical issue. It comes down to, as I said before, about how do
encourage the research and still maintain the trust that we have built up. We
did build up trust, we don’t want to lose that trust. We cannot lose the trust
of the public that gave us the information. Either they gave it to the
administrative agencies or they gave it to the statistical agencies.

So we have to figure out under current legal requirements whether or not we
need to modify the legal requirements. I don’t know the answer to that, but
that has got to be worked out in terms of how we can do it in a way that keeps
the information safe.

I think what we have tried to do, at least with these research centers and
moving towards other ways of permitting remote access or doing synthetic data
that may give micro data that is useful to researchers, are ways of
accommodating what we think are the best ways to get the data out to the
research community in a way that is safe.

There are probably other ways that we could consider too, but I think that
is really going to be the hard question, and it is a question that we have to
address.

I’m not sure what the right answer is, but the more we can discuss this,
and the more we can think about, is there something else besides those things
that gets those data that we need for the research, those linked data that we
need for the research out to the researchers. I don’t know.

DR. DAVERN: I think what we need is, we need researchers to be patient with
us, and work as hard as they can to get access to the research data centers to
use the data, and show that it makes the difference.

Our experience with the SIPP, when we came out and sid we are going to get
rid of the SIPP, we are not going to collect it, we are going to do a new
dynamic system, a lot of people out there, the research community agencies,
pointed to a lot of research that was used heavily and widely and said, we need
this data, these data are important.

We have a couple of projects out there in the research data centers that
are doing these linkages, are doing matches to evaluate that, and we need more
of those out there doing this to show results that get articles published or
articles used by agencies to show that these things are working, to show that
we need more of these research data centers or increased ability to access
these, and to work with us to get those out there and to keep working on that.

We have also contracted with CENSTAT to do a panel that looks at these
administrative data linkages, particularly as it relates to this new dynamic
system. One of the key questions are, how do you get these data out there to
the public to be used, and how do you deal with the privacy concerns, and are
there other ways like synthetic files or other types of things that can be
looked at.

So we are working with CENSTAT to look at these issues, but I think what we
need really are people who are willing to go through that process, no matter
how difficult it is, but that is the process we have, to show that this is an
important thing to look at.

MR. PREVOST: I just wanted to add on, I think there is a third part. The
third part comes beforehand, it is the prequel, if you wish. Gerry was talking
about getting the data and linking it, and then providing access to the
researchers.

But what I would submit is that there is a government business model out
there. We are all working to try to serve our clients, the general public, as
well as we can. The way that the agencies are set up right now is that we are
all working within these stovepipes of our own business models. I think some of
the things that have come out of E-Government have started to take a look at
generalized data systems and processes around the federal government.

One of the things that could be done is to come up with standardized
agreements and standardized processes. I don’t care if it is with Census or
not; if it is somebody sharing data with National Center for Health Statistics,
for example. Should have a streamlined process, so if you have a project that
is two years long, that you don’t spend 90 percent of your time trying to get
the data. That is the first part, and I think that is something that we could
deal with.

MS. PARKER: I am Jennifer Parker from the National Center for Health
Statistics. I just wanted to say something about the research data centers. I
went to three epidemiology conferences this year, and by and large nobody
wanted to use the research data centers. The graduate students in particular
said they didn’t have money, and the academics said they didn’t have the money
to place somebody in the centers.

If we want people to use them, we have to identify funding, because people
can’t write grants if they don’t know what they are going to do. So they need
preliminary exploratory analysis to get the job done. The graduate students,
they don’t have any money. So if this community wanted to promote that, then we
are going to have to come up with money for people to use them.

Thanks.

MS. OBENSKI: I would like to respond from a different perspective. I think
that there are big, big questions that are arising, because we are in a new day
here with these integrated data sets. We are addressing policy problems and
questions that we really haven’t had to address in the past.

But I think that if you talk about how do we get the word out there, I
think a project like Mike Davern’s project on the Medicaid under count study,
where we have working collaboratively with experts from multiple entities, all
coming at it from a different vantage point, but all trying to answer the same
question of solving this very complex question.

I think it is somewhat unique. As the director said, we met with ASPE and
our team last November. It is probably a unique experience to bring in these
other agency entities as part of the team. What it has done is, it has enabled
us to do a project that the Census Bureau never could have done as well on its
own.

So I think more of these, this model, is the way I believe it is going to
be in the future.

DR. STEINWACHS: Ed, you get the final word before lunch.

DR. SONDIK: Then it had better be brief. I think it is really interesting
that this discussion focused on the data centers and not what I thought it was
going to focus on, which was technical issues related to matching
administrative data to survey data or whatever. I think that is very important,
and we really should take note of that.

Second, I completely agree with Jennifer about the cost issue. We are
thinking about that in NCHS. There are several reasons for cost. One is
recovery and the other is regulating demand, if you will. So we are rethinking
that.

Third, we have a way of doing remote access with our data center. You are
going to talk about that. So I think that is really important. That is an issue
of trying to deal with Russell’s point, easing the access.

Gene’s point struck me that you are raising a very fundamental question. It
is this tradeoff between access, increasing access, and at the same time
preserving the confidentiality. The way we work it in this country, as we all
know, we have a set of different agencies that have different rules, but
basically they are pretty much the same. But there are alternatives.

The National Center for Education Statistics licenses the data. What has
always troubled me with the licensing, even though there are penalties for
violation of the license in some way, is what is the impact of that, what is
the cost of that to the individuals involved, to the agency, or whatever.

If we are going to rethink this, I think we have to think very
fundamentally about this, and understand what risk really is. I find it
fascinating. I would be surprised if any of the agencies had any quantitative
measure of risk. I know we don’t have a quantitative measure of risk. By that,
I mean what is the probability that we have a leak, whatever it is, or a
successful hack or hacking, whatever the term would be. I think if we are going
to rethink this, we have to think about that.

But the point about the need for all of this in time of an emergency
actually raises a very interesting question in terms of our preparation for
dealing with emergencies. I think that one of the points that have come out of
this is that we are not necessarily — we are not prepared for that to the
extent that we should be. It may very well be that we can prepare and have
maybe even special authorities that can be invoked in time of particular
emergencies, which again would bring this back to the legal side.

A lot of food for thought here.

DR. STEINWACHS: I very much want to thank Sally, Ron and Gerry. Thank you
for leading off. You have certainly gotten this discussion going. I hope you
may be able to stick around. I think it says very much that there is a lot of
common ground here and certainly common interest in trying to reach the same
ends.

In the information you have there is an identification of places to eat. I
think there are about four of them listed in the food court. Originally we had
you back by one. If you could come back by 1:10.

(The meeting recessed for lunch at 12:18 p.m., to reconvene at 1:10 p.m.)

A F T E R N
O O N S E S S I
O N (1:10 p.m.)

DR. STEINWACHS: We are very fortunate to have Julia Lane with us, even more
so because I understand she needs to rush off and catch a flight due to a loss
of a friend of family member; your uncle died. So we appreciate your fitting
this into not an easy time and an easy schedule.

Julia is a senior vice president at NORC, the National Opinion Research
Center. Julia is going to talk about using linked micro data. Julia, the floor
is yours.

Agenda Item: Using Linked Micro Data – Julia Lane

DR. LANE: Thanks very much. When Gene and Joan asked me to do this, I was
happy to be part of it. I think what we are doing is really important.

I spend a lot of time matching administrative and survey data with my
partner in crime there, Ron Prevost. Seven years of matching data from five
different agencies, SSA, IRS, Census, DOL and HHS, and then 50 states. So I
have the scars to show it; it took a very long time.

What I wanted to talk to you a little bit about, I am a health economist,
just to give you the benefits of what we saw in terms of putting linked micro
data together, the generic issues and then some of the challenges, and then
very much in light of the discussion we had before lunch, maybe some suggested
solutions.

Obviously much of the stuff that was discussed before lunch, clearly when
you have linked and administrative survey data, you can get much better
analysis of existing data. So for example when you are trying to explain
earnings and employment outcomes on individuals, when you have linked employee
data, instead of just being able to explain 30 percent of earnings in
employment variation, you are able to explain 85 to 90 percent. So the
explanatory power of existing data is far enhanced.

You can also do re-analyses that you thought you might not be able to do
before. When you put multiple sources together, the feasible set of research
increases. In my case for example, when we had information on firms, rather
than just the supply side of the labor market, we could look at the demand
side. In health, if you were interested in looking at just patients, you could
also look at health care providers and geographic information and so on. So
there is a rich new set of analyses that can be done.

The other thing in my time as a rotator at the National Science Foundation,
it turns out that the capacity to capture new sources of information is
increasing. We now have MRIs on individuals, we have got biomarkers. At NRC we
are doing work for NIA, which captures biomarker information on the NCHAP
survey. You can capture people in video and text. All of a sudden it starts to
be able to explain other aspects that you couldn’t explain from just admin or
even survey records per se, just the straight data that were captured. So for
example, it might be that increased earnings potential is due to excessive
testosterone or something, just hypothetically. At least, that is what my
husband tells me. That is what makes him more productive than me. That is why
he makes more.

This came up a little bit in the discussion before. This is simply good
government. Enormous amounts of energy and taxpayer’s dollars have been used to
collect these data. The more that you can leverage that investment in data
collection, the better off the taxpayer is.

Of course, all of that information that is collected — this is a Toles
cartoon that you might have seen — might cause some privacy concerns and
privacy issues. You may recall the stuff about medical ID chip.

With the data collection, and I thought Gerry and Sally and Ron did a very
nice job of describing what those challenges are, we really have a serious
challenge with providing access to the data. If you think about data utility as
being a function of not just the quality of data that are collected, but also
the number of people who access it and the number of times it is used, the big
challenge is not just to collect a fabulous data set, but to have people use it
and use it in the way for which it was intended, for which taxpayers are paying
those millions of dollars.

It used to be that statistical agencies handled this by producing public
use data. The problem is that the increased likelihood of reidentification of
public use data, together with an increased understanding of the consequences
of public use and the quality of analysis that is done, and I’ll give you an
example of that in just a minute, means that not only is the quality of public
use files declining, but the likelihood is that fewer public use data sets will
be released.

In particular when you think about public use data set for health issues,
the very skewness of the distribution means that the types of approaches that
are used to protect the data, in particular the top coding, will have serious
implications for the quality of analysis that is done.

So for example, I was trying to remember this morning, I should have looked
it up before I came, but I think either David Cutler or David Meltzer shows
that there is five percent of the population that is responsible for 80 percent
of the Medicare expenditures. So if you cut off the top spenders because you
are worried about top coding, you are not going to have much analytical
capacity associated with that.

We have heard a lot about synthetic data this morning. Synthetic data also
certainly has an enormous potential to help researchers. But one problem that
you have with synthetic data is obviously by its very nature, because it is
using a distribution

to synthetically impute values and replace existing values with imputed
values, it is going to get rid of outliers. If outliers are the ones that are
defining what is happening in the economy and what is happening with health
care expenditures, as in the example I just gave, you have really affected the
utility and the quality of the data.

These issues are particularly exacerbated with linked data. Clearly once
you add in admin records, they are wonderful because they add so much richness
to the data, but you immediately have a much increased use of reidentification.
The more information you get on people, the more likely you are to be able to
reidentify someone, and then you go to jail, which is not a very good thing.

Then the other thing, and this was also pointed out this morning, admin
records are often received from enforcement agencies. So as soon as you link in
the survey data which are protected, and you have got to protect the
confidentiality of the respondents, once you add in admin data, the enforcement
agencies retain those, and they can very easily reidentify the individual.

So that is a problem.

I wanted to take a little bit of a detour and just give you a sense of the
impact of top coding, something as simple as top coding on the quality of
analysis that is done. What I want you to think about is that the data utility
is going to be a function not only of the quality of the data, but also of the
number of researchers that are going to look at it.

Public use files have the advantage that lots of people have access, but
the quality of the data isn’t very good. So what happens here, here is an
example of an earnings regression, where earnings have been top coded. This is
on the current population survey. You want to look at the effect of the
black-white earnings differential over time. Because it was top coded at
different levels over time, the impact of the estimate on the returns on the
black-white earnings differential are quite different. The same thing in terms
of the returns to education.

I am not going to go through it in any detail, but basically the
black-white earnings differential, depending on how you correct for that top
coding, how you statistically correct for the top coding, you might say that
the black-white earnings differential is .35 log points or .63 log points. If
you correct in different ways, you might say that the change in the gap was .06
or .15. Those are huge differences, right?

So lots of policy makers to decide, is the racial earnings gap big or still
big, but twice the size? Is the earnings gap closing rapidly or is it closing
slowly? And because of the increased noise in the data, which is by definition
what they do to protect public use data files, because that biases coefficients
down, do you know whether a perceived vanishing of let’s say a racial
discrimination coefficient or a double on race, do you know whether that is
really the case in the underlying data or is it just that more noise has been
added all the time? The same thing happens when you look at the return to
education, which is obviously another policy area.

So those are two labor economics issues, but you can see how the same
issues would come up in health. So here we have a situation where lots of
people have access to data, but you don’t know what is happening to the quality
of the data over time, and you don’t know what is done to it. Typically the
agency can’t tell you what is done to it, because they don’t want people to
back out their disclosure protection techniques.

So one alternative approach is to say, let’s have really high quality data
at the Census Bureau or at research data centers. Then what happens is, people
can go into the research data centers and they get the good data, and they can
do the analysis on that.

I have to say, I am a big supporter of any access in general, so the line
that you picked up I think was a little bit taken out of context. But I wanted
to make it clear that there were issues with this approach as well.

For those of you who are not aware of it, researchers and many statistical
institutes throughout the world picked up this. The Census Bureau pioneered it.
Researchers physically go, they are monitored by employees. This is supported.
It is a very expensive program. I think the Census Bureau alone puts in about
three million a year for eight or nine sites.

There is an elaborate project approval process. By law all these projects
must provide a benefit to Census Bureau programs. They are required to have
special sworn status and there are penalties if they reidentify individuals.
There is all kinds of access constraints. I’m not going to go through those,
since they are in your handout.

As you eloquently pointed out, there are lots of problems associated with
that. You have got very high quality data, but the N of researchers that access
the data is very small. Furthermore, one of the issues that you run into is
that by and large at least at NSF, what we found is that the absolutely top
quality researchers would not go to the research data centers because they had
other things to do with their time. So the best researchers are doing analyses.

Danish data in Statistics Denmark, for example, in this particular case
they set up a very nice remote access system. As Brad said, this procedure is
expensive, very fragile and very tenuous. One of the issues is the link to the
review process. Ron Prevost and Sally mentioned the work that they are doing
with Bob Gerter at Tapinol. That has taken two and a half years, and they are
still not approved yet. A similar type thing that we had with NSF supported
researchers take a very long time to get through. Then it is expensive in terms
of time and in terms of money.

The panel was concerned because of disparate use, so you have got
well-endowed institutions who can afford to set up the research data centers,
but if you are a University of West Virginia or if you are Iowa State or in
Oklahoma, it is much more difficult for you to have access, so there was a
disparate impact. Then the concern was no remote access.

As a researcher I was concerned about this. My colleague within the Census
Bureau, Pat Doyle, was very, very concerned about the impact on the data
quality, the quality of the research and the quality of the policy analysis
that was done. Ironically she was focused very much on the SIPP.

At her memorial session at the ASA meetings, they asked me to write a paper
on it. What I wrote down was many of Pat’s ideas, except informed by some of my
time at the National Science Foundation. What I thought was, why don’t we —
given that we know there are all these problems, Pat and I co-edited a book in
2001 in which we said this is a real issue. It is not just for health, it is
for just about every area in social sciences. This is a major issue. How might
one go about thinking about how to structure this?

Let’s think about it in terms of learning from other disciplines. I got put
in charge of doing the cyber infrastructure initiative for social behavioral
and economic sciences, so I met lots of computer scientists and engineers and
so on. It turns out that obviously in other areas of NSF, for example in the
computer science area, there is this whole research funding associated with
cyber trust. That is funded very much by agencies like DARPA.

What they are worried about is setting up secure computer access for
extraordinarily sensitive information. Let me give you an example of how that
is applied. When you are Joint Chiefs of Staff for example at DoD, you need to
think about troop movements in Iraq. They don’t go to a research data center to
look at the data. They log online with tight protocols to access the data. They
have gone through training and so on, but the cyber trust initiative has
invested substantially on setting up secure protocols whereby the Joint Chiefs
of Staff for example can access troop movements online.

I should say that the cyber trust doesn’t just look at the physical
computer security. Any computer scientist, everyone I talked to at NSF, would
say anyone can hack into a system. So what they also tried to do was to set up
human protocols as well, so figure out adoptable protocols. You are going to be
surprised to hear that computer scientists worry about this, but they learned
— you know when they tell you to make up a password, they say you have to use
some exclamation marks and some numbers and some capital letters and some small
case letters. So you are making up this completely unmemorizable password. So
you write it down and you stick it to your computer.

So thinking out the humanly adoptable protocols as well as the computer
science protocols is a very large portion of what they are worrying about, and
they are funding that. Joan Feickenbaum, who many of you might know at Yale,
has been heavily involved in the Portia project. There are lots of commercial
applications. Financial services, very sensitive financial information gets
accessed through the web. People don’t physically go on site to look at very,
very sensitive financial information. They have figured out how to solve these
problems.

So thinking about, can we think a little bit out of the box here and think
of rather than just one magic bullet, let’s think about a portfolio approach.
Let’s think about setting up a set of computer protections. At DoD for example
there are actually three levels of computer protections. The cyber trust people
can tell you more about that. Try and minimize the amount of statistical
protection, take off obvious identifiers, but try not to much around with the
quality of the data too much unless you can document what the impact is on the
quality of the analysis.

This is going to work differently for different agencies. Particularly with
the Census Bureau we have got Title 13, public use or not public use. But you
can have degrees of statistical protection, the law says within a reasonable
doubt. So you adjust the statistical protection by adjusting the screening
which you have for people coming in to use the data.

Kathy Wolman asked me to go and talk to the Conference of European
Statisticians a couple of years ago because they were worried about
confidentiality protection. So I did the opening speech. There were about a
hundred chief statisticians from around the world, and over and over again they
said, we will give the data to people we will trust. In other words, there is a
very big difference between giving a lot of people access, the great unwashed,
to giving the access to a subset of people who have gone through a screening
process, who maybe have an institutional bond and so on. So think about putting
those sets of requirements together.

Having been and still am a researcher, instead of assuming that researchers
know how to protect data, actually go through an intensive training class.
Thinking back on how I treated data before I went to the Census Bureau, I don’t
want to tell you about it because I know this is being taped. I never did
anything wrong. But when you go to the Census Bureau and you work with
statistical agencies, there is very clearly a cultural confidentiality that
gets inculcated in you. So figuring out ways to train researchers so that they
understand that this is part of the mandate.

In other words, think about setting this portfolio approach up. What you
might have is multiple access modalities. You might have public use files and
synthetic data so you can get all your code run and muck around a little bit.
Then if you work through the remote access in a subset of cases, and then if
you wanted to go on site and work from the research data centers, go on site
and do that.

Instead of thinking about the silver bullet, one approach, think about an
integrated approach, legal, statistical, operational and educational, and think
about as we were saying having a consortium of agencies as Ron and Sally and
Gerry said. This shouldn’t have to be invented de novo by each agency.

You might think about developing a set of legal options, a set of
statistical options, a set of operational and a set of educational options, and
then different agencies and different studies within agencies. For example, the
CPS within the Census Bureau will have a different set of rules in the annual
survey of manufacturers, which is business data. So you might have different
sets of options that are chosen.

You could think about how the remote access might work. Instead of having
the event where you physically have to go on site, use 21st century technology
and think about encrypted connections and smart cards. You can restrict user
access from specified predefined IP addresses. You can have something like
Citrix technology, which is becoming increasingly used, or you could have your
own developed approaches.

For statistical protection you might remove obvious identifiers. You limit
access to the data that is approved and you could have the statistical
techniques chosen by the agency.

I strongly believe in training researchers. I think it makes a big
difference. The Title 13-Title 26 IT security training that they do at the
Census Bureau is very good, but that is web based. You would probably need
maybe a two-day training class before people are allowed to access the data,
and then maybe refresher things. But just basic principles of confidentiality,
why is this done, basic information about the layers governing the agency and
why those layers work.

So I really do think that — it has been five years since Pat and I edited
that book on disclosure protection by statistical agencies. I don’t see that
there has been much new that has been generated since then. Yet the pressures
are increasing. The pressures for high quality analytical work, the enormous
promise of matching administrative and survey data is there. We have the
potential to do what I think is needed to be able to understand what is going
on in different parts of the economy. I would suggest that we might want to
think about using both the cyber trust and human cyber infrastructure. The San
Diego Super Computer Center is doing some work, we are doing some work, and so
on.

So now I will shut up.

DR. STEINWACHS: Julia, thank you very much. I think you may have at most
five minutes to answer any questions or comments. So do people have questions
or comments for Julia?

DR. BREEN: You mentioned the portfolio approach. By that, do you mean an
array of different kinds of things that amongst them would provide protection
for the data and appropriate training and all of that stuff. It is a concept,
isn’t it?

DR. LANE: Yes. Because public use files did so well for so long, we have
been very focused on statistical protection for public use type data sets. What
I am saying is that the problem of public use all by itself is that you have to
protect against any onslaught. So what you need to do is to think about a
portfolio where you don’t just use statistical protections, you also think
about limiting access to authorized researchers setting up computer security
protocols like Statistics Denmark has done very, very successfully, so that
people can’t hack into the data, and you limit their ability to re-identify the
data by matching to information that is on the Internet, and legal protocols so
that if they do go ahead and try to reidentify, you have got some legal
framework within which to prosecute them. There is an institutionally binding
agreement.

DR. STEUERLE: Julia, thanks again for doing this. You are one of the many
stars we have here, and I really appreciate your coming.

I have two questions that are related to the ones I asked earlier. In your
experience at Census, are there really advocates within agencies for trying to
promote the research? I’m not saying researchers that want to do it, but I’m
thinking in the legal side, advocates who take the public good interest?

It seems to me that when you get down to the ultimate legal question, even
when I go through your matrix, somewhere, sometime, somehow, there is somebody
who could be identified. If I tell you eight characteristics down to the letter
that are observation in my data file and you can match that up and find the
ninth one, there is probably no circumstance at which at some level somebody
couldn’t be identified.

So the incentive of the legal community, if you are given the question of,
should you allow this to happen, is almost always going to be against trying to
figure out how to provide access, no matter to whom, even though we know that
at another level, a government worker could lose a veteran’s file. So the
chance of something happening is not necessarily related to people doing
research. So I am just curious, how to get around this.

DR. LANE: In most agencies it says reasonable means. So the big question is
whether a portfolio approach would match anyone’s definition of reasonable
means, and I would argue that it would.

I am not a statistician, we have got Tom Petska here who knows far more
than I, but my reaction would be, the law is reasonably clear as to what the
purposes are for which you can use the data. So you have to fit within each
agency’s mandate. It has to be authorized purpose. Without that you are
breaking the law, so you don’t want to go to jail.

The question is the degree to which — the interpretation that is put on
that. You raised the very good point earlier that a very narrow approach would
just be, you can only count. A broader approach is to — and if you take a look
at the IRS Census criteria agreement, there are nine categories, and some of
them are quite broadly written, under which you can do research as long as its
predominant purpose is to improve economic and demographic census and
population estimates.

So with a broad view, I think a lot of work can be done, but you still have
to find that authorized purpose. Is that a fair statement, Tom?

DR. STEINWACHS: My big job is time watcher, so I worry about you and the
plane. Do you think this is a good time?

DR. LANE: I think I had better run.

DR. STEINWACHS: Again, thank you very much.

DR. LANE: Thank you very much for inviting me.

DR. STEINWACHS: I am going to ask Gene Steuerle to introduce the next
panel.

DR. STEUERLE: We have a stellar lineup from our own Health and Human
Services agency, with whom we work closely in the National Committee on Vital
and Health Statistics. I have to confess, I know a couple of you but I don’t
know all of you, so I am going to be reading your titles, I apologize.

Jennifer Madans, Associate Director for Science. David Gibson of CMS. Steve
Cohen, the Director of the Center for Financing Access and Cost Trends at AHRQ.
This is going to be our first panel. Our second panel, depending on how long we
take to get to the presentations, is going to be Martin Brown, Chief of the
Health Services and Economics Branch at NCI; Gerald Riley is a social science
research analyst at CMS. John Drabek is an economist with the Office of the
Assistant Secretary for Planning and Evaluation, DHHS.

We are going to start with Jennifer and David and Christine.

Agenda Item: Health and Human Services Agencies

Jennifer Madans, Associate Director for Science,
National Center for Health Statistics

DR. MADANS: Well start with me. I am going to start this and then hand it
over to Chris who runs a linkage unit, and then pick up at the end on some of
the policy issues.

What we are going to do is just briefly go over the NCHS data linkage
programs, past, present and future, look at our current access procedures, some
of the challenges that we see in conducting linkages now and in the future, and
maybe some ideas about how to solve them.

Obviously we are doing record linkage the same reason everyone else is
doing record linkage. It certainly increases the accuracy and the detail of the
data that we can collect.

We generally use it to augment the information that we collect on our major
surveys. One of the early reasons for doing the linkages was to longitudinalize
what are really cross-sectional surveys. By doing the linkages we can follow
people over time without spending the money to recontact them. So it reduces
the burden on the respondent, it is cheaper, and it should get us better
information. So I think we are all doing it for basically the same very good
reasons.

Just to give you an idea of some of the potential that has come out of our
record linkages, this is just a handful of examples, a lot of them having to do
with linking our population based surveys with morbidity and mortality outcome.
We can talk about some of these databases that these studies are based on.

We do two types of record linkage. I think most of the time we are talking
about the first type here, and that is what we are going to focus on for this
talk, but I did want to mention the second one very briefly. This is where we
are linking at the person level or in some of our provider based surveys at the
facility level. So we find some external data source that generally is a census
of some kind, whether it is the Census, which we don’t link to, or all of the
CMS records or all of the information from the American Hospital Association,
where we can do a one to one link with our unit record on our survey, which is
the person or the facility, with a record or set of records on the external
file. We have done this at the person level and as I say at the facility level.

But a lot of our linkages deal with contextual data. All of our surveys are
geocoded, and we do a large amount of linkage of geographic data from a variety
of sources with these population based studies and also the facility based
studies. So this is where we might use census tract characteristics from the
Census. We can link those into the population based surveys. We have data from
EPA. We get information at the state level on things like Medicaid eligibility
requirements. So it is not at the individual level, but it is more the
contextual or geographic level.

A lot of the same issues in terms of access and identifiability that you
have on the first kind of linkage also apply to the second kind of linkage, but
the examples we give for today will be at the person level.

Now I’ll turn it over to Chris.

Agenda Item: Christine Cox

DR. COX: The NCHS record linkage program got its start back in the early
1980s, which is about when Jennifer and I arrived at NCHS, if that is okay to
say. We had both just graduated from college.

We began bringing together survey data with health care utilization data by
linking the national medical care utilization expenditure survey to Medicare
MADARS data, the Medicare Automated Data Retrieval System, for those of you who
have been around for a very long time as well. We did the same thing with the
epidemiologic followup study to the first national health and nutrition
examination survey.

We were pretty much pioneers in linking to the national death index with
the NIFA survey. In fact that survey had respondent verification through either
proxy interviews or death certification collections, so it has become what we
call at NCHS a truth data set, and we use it to develop our probabilistic
record linkage algorithms.

We expanded the record linkage program in the 1990s to include a routinized
approach to linking the national health interview survey to the national death
index to correct mortality data, to allow for as Jennifer said the longitudinal
study of our cross-sectional data.

Then around 2000 NCHS developed an organization — reorganized and created
a data linkage unit where these services were centralized and systematized, so
we could gain from the expertise in that unit and conduct most of our record
linkages.

We were very productive. As of the year 2006 we have linked a variety of
our large national public health surveys to three focus areas, mortality data
collected from the national death index, Medicare, CMS data, and that would be
denominator and the standard analytic file data covering the period 1991
through 2000, and retirement and disability benefits data from the Social
Security Administration. That actually goes from 1962 through 2003. So it takes
data back before the survey contact and extends the period of followup beyond
the survey.

The surveys we were able to link to, for those of you who aren’t speaking
NCHS acronym today, that is the national health interview survey, the
longitudinal study of aging, the first, second and third national health and
nutrition examination surveys and the 1985 national nursing home survey, which
is a facility survey.

We have completed other data linkages. We link infant death records to the
birth certificates to allow for study of birth characteristics of infants who
die within the first year of life. We link information from the American
Hospital Association to annual survey of hospitals on facility characteristics
to the national hospital discharge survey.

We are trying to develop an ongoing program. These agreements we think we
have mostly standardized, if such a thing exists. It is pretty easy to
standardize an agreement to match with the national death index outside NCHS.
We take it easy on each other, we only filed about 30 forms back and forth to
share data within the organization. But we intend to keep linking the HIS from
1986 through our current year, which is about year 2005 available at the
moment, to NDI data. Our HANES surveys are now yearly surveys, so we will
continue linking them, and our national nursing home surveys will be linked to
mortality information. That is cause of death and date of death data.

We also will continue hopefully linking to Medicare enrollment and
utilization data from CMS. Right now we do have the HIS 1994 through 1998 link
to that data. We hope to extend that through more current years of HIS, and the
same for the HANES data.

We would like to start adding the national nursing home surveys to that
agreement. And of course we will continue to link birth and infant death files.

We plan lots of interesting linkages. I was congratulating Ron and Sally
earlier; I think we are about to have a one-year anniversary on the negotiation
of an agreement to add HIS to the CMS Medicaid project that they discussed
earlier. We are nearing that one-year mark, on trying to get that agreement
finalized between our two agencies to share that data, so we are very excited.
We think it will probably happen this year.

We also intend to be linking the 2004 national nursing home survey to the
CMS minimum data set, and that will allow us to pick up characteristics of the
facilities, the residents they are staying in, as well as characteristics of
the residents in that facility. We hope to be linking the current NHANES to
food assistance programs such as food stamps and other food assistance
programs. We’ll see.

We don’t only do record linkage. We also spend a fair amount of resources
developing user tools, documentation, methodologic reports, where we describe
how the linkage took place and what kind of probabilistic matching algorithms
we use. We conduct bias analyses so that when researchers use this data they
understand what the limitations are and where we are not matching people and
what kind of people we won’t have linked data for. We also continually evaluate
and try and improve our record matching algorithms to make them just a little
tighter and reduce type one and type two errors.

I’m turning it back over to Jennifer.

DR. MADANS: As you can tell, there are no problems with data access for a
linked file.

I’m sorry Julia is not here, because we were having a conversation in the
hall about this remote access system. I’ll talk about ours just very briefly.
We have the same kind of confidentiality concerns that the folks in the Census
Bureau discussed this morning. As everyone else has mentioned, our general
files are becoming harder and harder to release through public access because
of not only the kind of information we collect, but the kind of information
everybody else is putting out on the web. So we are having a reduction in what
we can put out as public use. But the linked files are particularly vulnerable
and we worry about them more than we worry about many of the other files.

We try to put out as much as we can on public use. There is the problem of
reidentification by the people who gave us the data, which is an added problem,
but we know that there will be files that are not appropriate for public use.
This is the kind of thing where we make an announcement, the data are on the
web. We have no restrictions. There is no control, there is nothing.

We also have a data center. There are three means of access in our data
center. We only have one data center in Hyattsville, so anything that you can
say about the nine that the Census Bureau has, it is much worse when you only
have one.

I will report, I can announce that after many years of negotiation we have
reached an agreement with the Census Bureau to as a pilot project allow access
to our data in their data centers. So we feel like we have expanded access nine
times over what we had before so it is very discouraging to hear that nobody
likes their nine any more than they like our one. But it is still better. I
agree that there are problems with data centers, but they aren’t going to go
away, and I think we have to make them work better.

Our data center like the Census data center does have on site access. We
have a nice little room in NCHS. You have to go through two guards as well, and
it has all the other operational characteristics that the Census Bureau’s have.

We also started a remote system when we started our data center in
‘98. This I would call a first generation remote system. It is automated,
so it is remote and automated. It is called ANDRE, and I never remember what
that stands for. You submit programs through e-mail. Those are collected by the
system, the system evaluates them for illegal operations. You can’t do things
like list cases, and it won’t run them if it sees something illegal. It puts
them through a batch processing. The output is again looked at in an automated
way to see if there is any disclosure risk for what is sent back, and if there
is none it sends the output to the person requesting it.

A lot of the things Julia was talking about in terms of the front end are
very similar. You have to be a registered user. We will only send it to certain
e-mail addresses, all of those things. For our system we want to get to the
second generation, have a web-based system. We have to worry about the
firewalls and all of those things.

What she didn’t talk about, and I was curious to hear her talk about, is,
for us the real challenge is not the front end of that system, it is the back
end of that system. It is how do you do the appropriate kind of disclosure in
that kind of environment. A lot more control when somebody is sitting in our
facility. We have a lot less control when they are across the country. The
response of most of the system is that they let you do less remotely than they
will let you do if you are actually sitting there.

One does have to worry about things, like how do you deal with multiple
submissions of programs, where each one is fine but together they are not. You
have some of this when you are on site as well, but it is not as bad as when it
is remote. If I can do my commercial here, I’m not sure these problems are
insurmountable, but I am sure that we haven’t put the kind of resources that we
need to into solving them.

My other favorite soapbox is that we are reaching a point where we spend a
lot of money on data collection, but we missed a balance in terms of resources
for data collection versus data dissemination. The questions that we are being
faced with now in terms of dissemination are going to require some more funding
than we have been giving them in the past, and unfortunately if you have a flat
budget, if you are lucky enough to have a flat budget, if you are spending more
on dissemination you are not collecting as much data as you were before.

I think our users have to understand there are tradeoffs here. We can make
it easier to get access, but there will be less to have access to. That is a
fact. The question is, what do you cut, and that leads to many, many
conversations.

Let me also say about data centers, I think all of us who are running these
data centers do understand some of the frustrations that our users are facing,
and are trying to solve some of those, especially the front-end stuff, getting
proposals approved, facilitating peoples’ access as they wander the maze of the
data center.

I sometimes say we are going to get a Walmart greeter. Everyone will be
assigned a person whose job it is to make their stay in our data center a happy
one. I think we are all moving towards that. It will solve some of the
problems. It certainly won’t solve all of them. You still have to come to
Hyattsville or one of the other seven data centers. So I personally think a
remote system is going to be more beneficial, but it is going to be a lot more
expensive to develop.

We also have staff in our RDC that can provide on-site programming and help
people out, but I know most researchers don’t like that, like that the least,
because they want to get down with the data, they want to be able to play with
it and see what is happening. But it is an option. So I think we are beginning
to get that portfolio that Julia was talking about, different ways of accessing
data.

I think the main theme we are worrying about is the disclosure of view, and
how do we do that in a way where — to answer your question, Gene, I think
there is an issue of due diligence. I think if you force any of us into a
corner, we will agree that we can’t hundred percent protect, although that is
what we try to do, but when push comes to shove, we have to be comfortable that
we did everything that we could, and that an outside person looking at what was
done to try to protect the data will say that we exercise that due diligence.
Obviously it is a judgment call, but I think that is where we are comfortable
in saying that we have done what we can.

So where are the challenges? These have also been mentioned. First we have
to get the informed consent. We have to get the informed consent from the
person who is giving us the data. So we have to meet their needs, and we have
to do that in a way that will satisfy NCHS’ institutional requirements for
permission to link, what do we need to get from the person so that we can go to
our IRB and make a case that we should be able to do this linkage. We need to
satisfy the provider’s institutional requirements for permission to link, and
we have to be able to get that informed consent from the — in our case, from
the subject whether it is a person or a facility, where we on the one hand can
tell them about the importance, also be fair about what the risks are and how
we are going to protect them.

So those are hard things to do. One of our concerns are to get a more
systematized way of doing that in a way that will be acceptable to the
community. Up until now we have all been doing this on our own. There is not as
much collaboration among the agencies, both the providers and the receivers of
these administrative data as there could be. Every time we start a new linkage,
it is as if we have never done a link before. We start out from square one; do
you have the right permission, is it the right permission for us. To get the
lawyers involved is always a bad thing. So we really need a better way of
working this out.

This just shows you what is happening to our ability to get adults to give
us their social security number, similar to Census Bureau. If we got the
number, we took that as approval to link, and it is going down. There was a
change in how we did this in 2001, but even at 41 percent which is the better
number, that is not enough to have a viable data set. So using this as a
mechanism for getting permission isn’t going to get us very far. We are also
thinking of asking a more explicit question about linkage, because most people
don’t want to give you the social security number when they don’t want to link.
But how one does that not only so that we will get the number, but that we are
doing it in a way that is protective of the subject and acknowledges their
rights not to be linked. These surveys are not mandatory, so we have to provide
the true informed consent and get their permission.

As has been mentioned many times, the institutional requirements among
those of us who are receiving and providing data are very complex. We have some
differences in our privacy requirements and how our privacy requirements are
interpreted, the legislative mandates. Two years, we don’t see that as odd. It
takes a long time to develop these, but once you develop them you can extend
them. But it would make life a lot easier for everyone if this was done in a
more collaborative way.

Again, the balancing of the resources. These linked files are fairly
expensive to produce, to provide the right user documentation. They often don’t
come in a way that is easy to analyze. You have to transform them into
analytical data as opposed to administrative data. You have to tell people what
is good and bad about them and what mistakes they can make. We sometimes don’t
have the right expertise to do that, so we have to work closely with the
provider. Then what kind of resources are appropriate to put into assisting
users with these kinds of data versus other things we are doing, like
collecting new data.

We try again to create the public files from the linked data. It is very
difficult. Once you create a file, when we get data for example from CMS, and
if we create a file from that data that CMS can’t take and reidentify, there is
not much left on that file, but there are some things, and we need input from
users on what that something should be. We are making decisions about what to
take off and what to leave on and how do we satisfy the most use.

If you can get someone for a public use file, that may go a long way to
making their stay in the data center much more pleasant, because they have
experience with the file. We also use sworn agent status, and we are trying to
think of ways that we can use that authority for different kinds of data to use
different kinds of access mechanisms. We are also looking at whether or not we
can use perturbation on these linked files. I think we also looking at trying
to get synthetic files.

So again, we really want to share the knowledge and experience across
agencies so that we are not reinventing the wheel, and we can learn from each
other’s experiences.

The standardization should increase efficiency. We may be able to do it
faster, it will be cheaper. It might help us do the user documentation if the
providers are thinking about possible other uses for their data. We saw big
changes in the CMS data when we first started linking back in the ‘80s. I
remember, we couldn’t figure out how to read the file, the first one we got, to
now when things are much, much easier. And again, the development of standards
and best practices for linking, for data handling and how to get the extracts
and the documentation.

We would like to increase collaboration and communication among agencies. I
think this has to be done, maybe under OMB, I don’t know, but there has to be
some way of allowing us to work together without violating certain ways of
interpreting our authorizing legislation. We have to find safe havens where
certain

things can be shared in the development.

We want to develop more linkage projects. If we see a file that is Census,
we are going to try to incorporate it into our data systems. I think if we can
expand the access to RDCs through the development of new disclosure
methodologies so that we are more comfortable in using remote access systems,
we have more control over being able to evaluate whether something is a risk, I
think that would go a long way into improving user access.

Thank you.

DR. STEINWACHS: One way might be to have a few questions now or comments,
and — it’s up to you, Gene.

DR. STEUERLE: Let’s go ahead and take some questions now. Then we will have
the next set of speakers, then we will have questions there. Somewhere in
between we need to take a break.

MS. TUREK: I’m just curious, on the re-disclosure, why can’t you have an
agreement with them that they won’t do that? That would seem to be relatively
straightforward.

DR. MADANS: I think all of us would interpret our requirements that we
can’t allow a redisclosure to take place, because if we were to do that, we
could let anybody have privacy access to data that would say they wouldn’t try
to re-disclose.

MS. TUREK: But presumably the only person that could do it would be the
agency that supplied it, right?

DR. MADANS: That particular file. But we have many data files. The only way
you could re-disclose it — most of our files, this may not be true for Census,
on their own, if we took out the names and the addresses, those straight
identifiers, when we get them out of the field they are not identifiable. They
are only identifiable if you link them to something else that is in either the
public or private domain.

In the case of a university user, we don’t give them our linked file with
the NDI, because they would be able to do re-disclosure. But if they signed
something that says we won’t do it, it should be the same thing as another
agency doing it. If an agency that has programmatic or administrative
responsibility did do the re-disclosure, there is harm to the subject. I guess
our bottom line is, do no harm to people who have done their civic duty.

I think this is this due diligence thing. If there was a breach, we would
feel in that case it would be our fault there is a breach. So whether or not
you can say they can have anything, or we won’t put out anything, it is what
steps can you go through. There are legal requirements that if somebody at CMS
would do this that they could be fired, fined, whatever. We try to do
statistical perturbation so they can’t. But I think it is the responsibility of
those of us who have made a pledge to the people allowing us to link their data
and giving us sometimes very sensitive data, that we will use a variety of
mechanisms to make sure that they are not harmed. We have thought that someone
signing a piece of paper that says they won’t do it is not sufficient
protection for that person.

DR. STEINWACHS: You were mentioning about the declining willingness to give
the social security number. Your explanation sounds not surprising. Sally or
Ron were talking about a capacity now to match to unique individuals without
the social security number. I thought whether or not there was any possibility
here of a short cross conversation, is there any way NCHS could use what you
have, or is that somehow not in the feasible domain. I can already see, Gerry
is ready to take this.

DR. MADANS: We are having that conversation. I don’t know. That is one of
the things we are trying to work out. We can do similar things and we do link
to the NDI without the social security number. We can do that. It is not as
good link, but we don’t have access to as much information.

So we started these dialogues, but we are on new ground here, whether or
not some of these things are possible. We are hoping that we can do things more
jointly. We have lots of joint projects. It does seem a reasonable thing to do.
So maybe next year when you have this we will have signed all these things and
can move forward.

But I think it is important for us to do it in a proper way. So we try to
work out what are we asking for, what are the problems, what are some of the
reasons you wouldn’t want to do it, and then have an open conversation.

DR. STEUERLE: A lot of your comments on disclosure seemed to me to be
related to surveys, where you had to get permission or you felt implicit that
you have got permission for people to not use the data in certain ways.

But among the other linkages that we are talking about here are linkages of
administrative data sets. I can imagine some private researcher going out and
gathering some data from hospitals somehow, if there was even such a data set
that exists, he wants to link to Medicare data on hospital payments or
something like that.

I am thinking there are a lot of pieces of data where we have never
actually gone to the public and asked for permission, but we may have gathered
the data in some form they followed in order to apply for a program.

There, correct me if I am wrong, but the companies uses a different one. It
is what we interpret the law as allowing us to do. Are you also saying that
there is often an implicit agreement in these administrative data sets that if
I filed for whatever it is, if I filed for Medicare, that that data is
therefore usable by some other agency who also has administrative data records,
that they could be merged?

DR. MADANS: Most of what we are linking are survey data. We have one data
set that is registry data, where our relationship is with the states.

I think where this becomes an issue is how is the owner of that
administrative file, how did they view their responsibilities to the people who
are in their file. That is why we have these very long negotiations. So if we
want to link to the Medicare file, we think we are in a very good position,
because we ask people and they said we could link to Medicare. But then we go
to the agency and they say, we view our requirements as not allowing you to
link. We never had that conversation with Medicare, but we have with others.
Even though we have an explicit statement that says we are going to link to
these kinds of records, and sometimes we even mention the kind of record that
we want to link to, their legal interpretation is that they have to get
approval, even though we have approval.

So it takes awhile, especially when this is the first time it is happening
with that particular agency or that file, to work through why do you think —
like IRB discussions; what is the risk, what is the benefit, how can you
protect. It takes a while to go through that.

But I think our interpretation has always been, if we have permission, then
we should be able to go into an administrative file and pull out the records
for that person.

Now, in terms of linkages where it is not based on a survey, you are just
taking two administrative files and linking them, we don’t do that very much.
So I don’t know much about it.

DR. STEUERLE: There may be other questions for the group when HHS finishes.
There are questions about administrative data sets we don’t combine at all,
independent of the ones that you are thinking about combining at this point.

Other questions?

DR. BREEN: Jennifer, you said that — thank you very much for your talk,
both of you — you mentioned that you would put more resources into data
dissemination, that you thought that was an area where you could use more
resources. But what exactly would you do with the resources? Have you thought
about what you would like to do and how you would like to expand that?

You said you were going to increase the number of data access locations to
utilize those of the Census Bureau, but knowing you, you probably have some
other ideas on what you want to do.

DR. MADANS: I think we should be spending more on this. I don’t know how Ed
feels about this. For me there needs to be some basic science methodologic
development on the disclosure review process. That is where I would put my
first $200,000 or whatever it is going to be.

We are committed to trying to use the Census data centers. There are costs
associated with that. I think there are costs associated with our own data
center, it has been underfunded. So that is two very easy things, to more fully
staff the data centers so that the process is simpler, easier for the user, and
then to develop these disclosure review processes.

I think making it easier on the front end is a staffing question that is
not rocket science. We have some fewer requirements than the Census Bureau does
in terms of how do you qualify to use our data center. We have found it very
difficult to say people can’t use the data center. I think that Julia mentioned
something about, we give it to people we trust. That is not a way we can work.
We have to have very clear rules about who gets it, and we can’t have a lot of
judgment about who can access the data if they have a reasonable proposal.

But I think we can fix that. I think that is a staffing and money issue. I
think it is an increased knowledge issue to deal with how do you do disclosure
review in a faster view.

Then we didn’t talk about this, to really use the administrative data and
use it in a way so that it really does augment the survey data, you have to
figure out a way to get the administrative data faster. That was brought up as
well. So that is money that we are not going to spend, but I think money that
our users need to get someone to spend for the providers of the data so that
they can get it out quicker.

But I think in general, within each of our data divisions we need to put
more resources, more staff primarily and more developmental money into doing
different products, easier access to the products, more technical assistance.
As they get more complicated, people get frustrated because they didn’t quite
understand how to use them. There are some things I don’t know how to file,
some I do. If we had the money we could solve them, they are pretty
straightforward. But the thing we don’t know so much about is how to do
disclosure quickly and efficiently.

DR. SONDIK: I completely agree with that. I think one of the areas where we
could really put some more resources is into smarter front ends to the surveys
and to the vital statistics, for that matter. I view that as part of
dissemination.

I think trying to deal with some of these issues in the RDCs that have come
up — I just want to make one brief comment. I was struck by the prior
presentation and this one, but maybe it is a bias on my part, but I had a sense
that in the prior presentation there was a sense of, it is useful to
disseminate the data, to have the RDCs, but not necessarily the primary purpose
of the data collection that was being addressed. Maybe that is a little harsh.

That is not true with NCHS. I think the primary purpose of NCHS is to
disseminate the information that we are collecting as effectively as we can. In
some sense it is the same problem, but in another sense I think maybe it is a
little bit different emphasis. But if we collect NHANES or HIS and we sit on
it, then we are not doing our mission.

So the idea of making this data as accessible as possible is really a
principal drive for us. Jennifer has said on many occasions that this is our
principal mission, and I agree with that. That is in a way why this is so
important.

MR. LOCALIO: I am citing here the Institute of Medicine report on expanding
access to research data. This is page 12. One of the sets of research that they
cite has to do with evidence of increased public concern about privacy and
distrust of government, assurances of confidentiality. In other words, if you
don’t get people participating in these interviews, it becomes a problem.

I just want to tell you, I read this waiting to see a physician. I had to
wait a couple of hours. In order to get to see a physician today, you have to
give your social security number.

But anyway, one of the things that struck me while I was reading this, in
these studies it doesn’t appear that people, when they are asked about their
attitudes about trust, were asked about the various reasons that concerned
them. Are they concerned that the information they are going to get is going to
be turned over to the Immigration and Naturalization Service, very valid
concern these days, to the Homeland Security Department, to the Internal
Revenue Service, to law enforcement. I would say that those are concerns that a
lot of people have. Having their information linked, de-identified and then
given to some researcher who could care less about who these people are is a
very different level of concern.

It seems to me that you have a one size fits all set of guidelines here,
that we need to partition. So I would encourage people to do more research,
more methodological research on why people feel constrained about participating
in interviews, for example, exactly what the reasons are, and then design
policies based on those reasons, in terms of whom you give access to data to.

MS. GENTZLER: I wanted to ask Jennifer, you had touched briefly on your
experience trying to link with food and nutrition assistance programs, given
that they are state administered mostly.

DR. MADANS: We have just started that. There was that recent IOM or
somebody’s report that said we should link HANES to the food stamp and WIC and
a variety of other things. So we have something jointly going with the Census
Bureau to try to do a pilot on that. We have been talking with the folks at the
Food and Nutrition Service to figure out how to do that link. That is one of
the ones where it is going to have to work through some conflicting
interpretations about how you do it, and the state involvement and what kind of
approvals do you need and who has to give the approvals and where do you get
them.

So we have started that, and again, next year we can tell you what
happened. I think there are those basic concerns, there are a lot of other
concerns with doing something on a state basis. It is not so bad for us because
we are only in a certain number of states every year, but you have to negotiate
with every state eery time you do something; it gets to be a little time
consuming.

If we were in all 50 states, this wouldn’t probably be a way to do it,
HANES isn’t in all 50 states every year. But I think that this is again where
we are at the beginning. It probably won’t take us three years, but it probably
will take us a little time to figure out whether or not this kind of linkage is
going to work, and the data when it is linked with the survey data is useful.

Again, there is cost doing these linkages. The final reason for doing it
is, after you do it for awhile, is anybody using it, is it helping them. That
sometimes is a function of the quality of the data, or not so much the quality,
but is it appropriate for the research questions that you can address with that
data set. It may be that that is not the case, so why put everyone through the
process. But yes, that is one where we are at the beginning.

MS. TUREK: I am struck by the fact that the confidentiality and privacy
laws seem to be agency specific. I think my big example of this is, the HIS
only puts out range for its total family income question, and you have to go to
the research center to get the dollar amount, where on the CPS I can get the
dollars of income for detailed sources of income on 20 sources. I don’t see
where the risk on the two surveys is all that different.

It seems to me that this is a case where some consistency across agencies
would also be helpful.

DR. MADANS: I don’t disagree with you on that particular issue. I think
that there would be consistency.

But it is not that item again. When you make a public use file, it is a
tradeoff of where do you want your details. So when those files were created,
it was decided because it is a health survey to give up some detail on that
item, as opposed to detail on anther item. I don’t know where the tradeoff was
for that, but it is a balancing act.

I think the folks who create the files try to do the balance in the best
way they can with their knowledge of what the data are going to be used for,
and they try to create the files in a way that they will meet the needs of the
greatest number of people, which sometimes means people like you don’t get
exactly what you need.

But in this particular instance, we have heard you. They are going back to
look at that to see if we can change that and would it affect the other items.
But when our disclosure review board looks at a file, they look at a whole
file. They may go back to the program and say, okay, you can have this or you
can have this, but you can’t have both. In that decision it might have been
where top coding comes.

DR. STEUERLE: I think we are going to have to cut here and go to our next
speakers. I have got the list here, but I don’t know how many people we have
got left from HHS speaking. So just so I know how many presentations we have,
can you raise your hands so I can count? Five formal presentations.

Why don’t we go then immediately to the next presentations. That would be
David Gibson and Steve Cohen.

Agenda Item: David Gibson, CMS, Office of Research
Dissemination and Information

MR. GIBSON: My name is David Gibson. I work in CMS Office of Research
Dissemination and Information. I am here to replace Spike Duzor, who is the
project officer on this particular project that you asked me to speak to you
about today. To the extent that I can answer questions, I will certainly try to
do so.

First of all, you have to realize that when Medicare and to a large extent
Medicaid came into being, we saw our missions not to do research; it was
basically to pay claims. Originally it was oriented to acute care, at least on
the Medicare side. It was oriented towards cost based or reasonable charge
reimbursement. There was little concern for the impact of administrative
decisions on research and how to use the data.

Consequently, we organized our databases in ways that often made it
difficult to do research. But as time went on, we saw that we wanted to come up
with more rational ways to make payment, for instance, to bundle the payment
for a number of services together rather than paying for each service
individually. We also saw that we wanted to match the services that we were
providing and that we were covering to what the needs of the population were.

We also realized that when we were in this mode of trying to pay claims
quickly, that we often made payments when it was inappropriate or even
fraudulent. We were often in what they call a pay and chase mode. We made the
payment, and then we had to go back and get the dollars, and that was often
hard when a home health agency or a DME was using a former BP station as their
place of business.

What are some of the barriers that we have in using CMS administrative and
other data to study health outcomes? Spike asked me to go through some of the
list of items. Many of you probably know many, many more; I don’t pretend that
these are exhaustive. Then I want to describe to you a database that we are
building that we hope address some of them, and it is not going to be all of
them, unfortunately.

One of the big problems that we have in the Medicare and Medicaid systems
is a lack of unique identifiers within programs across types of data. The
biggest one we have is the health insurance claim number or HIC that we use to
identify beneficiaries in the Medicare program. We use an identifier that as
most of you know is a combination of a CAN, a claim account number, which is
the social security number or an RRB, Railroad Retirement Board number, for a
beneficiary, and then a beneficiary identification code that relates that
person to the bennie.

Unfortunately, some of these identifiers change over time. You have
situations where a wife gets benefits under her husband’s account, and over
time she may earn enough benefits under her own account to get benefits. She
will change the CAN portion of her health insurance claim number.

This causes problems in following people longitudinally. If you have a
situation where you are working with a five percent sample, you already have a
fairly small sample. It is robust, but think about if you want to follow a
cohort over a long period of time. If one to two percent of the people cross
reference out of your five percent sample, think of multiplying .05 times .98
times .98, and you keep seeing the effect of that diminishing of the population
that you are interested in.

So we have that problem of unique identifiers for beneficiaries. We have
the same problem with some of our providers. If you look at some of our data
that is provider oriented, say hospital oriented, you will see that the number
of hospitals has supposedly dropped drastically over time. What that is, is a
recategorization of those hospitals as critical access instead of short stay.

Second, we have a lack of unique identifiers across programs. We have
Medicare, we have Medicaid. As the speaker before me mentioned, we have
assessments now for home health, for skilled nursing facilities, for rehab
facilitates and for swing bed. These numbers are assigned by the state, not by
the Medicare or the Medicaid programs, so you have a different set of
identifiers for bennies for this very rich source of information that will help
to give more information about the facility and about the patient.

What are some other barriers that we have? We have the separation of
billing and associated diagnostic and therapeutic care into separate bill
types. By this we mean, if you look at the services that we have, you can see
that many times if you wanted to create an episode you would have to go to as
many as seven different bill types to try to find the information that is
related to that particular episode.

We also have the use of what we call ruleout or confirmed diagnoses on
certain types of bills. Some of the bills that we have are not at the point
where final diagnosis has been made. So if you are looking at the principal
diagnostic and even sometimes the secondary diagnosis, you are going to be
seeing the diagnoses that the physician is trying to determine whether the
patient really has that particular condition or not. So that diagnosis and the
presence of that diagnosis in the field doesn’t necessarily mean that you would
want to include this person in a particular category.

This causes us many times if we are doing some cross sectional analyses,
that we are looking at patients who have diabetes or whatever, that we are in
fact including bills there where the physician was trying to confirm or not
whether the patient really had diabetes or not.

So what does this cause? This causes problems with trying to identify
persons by diagnosis in a cross sectional analysis. Just getting prevalence
rates is often difficult for us, because we are not sure that we really are
looking at those people who in a cross sectional time frame have a particular
diagnosis. It also causes problems with us trying to find when certain
conditions started, the incidence rates associated with particular diagnoses.

We also have problems with the fact that we use different coding systems.
Many of you have encountered this. If you are looking at someone who had a
surgical stay in the hospital, you will find that on the hospital sides we use
the ICD-9 procedure codes. On the physician or professional side you will see
that we use what we call the HCCPC, the health care common procedural code,
which is basically based on the CPT-4 that the AMA puts out.

Other problems are the lack of what many people think is most crucial,
clinical information that determines and differentiates personal level critical
pathways. In other words, if a physician has decided that a person has a
particular disease and they want to adopt a particular regimen or protocol for
that patient, it would be helpful for us if we had that clinical information.
We do not have that.

Next is the lack of information on the cause of disability for the
disabled. About 15 percent of our people are disabled. They are under 65. We
can be looking at the diagnoses on the claims to try to figure out what was the
cause of the disability that they have, but we do not actually know the cause
for disability for this individual. So we do not have from SSA the actual cause
for why the person is entitled to Medicare.

Another problem that we have with our data is, you will notice that most of
the time the data that you will get from us will be listed by what we call MSC
code, Medicare status code, which will just tell if they are aged, if they are
disabled, or if they have ESRD. This will not tell you for the aged population
if they formerly had a disabled status. For instance, many of our aged
population aged in, if they survived long enough, into our aged population.
They are quite different, we find, from the aged folks who come in, many times
postponing to come in when they are eligible at 65, if they are covered by a
working aged situation. They may be 69, 70 or even older before they sign up.
Have very, very few causes for disability. Yet our data tends to group them
together if you just look at things like age or Medicare status code.

What else? Lack of comprehensive, and by that I mean both breadth and
depth, of person level data on primary and secondary health insurance coverage.
We group people who are working aged and are covered by their employer sponsor
plan with those people who use Medicare as their primary source of insurance.
Consequently you may miss a lot of their utilization if you go out looking in
the claims, because the provider if they are going to Blue Cross Blue Shield
may not even bother to send the information in. So when you compute rates and
whatnot, you find that you often underestimate what the actual cost is for
those people who use Medicare as their primary, because you are grouping people
in who you shouldn’t be including.

Next is the lack of information on socioeconomic status. That would be an
ideal thing to add in. We also are unable to get the cause of death, and that
has been discussed earlier today in other presentations.

This one is an odd one. Since we have just one into Part D we consider that
we would be supplementing the Part A and the Part D benefits with our Part D
event data. Unfortunately we are unable to do that. The law prohibits us from
doing that. We have a notification that we are hoping to get out soon that will
allow us to start linking the Part D drug events for those people. It is not
all the people who had drug benefits that we are going to be getting their drug
events. If they are in a situation where they are getting coverage by their
employer, their employer may get a subsidy, but that doesn’t mean we are going
to get the drug events. We are going to only hopefully be getting the events
from the folks who are going through PDPs.

Next is the size of the sample. Traditionally we give out the five percent
sample, but for many conditions — and I will be talking about the chronic
condition warehouse in just a couple of minutes — it would be ideal if we had
a larger sample. But right now we have a five percent sample that we are
dealing with, and that is primarily because of the size of the physician
claims. There are about 800 to 900 million physician supplier claims a year,
and these are very complex records going often over 2,000 and 3,000 characters
per record, and it often becomes difficult to do large scale analyses where you
would like to do small area analysis.

Next is the inability to disaggregate the program payments, payments by
other payors such as some claims for the working aged, and the beneficiary
payments. If the services are all bundled such as for the DRG under the
inpatient TPS system, to disaggregate them to get accurate revenue functions
for the provides. If you wanted to disaggregate an entire stay that involved 20
or 30 different revenue centers and different types of services, you have all
the information that is only stored on what we call the fixed portion of the
claim.

Next is the inability to link specific services on claims with provider
costs to data provider cost functions. I think this was mentioned earlier.
Wouldn’t it be nice for institutional providers to link the claim back to the
cost report, take the payment amount, and using some methodology, whether it is
cost to charge ratios or some other methodology, allocate the provider’s cost
for providing that service to all the bundled services within that entire stay.
On the physician side, it would be good to have information at the office
setting for determining the cost to the physician for providing a service. We
do not have that.

Next is the inconsistent use of the UPIN or the unique physician
identification number. We have a problem, that many times the physicians in a
practice will all use the UPIN number for the main doc. Especially we see this
with radiologists, pathologists and to some extent anesthesiologists, where
they all use the UPIN for one doc. So you may see one service only for many
physicians, or you may see 25,000 for a limited set of physicians.

Now, admittedly for pathology they may see a large number of samples, but
many times we think this is the result of using the same UPIN number for all
the physicians in a practice.

DR. STEUERLE: David, if I could interrupt you for a second, this is partly
my fault, but we have seven speakers on this session.

MR. GIBSON: I am going too long, I apologize.

DR. STEUERLE: It is not entirely your fault. We ran over in the morning
meeting, and we haven’t enforced on anybody yet, so we are starting to enforce
on you.

MR. GIBSON: I apologize. Let me talk about Section 723. There are a lot of
these barriers to using data. I would like to mention a couple of them that we
are trying to address with Section 723. That is the final one down here.

723 was created by the Medicare Modernization Act. It was signed in
December of 2003. It made a lot of major changes in the Medicare program. I
talked about the outpatient prescription drug benefit, basic changes to
Medicare Advantage, to the managed care program, but it also mandated studies
and demonstrations to improve the effectiveness of the Medicare program and the
quality of program recipients.

What that law did was establish a research database for the chronically
ill. It was recognized that the chronically ill account for the great bulk of
program payments. Before the program had been geared towards acute episodes, it
was recognized that the chronically ill represent a large proportion of the
dollars. In fact, they estimated that it is closer to 80 percent of the dollars
are accounted for by the chronically ill.

So what they wanted to do in this database that would be geared towards the
chronically ill, they wanted to provide the capacity for improving quality of
care, for reducing cost of care, et cetera.

So what we did is, we went out, we talked with some clinicians. We
developed algorithms for defining 21 chronic conditions that we could use the
claims for to identify these individuals. These are just some statistics that I
just quoted to you. The idea here was to identify individuals who had these
chronic conditions, and to eliminate some of the barriers that I mentioned
earlier, such as, we developed a system to do away with the problem of cross
referencing, and the fact that many people change numbers over time.

We developed a methodology that would also allow us to link data in these
different systems. There is a diagram here that describes that. It is called
the enterprise cross reference system. The idea here was to link Medicare,
Medicaid and assessment data. We assigned one unique number within each of the
systems, and then across the systems we allowed a methodology that will
identify individuals in both systems and pool their information together and
put it into the chronic condition warehouse.

This allows us to create patient level profiles. It allows us to identify
and keep a history of people who have these 21 chronic conditions, but in
addition it allows us if we want to search the database for other conditions or
do ad hocs, it will allow us also to do that as well.

The difference that you see in this database versus the other that we had
is that most of the databases that we have worked with in the past have been
flat files. This isn’t a relational database.

I am trying here to pick up a couple of points that might be of particular
relevance. I think the idea that I want to emphasize in this database is that
it will allow us to link data across the different programs, Medicare and
Medicaid and the assessment data, pull it together into a relational database
and allow us to query it and develop statistics off of it. That is what we are
in the process of doing right now.

It is based on a five percent sample. We have loaded from ‘99 to 2000
and four at the five percent level. We are testing to see if we can go a
hundred percent with the 2005 data, and if so, we are going to load the 2006 as
well. One of the things that is particularly appealing to me is that instead of
waiting for the staff files, we are going to use the claims as they come right
out of our system.

I talked about the ECR a little bit. Core research files, that is the idea
of paring down our records so that they are not so long. I mentioned the
algorithms that we have gotten for the 21 chronic conditions.

The Iowa Foundation for Medical Care, that is one of the QIOs. It is a
recognized data center for QIOs. They are also the maintainer of our assessment
data.

Moving on, we are working also with the University of Minnesota. They have
a group out there called Resdac, that works with researchers to try to help
them to see if the database is of particular use to them.

I think that is it. I hope on the last part I didn’t go through too
quickly, but I got the feeling I was running a little over time on that.

DR. STEUERLE: Again, I have to apologize both to you and to our main
speakers. I think we probably crammed way too many speakers into this session.
But I think to make sure that all the speakers to make sure they are able to
give their talk, I’ll ask them to stay within maybe ten minutes for the
remaining speakers.

Agenda Item: Steven Cohen

MR. COHEN: Given that I have a half hour presentation, I am really going to
cut to the highlights. I have some handouts, so you could get into more detail
here.

In addition to everything that we have covered so far, this presentation
takes a little bit more of an intensity of integrated survey designs,
analytical enhancements through the integration of surveys, both with
administrative but also other survey data.

I will go over briefly some activities in the department and in AHRQ in
terms of implementing integrated survey designs to enhance analytical capacity
and see data quality enhancements, and see some efficiencies in the collection
of data, and how this model is a good model to help also improve the accuracy
of survey data. There will be a few examples at the end in terms of the
application of this AHRQ data portfolio, in terms of improving health outcomes
and limitations of the approach. Much of the limitations in terms of some of
the confidentiality issues have already been covered. The data portfolio
activity fits in very nicely with the mission of the agency to improve the
quality and the safety, the efficiency and the effectiveness of health care for
all Americans.

On to this integrated survey design model, the model itself looks at core
health survey, and it looks to other existing larger surveys or administrative
databases from which the survey could be derived. Rather than having an
independent large scale screener which would be very costly, this integrated
survey design makes use of ongoing surveys, so one can have that
predispositional information and use it in a very cost efficient manner. It
also makes use of the capability of linking secondary data, and if there was a
prior host survey, record of call information in terms of where there were
reluctant respondents, where there were multiple contacts to get the higher
response rate, and after the fact quite a bit of detail in terms of
sociodemographic factors for non-response adjustments.

In terms of the core survey and the department that has used this, it is
the medical expenditure panel survey. It is linked to the health interview
survey. The medical expenditure panel survey is designed to provide estimates
to health care use, expenditures, sources of payment.

Some of the core issues Julia Lane talked about was outlier cases and in
the expenditure distribution, what percent of the population is tied to 25
percent of the expenditures. So in terms of confidentiality issues and linkage,
we would opt to do everything possible to give the greatest accuracy on those
high expenditure cases and be constrained in terms of some geographical
dimensions for data linkage, so one can get at the core analytical variables,
one could go to either the AHRQ data center or the NCHS research data center,
and I will talk about how the health interview survey and the MEPS are linked.

In addition to those linkage enhancements, we have a survey much like MEPS
which is core, and you have a predispositional survey such as the health
interview survey, a general purpose public health survey, 40,000 households,
110,000 individuals. In addition to screening perspectives, you have an extra
time point to enhance predispositional information, longitudinal analyses.

So we draw up the health interview survey with all its strengths and
analytical capacities. It facilitates a very timely effective over sample of
cases, joined to the medical expenditure panel survey for two years. I am going
fast, but there are a few other compelling additions that have helped both
surveys tremendously.

NCHS has been on the hook to get data to us fast track, really fast track,
so we can get the information and then sub sample. From that they have
developed a fast release of insurance coverage estimates. They are putting it
out. I think the 2005 estimates came out roughly like June of this year, and I
think they do quarterly estimates. So both organizations benefitted by demands
to get data out quickly to make it more usable to the research community.
Again, very efficient over samples of policy relevant groups.

So that is the first core example of the integrated design. MEPS recognizes
that while households could give you quite a bit of very accurate information
on their use, on their insurance coverage, on their access to care and their
perception. There are a lot of things that are problems with it in terms of
health expenditures, very, very demanding questions. So the underpinnings of
the design makes use of the recognition of a lot of item non-response, and it
links to other follow-back surveys. So you get permission to go to medical
providers to get details on health expenditures and the diagnoses and the event
dates.

We have an insurance component. There are two stages of linkage that I will
go into later, but this gives the depth of information on what we are spending
for health care, where the asymmetry is in terms of those expenditures, how
that impacts on take-up and overall access to care, and how that translates to
health outcomes. The insurance component is linked to the household component
to help support what the premium costs are.

A little bit more of this integrated design. As I said, the health
interview survey serves as the sample frame for the household component of the
MEPS.

We also benefit by a Census Bureau business register, which has tremendous
timeliness, is the best frame available in terms of having good coverage rates,
great information for modeling and over sampling, an annual survey of roughly
40,000 establishments. It builds off all the precursor information and allows
for tremendous modeling design work. Then the linkage from the core surveys to
follow-back surveys, and as we have seen on other examples, linkage to
secondary data.

Instead of a typical survey where you just have information at the county
level or the MSA level, where you are drawing in the analytical units, here you
have precursor information from the health survey on the sociodemographics, so
you can form non-response adjustments. A few equations have to be part of any
presentation.

But in terms of one other thing about the linkage between one survey and
other survey, this goes into one of the very core outcome measures rather than
the SF-12 that we also have in the MEPS. This looks at in the last year of the
MEPS a self assessment of health, and then it looks at of those people, were
they ever uninsured over a two-year period, were they consecutively uninsured
for two years, and through linkage between MEPS and HIS, MEPS can give you two
years of coverage with several rounds of data collection to minimize recall.

At the first round, if a person is not covered, it has a question with a
two-year recall, very, very risky to put that out without any verification.
With the linkage to the health interview survey where they have a point in time
and they also go back in time, we can make edits, and now we can put out
estimates of the long term uninsured in MEPS. You can see how that could factor
into certain analyses. Again, on expenditure estimates, one other dimension in
terms of precursor information from HIS to MEPS.

Concentration of expenditures is critical. Persistence of the concentration
of expenditures is critical. So year one, year two. A lot of analysts come to
us, groups like Kaiser Permanente, they are looking at the top one percent of
the population and what can be done in terms of more efficient use of services.
By having HIS information, we can look at two prior years in terms of
predicting a third year. Two surveys married, and the whole is greater than the
sum of its parts.

A little more detail on the medical provider survey, compensating for item
non-response on expenditures, gold standard for expenditure estimates, greater
accuracy and supports imputation. For our households, we get permission to go
to their hospitals, their associated physicians in the hospital, office based
physicians, home health agencies and associated pharmacies. The pharmacy
verification component is critical as we wait for CMS to come on line with the
Part D data to inform a number of the comparative effectiveness analyses. In
the meanwhile, MEPS for both Medicare beneficiaries and the population in
general, the civilian non-institutionalized population, has a pharmacy
verification survey.

We have units of building blocks from the household and the provider. We
take the provider when we have it from both sides. We take the provider when we
only have it from the provider. We take the household data when it is from the
household, but we recalibrate based on those two points in time.

I am going very, very quickly. I hope this is helpful.

I am going to slow up a little bit. The non-response is based on
imputations from our medical provider survey. Hot deck imputation; we build on
provider expenditure data. We use model based predictors in terms of defining
the hot deck cells. We look at predictors of expenditures, predictors of
non-response. The intersection of the two is what we prioritize on. Then we go
to factors associated with expenditures. Then if we have room for other
variables, we will go into the item non-response.

On the pharmacy verification component, the household gives us entree to
the pharmacies. We get details on the pharmacy itself and the use. Then the
pharmacy gives us the medication name, the NDC code, quantity dispensed,
strength and form, source of payment, amount paid.

Fictitious person so there is no confidentiality violation. This is Sandy
King. She might exist, but she doesn’t exist at this address.

You can see some of the wealth of data. With that expansion from the
household data to get entree to the pharmacy to get all this data. It
sequences, where we can then link that NDC code to other proprietary databases
that can get us more granularity in terms of the therapeutic class and the
sub-classes. So it really enhances analytical capacity. We can link to
databases like the FDA, the year of approval for the drug, whether it is a
brand or a generic indicator.

Some of the outcome analyses. We look at trends in out of pocket burdens
across all major population subgroups, prevalence of potentially inappropriate
prescribing patterns. We also look for substitution effects over time, new
higher priced medications, but maybe there is some cut utilization overall,
looking at outcomes. Trends and use by therapeutic category, and more and more
work on predicting models of future year expenditures.

We just had a meeting with the Medicaid chronic disease directors. Very
concerned in terms of looking at chronic disease, Medicaid expenses, looking at
things that can inform them in terms of cost avoidance. I don’t know if there
are any cost savings, but just something that is informative.

On administrative data, we work very closely with CMS. This is not a
linkage exercise, but it shows how administrative data would have national
estimates. To be reconciled to MEPS, they cover infrastructure, research and
development, they cover the nursing home and institutionalized population.

We have to bring the national health counts down to MEPS, so we can see if
the two data sources are providing concurrent information. If they are not, if
the reconciliation shows areas of disparity, we have to look to see how each of
those data sources as they speak to one another at a higher level, can be
improved.

We also link to CMS data through our IRB, getting information from Medicare
to validate household reports on the use and going into the details of the type
of service and some complexities in patients on separate billing doctors.

A little bit more on our establishment survey. The estimates that come out
from this look at differentials nationally and state by state in terms of
take-up of coverage, the cost of the coverage. These estimates go into health
update, GDP estimates of the premium cost for the provision of health insurance
coverage. We benefit by a linkage to the business registry. With that comes a
lot of conditions. We cannot release the micro data to the public. This data is
residing at the research data center, gets tremendous use, but it is in a
secure environment at the RDC. But we do release tabular data.

The integrated design allows for the detail of the sampling to optimize
sample designs to minimize variance for fixed cost constraints. Serves as an
imputation source for editing, for small area estimation models, for table
production and also just like HIS and MEPS, non-response adjustments.

These integrated results, rather than more surveys, would have some
palliative effects on reduction in respondent burden, sample precision and
improvements, and modeling research.

Just as we close out, let me just give a little flavor to some of the other
elements in the AHRQ data portfolio. We are having some discussions, and this
might feed into some of the questions in terms of what is permissible on
administrative data.

This is a different part of the data portfolio, the health care cost and
utilization project; 37 state partners, roughly 90 percent of all payor
hospital discharges in the U.S., inpatient data, ambulatory surgery and
emergency department databases. The patient enters the hospital, get billing
records. The state partners send it to the major data contractor tied to this,
and produces a standardized administrative data set.

The standard linkages on the administrative data would be the AHA data,
more details on characteristics of the hospitals, information on the providers
and secondary linkages that we have heard from all the other speakers so far.

The identifiers are encrypted. There is some limitations with this
encryption, different states do different things, so we don’t have it across
the board. But some states are better than others, and you can have some more
episodic, I wouldn’t say longitudinal, but episodic type analyses across the
databases. Some of the state partners in their own right make use of linkage to
vital records, to disease registries, to state program files to enhance the
analytical capacity.

Some of the challenges as I said are, the encryption methods are not
uniform. You don’t have consistency across time, and some sensitivity to some
of these supplemental linkages.

Some of the analytical outcomes with this database would be looking on
racial and ethnic disparities and readmissions for diabetes, the incident cost
of motorcycle injury to inform decisions on state helmet laws, financial status
of safety net hospitals, impact of motor vehicle exhaust on pediatric asthma
admissions, certainly marrying a whole bunch of disparate data sources for an
informative study.

In terms of pushing further, the department needs more information,
particularly with Secretary Leavitt’s transparency initiative, and looking
further in terms of quality metrics. Building on the capacity when it does
become more visible on electronic medical records and supplements to the claims
data, better links across the different states that are uniform. More
information on hospitals that are missing right now to help in terms of
decision making by consumers on organizational culture, clinical integration
and the availability of data from health information technology, and more
quality metrics and nursing staffing data is always critical.

Where we need to be. There is an initiative in the Department called
Joining Forces to expand on existing administrative data for consumer choice.
We need information that is more timely, that is less expensive, that is
actionable. Right now we have quite a bit of administrative data, but a lot
more work needs to be done, and dealing with some of the confidentiality issues
as well.

Some of the examples we are pushing on would be more HIT applications for
timeliness, more augmentation on clinical detail, on what condition was present
in admission. I think our colleagues from CMS said that would be very, very
helpful, too. And some lab values and more cross-site data, and promising all
the confidentiality commitments are adhered to.

We have some staff from HCUP, so if you have some other questions, they are
here to answer some questions.

Just before I close out, the agency with CMS and others in the Department
have quite a bit of activities going on with MMA. The agency in particular has
a role in terms of studies on comparative effectiveness analysis. One of the
programs at the agency is the Decide research network. That focuses on
developing evidence to inform decisions and effectiveness. The purpose of this
program is to expeditiously develop valid science evidence about the outcomes,
comparative clinical effectiveness, safety and appropriateness of health care
items and services.

Several well-known academic centers are a part of the network. The onus of
the Decide network is to analyze existing health care databases for comparisons
of effectiveness and outcomes of treatment, analyze existing disease, device
and other registries, and improve the accuracy of those data sources. We have
spoken on ancillary and secondary data sources for some of the linkage.

We have a data center. It is the second one in the Department. Some of the
models that NCHS are pushing on for getting more and more information in the
hands of the analysts to inform policy and practice directions AHRQ intends to
go in as well. There is quite a bit of coordination in the Department on that
front.

Some linkage variables that you can look at; it is probably hard to read on
this screen. I think the presentations gave great justice to some of the
limitations in the availability, so I am not going to repeat that.

Let me just close. I might have done 25 minutes in ten, I don’t know, but I
went very quickly. I did cover the capacity of integrated survey designs, the
ability to reduce non-response, related enhancements and data quality,
analytical capacity, some attention to MEPS but some of the other data
resources, both within AHRQ and the Department.

That’s it.

DR. STEUERLE: Stephen, that was not only a brilliant presentation, but a
tour de force on your speaking ability and speed.

Agenda Item: Gerald Riley, Social Science Research
Analyst, Center for Medicare and Medicaid Services and Martin L. Brown, Chief,
Health Services and Economics Branch, National Cancer Institute

MR. RILEY: Good afternoon. Martin and I are going to team up to describe
the SEER Medicare linked database, which represents a joint effort of the
National Cancer Institute and CMS. We will also briefly discuss some other
linkage projects. I think we are planning to go about ten minutes each, if that
is okay. We will try to make it a little shorter.

I will give some background information on the linkage, and Martin will
discuss some of the uses of the data and some of the access to the database for
outside researchers.

Before I begin, I should acknowledge Joan Warren of the National Cancer
Institute, who helped prepare this presentation. She has played a very central
role in developing and improving the linked database.

SEER Medicare consists of cancer registry data from NCI’s surveillance,
epidemiology and results of the SEER program linked to Medicare records on an
individual basis. The linked database has been in existence since 1991, and has
became a significant resource for cancer related health services research.

Under SEER, NCI contracts with individual cancer registries to collect and
identify information on all incident cancer cases in their reporting areas,
with the exceptions of non-melanoma skin cancer and in situ cervical cancer.

The participating registries attempt to identify cases treated in all
settings, and they examine death certificate records and autopsy records to
identify additional cases. Incident cases only are reported, and recurrences
are not captured. The SEER program began in 1973 and has covered 11 geographic
areas since 1992, representing 14 and a half percent of the U.S. population.
The program expanded to four new areas in 2001 and now covers about a quarter
of the U.S. population.

This map shows the SEER reporting areas. Until 2001, the program covered
five states and six metropolitan areas. In 2001 the states of New Jersey,
Kentucky, Louisiana and the remainder of California were added.

SEER areas were not selected to be representative of the U.S. population,
but were selected for the quality of their cancer registries and the diversity
of their populations. Analyses of the elderly population in SEER areas have
shown that there are lower percentages of whites and people living in poverty,
and a higher percentage of urban dwellers compared to the U.S. in general.

This slide shows some of the detailed clinical data that are collected
under SEER. Each individual is assigned a unique case number. If an individual
is diagnosed with more than one primary cancer while residing in the SEER area,
information on each cancer is recorded separately under that person’s case
number. Information on month and year of diagnosis is collected, as well as
site of cancer, histologic type and extent of disease at diagnosis.

The SEER program staff uses information on extent of disease to assign a
stage of diagnosis. SEER also captures type of cancer related surgery if any,
and any radiation therapy that is given or planned as part of the first course
of treatment. Vital status is also tracked over time and includes cause of
death for most cases.

Most of the Medicare administrative records are included in the linked
database. Enrollment data provide information on entitlement, demographics,
Medicaid buy-in status and managed care enrollment. Individual claim records
are included for all kinds of coverage services.

Most claims records are available from 1991 forward. The continuous
Medicare history sample file contains longitudinal data on a five percent
sample of Medicare beneficiaries from 1974 onwards. In addition, a five percent
random sample of Medicare enrollees has been identified who are not in SEER but
who reside in a SEER reporting area. These individuals serve as a cancer-free
comparison group for studies on cancer screening and other topics.

It is our intention to add enrollment and Part D prescription drug plans in
future updates of the database, assuming those data are available. We hope to
add Part D drug claims if this becomes feasible. As Dave said, we are working
to try to get access to that now.

The SEER and Medicare data complement each other in providing information
on a variety of cancer control activities. The data are useful for patterns of
care studies that control for the effects of comorbidities. Post diagnostic
surveillance can be measured for many years after the initial course of
therapy, and recurrences can often be identified from the claims data. End of
life care can also be analyzed, including the use of hospice services, which
are covered by Medicare.

I will briefly describe our linkage activities. NCI receives files with
personal identifiers for cancer cases directly from the SEER registries. NCI
checks the files, and if they appear satisfactory, they are forwarded to a CMS
contractor, who matches the data to the Medicare enrollment database. For all
cases that are successfully matched, the unique health insurance claim numbers
or HICs are identified and all Medicare claims for those individuals are
extracted. The contractor then removes all identifiers including the HICs, and
the claims and enrollment data are sent to NCI’s contractor for creation of
analytic files. All the analytic files retain arbitrarily assigned SEER case
numbers to distinguish the individual cases.

The linkage is updated every three years, with the next update scheduled to
begin in August 2007. We do this every three years because it is a pretty time
consuming complicated process to do the updates.

MR. BROWN: And when we update, we update the entire thing retrospectively
because of these changes in enrollment status that were discussed.

MR. RILEY: The current files contain SEER cases diagnosed through 2002, so
the next update will carry us through 2005.

I will just briefly describe the matching algorithm that is used to match
the SEER and the Medicare records. In most cases, social security number is
reported by the registry. That is the most important variable that we use in
matching. We do require some agreement on corroborating variables such as first
and last name, month of birth and sex. Agreement on sex is relatively important
to prevent us from inadvertently matching records for husbands and wives. If
SSN is not available or doesn’t match, we use first and last name, date of
birth, middle initial and date of death to match records.

Our matching criteria in the absence of SSN are rather strict, because it
is relatively easy to get false positive matches with such large databases
involved. Our match rates for persons aged 65 and over are quite good. We have
been able to find a Medicare record for about 94 percent of the elderly in the
SEER database. This varies somewhat by race and ethnicity. The match rate for
Hispanics is only about 88 percent, and that for Asians is 90 percent. The
match rates for persons under age 65 is rather meaningless because we don’t
expect to match most SEER cases in that age range.

The next table shows the number of linked cases for some of the most common
sites of cancer. There are about 300,000 prostate cancer cases and over 200,000
cases of breast, lung and colorectal cancers, so this is a very large database.
These top four cancer sites account for 60 percent of all linked cases. The
database is large enough to support detailed studies of many less common types
of cancer as well.

Before Martin describes some of the applications of SEER and Medicare, I
will briefly describe some other database linkages involving SEER and Medicare
data. SEER has been linked to CMS’ health outcome survey or HOS. The HOS
gathers data on health status measures among the Medicare managed care
population. The survey is administered annually to 1,000 Medicare enrollees in
each Medicare Advantage plan. The survey is used to assess the ability of plans
to maintain or improve the physical and mental health of its Medicare members
over time. The linkage with SEER will permit better studies of quality of life
for cancer patients in Medicare managed care.

There are also current plans to link SEER with the Medicare consumer
assessment of health care providers and systems or HCAPS survey. HCAPS measures
the experiences of beneficiaries in their health plans with fee for service
Medicare and with providers like hospitals and nursing homes. The survey is
used to monitor quality of care and to measure the performance of Medicare
health plans and providers. So the linkage of HCAPS with SEER would permit more
detailed studies of patient experiences with cancer care.

Medicare claims and enrollment records have been linked to many other
databases besides SEER. Medicare administrative data are routinely linked to
the Medicare current beneficiary survey. They have also been linked to other
surveys, like the national long term care survey and health and retirement
study data. Data have been linked to social security administrative records
under interagency agreement between CMS and SSA, and Medicare data have been
linked to several NCHS surveys under an interagency agreement, as Jennifer
mentioned a little while ago.

These data linkages have greatly enhanced the value of survey data by
adding information on use of health care services that is difficult to obtain
via survey.

I will mention one attempted linkage with SEER that did not prove very
useful. NCI considered linking SEER with Medicaid data to get better
information on health care use and costs for cancer patients with low incomes.
This link is difficult to begin with because the Medicare data are state
specific, which complicates the privacy and the technical issues involved.

The validation project was conducted by NCI with SEER and Medi-Cal data
from California. Medi-Cal claims were found to be not very useful, primarily
because of heavy enrollment in managed care plans. That is, the state does not
give claims data for Medi-Cal managed care enrollees, so the information
available on them is minimal. NCI has therefore dropped its effort to link SEER
with Medi-Cal data because the claims are not complete enough.

So Martin is going to talk about some of the specific applications of SEER
Medicare data, and also some of the conditions of access to the data.

MR. BROWN: There are three or four contexts in which we use the SEER
Medicare data. This whole project was instituted ten or more years ago, because
at NCI we are associated with the SEER program that Gerry mentioned. This was
seen as a natural extension of our mission to do surveillance research and
monitoring.

Part of what we do is what we call inhouse research at a research
institute. We are an extramural branch, but we do inhouse research as part of
our federal mandate to do surveillance.

Secondly, we have done a lot of work to publicize the existence of the SEER
Medicare database, to provide technical assistance through various mechanisms
such as a very extensive webpage, conferences, outreach, Joan is up at Harvard
today, giving an extensive seminar, for example, and a funding mechanism. We
have developed a large stable of extramural researchers who use SEER Medicare
data.

Then we have hybrid studies in which we have developed over the years this
partnership with extramural researchers. Quite often we will have a need to do
a particular analysis at NCI. We will get some of the clinical or health
services expertise from the outside and we will get together and partner a
study. We oftentimes have people come to us and say, can you do this for me. If
we think it is a worthwhile thing we will say yes. For example, we have been
working with the American Medical College’s Center for Workforce Studies to do
a study of supply and demand for the oncology workforce over the next 20 years
or so.

So that is the context in which these things have come about. We now have
about over 200 published articles that have used SEER Medicare data.

A couple of examples that fall in these various categories are trends and
use of cancer related services, procedures, resources and costs, descriptions
and disparities in use of cancer care, patient, physician and health services
and determinants of patterns of care, and volume outcome studies.

For example, this paper by Gerry Riley looked at differences between HMOs
and fee for service settings and stage of diagnosis and treatment approaches
for early stage breast cancer. This was done in 1999 when there was a lot of
heat and emotion about whether the quality of care was substandard in so-called
managed care or HMO settings versus fee for service settings. So at least by
these parameters, that certainly is not the case.

The next one is an example of a study done by an extramural researcher
which looks at trends and use of adjuvant deprivation therapy among men with
prostate cancer. This was study that was shown in clinical trials to have
potential benefit in the early ‘90s. So the question is, what is going on
with this treatment, and this is one way we can track the dissemination of this
kind of treatment. This is the kind of treatment we can’t capture very well in
our routine SEER data collection, for example.

Then the next one is dear to my heart. This is an example of estimates of
cost of treatment as defined by Medicare payments. We can have a long
discussion of whether Medicare payments are a true or not true proxy for cost,
but compared to almost anything else that is available they are pretty good.

This is an example done by an extramural researcher. We do a lot of inhouse
research on this topic. I won’t go into details, but it is interesting because
it shows that the incremental costs for treating colorectal cancer relative to
if that person had lived and not been diagnosed with colorectal cancer, lived
and died of another disease, it is inversely related to stage. The more severe
stage the lower the cost. In fact, it is a negative cost compared to not having
this disease. So in the context of cost effectiveness analyses this has some
obvious implications that are oftentimes not well appreciated.

This is an example of health disparities done by a fellow who worked with
me, which showed a pretty robust disparity in African-American men in receiving
— not men, just African-Americans, in receiving surveillance colonoscopy after
initial treatment for colorectal cancer. The level of detail isn’t shown on
this diagram. Not only were African-Americans less likely to receive a
surveillance colonoscopy, but they were much less likely to receive a
colonoscopy exam versus a sigmoidoscopy plus a barium enema exam. That level of
details you can get with the Medicare data that would be hard to come by
otherwise.

Finally, Deb Schrag who is in the audience is the first author of this
paper, which is one of many studies that have used SEER Medicare data to look
at the issue of volume outcome for cancer specifically. The marginal is very
interesting, because they show that there is a hospital volume effect, there is
also an individual surgeon volume effect, and the two of them together explain
more than each one of them separately. There has been some very good
descriptive work done using SEER Medicare data, but also some very good
methodological work that suggests some of the limitations of these analyses as
well.

Those are some examples of the types of studies that have been done.

This shows that there has been a growing interest and use in the SEER
Medicare data by the number of data requests and also the number of
publications.

I don’t know if we need to repeat this one. The advantages of SEER Medicare
are obvious from what you have heard from several speakers today. The link to
SEER does provide very rich clinical information at the date of diagnosis which
you couldn’t get from Medicare alone. Going to Medicare allows us — despite
all of the problems that were mentioned earlier, nevertheless it provides a
pretty good longitudinal set of data that we can then follow from the original
incident cohort that we define in the SEER data.

The limitations are legion. Most of those have been mentioned. It is
limited to individuals over the age of 64. Our cancer diagnoses are not. We can
link someone diagnosed prior to the age of 65 once they get into the Medicare
program later on. We have used that to great effect, actually.

There is a problem with HMO enrollees that you know about, and Part D, we
will see what happens. A lot of people would like to use SEER Medicare to look
at issues of cancer screening. There are limitations about that, because the
Medicare codes for things like mammography and colonoscopy don’t always allow
us to accurately identify whether the exam was a screening exam or some other
type of exam. The SEER registry areas are not totally representative, although
they are large, they are national.

Probably the biggest limitation in our viewpoint, and this has also been
mentioned, is the time lag. We have a significant time lag between when the
events occur and when we can make the data available, three or four years. We
would definitely like to do something to shorten that time lag, and we are
doing everything we can to explore potential ways to do that.

We are in touch with CMS and with NIH about various efforts to improve the
timeliness and efficiency of using CMS data for research purposes. We also have
other efforts that are complementary, for example, we do have a large HMO based
research network. We had a grant and we recently received a fundable score, and
its basic purpose is to create a HMO parallel system to the SEER Medicare data
using these large HMOs.

This maybe speaks to some of the early concerns about how do you get SEER
Medicare data. I won’t go into any detail on this, but maybe we can talk about
it in the discussion. This is not a public use database, but it is data that is
available to outside users upon filing a data use agreement that has a certain
amount of requirements that we then enforce. We have not found it to be a big
problem so far. We have had lots and lots of users and we haven’t had any cases
of the kind of abuse that would cause problems.

DR. STEUERLE: Thank you both for rushing through and taking time to let
your colleague John make the last presentation. So John, we are going to let
you go next.

Agenda Item: John Drabek, Economist, Office of the
Assistant Secretary for Planning and Evaluation

DR. DRABEK: My name is John Drabek. I work on OASPE and HHS. I have a few
comments about one of the studies that was displayed in Jennifer and Chris’
slides. That concerns the linkage of the national health interview survey data
with CMS and social security data.

This project has taken a number of years to bring to this stage. It first
of all required people from four agencies, NCHS, CMS, Social Security and OASPE
to meet and discuss how to accomplish such a match, that would satisfy the
privacy concerns of all the agencies, but yet yielded a database that was
usable for researchers. We were able to do that, and we were able to agree on
the technical details of how the match needed to be conducted and how the files
needed to be sent back and forth between the agencies.

The data are now available at the NCHS data center. There is extensive
documentation on the NCHS website under the data linkage page. There is a large
amount of data that are available. It is four years of the national health
interview survey linked with extensive Social Security and Medicare histories.

The data are useful from the standpoint of linking people in the HIS to
those administrative record sources, but also surveys that are built off the
NIS are also available to be analyzed, particularly the LSOA with its followup
interviews of aged respondents. Although MEPS started being linked with the
national health interview survey late in the process, the potential to link to
MEPS is there as well.

There are a couple of things I want to draw attention to. One is the fact
that while the HIS is well documented in terms of user friendly files and lots
of help in terms of being able to analyze the data, the Social Security and
Medicare files are more oriented around program operations. So learning how to
use the Social Security and Medicare files is a level of complexity that is
considerable on top of the HIS.

To help in this process, our agency, OASPE, provided funds for the
development of analytical files for the four HIS merged surveys, and also
developed documentation. We are providing additional funds this year to make
further support available to users.

Gerry Riley, who just presented the SEER data linkage, has done one study
with a linked database, looking at people in the waiting period, those who have
received SSDI and are awaiting their two-year interval before they are eligible
for Medicare, and looking at their characteristics and seeing how they differ
from other individuals. So that is a unique project that could only be done
with this type of database.

We are also in the process of awarding a research contract to use the
database to look at three areas. One is understanding the interaction of the
population with disabilities and the SSI and SSDI programs. There are many more
people who have disabilities than those who are in the Social Security
programs, and what their characteristics are and severity of disability and
things like that are important.

The second area is to take advantage of the fact that we have a survey, say
the disability survey conducted in 1994. We can go up to 2003 and look at what
happened to people after that in terms of their interactions with Social
Security. So we can see how many people who had conditions in ‘94 applied
to received disability benefits in later years.

The third area is understanding family and caregiving support for people
with SSI and SSDI and other disabilities. There is extensive questions about
disability and use of services, household structure and things like that in the
‘94 and ‘95 HIS that go beyond the normal core data in the HIS.

We have supported this, because we think it is essential to have good data
for policy research and program evaluation. Linking databases in this manner is
a relatively new activity, and the design and use of the products is not well
understood at this point. We feel that only by conducting analyses with the
linked data that has been conducted so far will we understand the benefits and
the limitations of what has been done so far.

In particular I think we need to get a better handle on the different time
frame available with various parts of the database. We have the health
interview survey in 1994, but we have extensive data before and after from the
case history files. Figuring out how to summarize those case histories and
integrate it with the survey data is a challenge that hasn’t been faced before.
Similarly, if you have longitudinal interviews, how you integrate that with
case history data. So hopefully we will learn better how to do that in the
future.

Thank you.

DR. STEUERLE: Thank you again for helping us to get back onto schedule. I
now have to apologize to everybody; we will probably have to let you take your
own break again today, because I want to make sure — we have Howard on at the
last session, and I want to make sure we give him his full time. So I am going
to go right to questions, and if people feel you need to step out, please do
so, and we are just going to continue straight on through.

I am going to take the liberty of asking the first question, and I can
address it to any of you. As I heard your presentations, one thing that stood
out in my mind, and I was comparing your presentation to those of the Census
Bureau, it seemed to me that one thing that seemed to make a huge difference
was whether the particular part of your agency or particular program you are in
or whatever had positive incentive to do something.

So the Secretary decides he wants a Decide program. That creates an
enormous weight within the Department to go out and make things happen, which
had he not done that might have much more left it up to some outside researcher
pressing to have something done, which maybe the inside lawyers would say we
might get into trouble for doing this.

The SEER program, I’m not sure what the underlying legislation impetus was
there, but cancer has always been a hot topic. So it appears that if in the
system somewhere there is somebody who creates this incentive that information
should be provided on something, that it often is the motivation for breaking
through so many of these barriers we are talking about. Am I correct or
incorrect?

MR. COHEN: You really are on target, but let me give you two examples where
you are on target. The integration of the MEPS and the HIS were part of a major
initiative within DHHS, I think it was ‘95, as part of Vice President
Gore’s RIGO-2 activity, reinventing government. There was an initiative to look
at all the departmental surveys in a very short period of time, and that
created opportunities and synergies, and that was a catalyst.

More recent activities, there are Secretarial initiatives. The Department
gets a lot more efficient by bringing the core forces together and getting it
rolling.

With that said, it has a momentum of its own. There are other
opportunities, once these things are put in play, that other users see things
that weren’t on the table before, but because of this coalescing there are
these opportunities that present themselves.

I would say there are probably other cases where you have independent
forces coming together, but there is a framework that makes this all jell when
it is a mandate or an initiative and you get the support that is needed from
the leadership to bring people together, because we are all so busy meeting all
our objectives. That is first and foremost, but many times you get these
opportunities.

DR. MADANS: I guess we have had a different experience, although the
MEPS-HIS link clearly was something that was started by the Department and
certainly encouraged.

I think the other linkages that we have done actually haven’t been a real
push. We have had a lot of support from the Department, particularly from
OASPE, to do them. The earlier links with Medicare data and the original
thinking about it I think was coming from the agencies. We have a lot of
conversations among the agencies in looking for enhancements.

But that said, it is much easier to do if there is a structure within the
departments that help you do it and encouragement for it, so that has been very
useful.

MR. BROWN: I think the fact that we have a surveillance mandate, NCI was
important. We said should this mandate just cover incidents in survival or
should it cover expenditures. It was expenditures that was first published,
because we also had constant demands from Congress and still have constant
demands from Congress to answer questions like how much does cancer cost, what
is the economic burdens of cancer. As great as MEPS is, and we use MEPS
actually and that linkage asks for quite a few studies, it doesn’t have the
numbers to get at specific cancer sites.

Some of us, and Gerry was there, Ed was there and other people were there,
said this is something we need to produce. Then later on, once it was on its
own, and it was a struggle to get this thing moving to the point where it is
now, the whole quality of care tsunami struck. Then it turned out there was a
huge interest and a lot of extramural researchers have taken advantage of this
to use the SEER Medicare database not for what we thought was going to be its
main purpose of looking at expenditures, but looking at cancer care and health
disparities.

MS. TUREK: One interesting point. You can get the linked NHIS Medicare file
and do analysis, so you can analyze how similar questions were answered on the
two surveys. I get it for income. I thought it was wonderful as a user, and it
is in public use form. I just wanted to commend both organizations for making
that available in a way that protected confidentiality, but really enriched the
data set a lot.

DR. STEUERLE: The speakers are also free to ask questions or comment on
these others if they have something that they felt was missing or needed to be
added.

DR. SCHRAG: I am an end user, so I am going to ask my question from that
perspective. We heard about two examples of data sets that are linked to HIS
and Medicare and SEER and Medicare. But of course, what end users really want
is to be able to link all three.

So two questions to put on the table. We want to link data sources. There
are many examples where two agencies have gotten together and created a useful
link, and Medicare is most frequently at the nexus, but we want to link three
and four, which gets a bit more complicated.

The other challenge I would put on the table is, we in the extramural,
extra government, extra academic research community often want to link agency
data with data that I will call — they are non-governmental, but they are
extremely informative. We heard a couple of examples, AHA data, data from the
American Board of Internal Medicine, about where different kinds of doctors are
located and what their qualifications are. Those would be the biggest examples
probably.

But there are whole different sets of rules that govern those private data
resources from the governmental resources. As an extramural researcher, it is
very difficult to navigate those waters when you are trying to link data sets
that cross public-private boundaries. I think that is where some of our
greatest challenges lie.

The only time we have been successful at it is when we find cooperative
investigators within the agencies, like Martin’s branch in particular for those
of us in cancer, who recognize the importance of those private sources and
accomplish the linkages for us, so that we can get our grubby greedy little
hands in there, carefully of course, with confidentiality.

Those are some of the challenges that we face. It is great to hear about
two, but we don’t want two sources. We want three, four.

DR. MADANS: The HIS link is the three, because it is SSA, CMS and us. So it
is going across.

We did do SEER once. It was the HANES-1 I think. But SEER isn’t everywhere.
So by the time we did the link we were down to no cases. So we don’t have a big
enough sample in any state. That is part of the problem. There are lots of data
sets, but there are probably not that many that would meet your particular
needs.

I think you are right about the proprietary data. If someone comes to us
and says we want to take your survey and we want to link in our proprietary
data, they can do that in our data center. But they have to make the
arrangements with the other entity, and that means you can’t share it, because
they are paying for it.

That would be an area where it would be nice to have some government wide
leadership so that we are not negotiating those things over and over again.
Some of the AHA stuff we buy so we can put it out. We know other people have
linked in. But you’re right, it gets very complicated.

DR. STEUERLE: Any other questions?

DR. BREEN: I have one question.

DR. STEUERLE: Yes, please.

DR. BREEN: This is a question for Steve. You said that the sample frame for
MEPS from the NHIS was at the health level. I always thought it was at the
person level, so that you had all of the data that you asked to sample adults
included in your MEPS database. Is that not true?

MR. COHEN: It is actually a household based survey. It is person specific
measures that would be factors that we would look at. But MEPS is a household
survey, so you could reconfigure once we take the household in from HIS to
person based analysis, to health insurance eligibility units, to tax filing
units. You could reconfigure nuclear families, secondary families.

I think one of the difficulties and challenges is when on gets to something
like a condition over sample, where you have a sample adult in the household.
It is a bit tricky that everybody in the HIS didn’t get the same set of
questions, so it is a little different than sampling on what is in their core,
in terms of some of the demographic, the insurance, some of the other measures.
But that said, once we sample a household, everybody comes along for the ride.

DR. BREEN: Thanks.

DR. SONDIK: To add a point to the answer to your question, I think one of
the things is that all of these areas have a very strong analytical staff
behind them. So this is not something that is, here it is, here is the
directive, go do it, and you get people whose interest isn’t primarily in this
area and using it. I think that is really important.

The other point is that Martin mentioned, and it is certainly true with
AHRQ, unfortunately it isn’t true with NCHS, but I wish it were, which is a
support of research funds that can be earmarked to build a user community which
can then spread. I think that is really important, because the evaluation of
these things is in the doing. Even though we have strong staffs involved, the
real evaluation is when you get people outside using it as well. I think that
is really an important point.

So if we did get such a directive, I would want to be sure that the
directive as the promise of funds along with it. It doesn’t have to be a lot.
We are talking about tiny amounts of money here, relatively speaking, that can
seed the research community.

DR. STEINWACHS: You were talking about linkages. Medicare is an ideal
population, Medicaid for some reasons.

One of the health issues that keeps coming up over and over again is the
effects of people going to war, which is potentially the veterans. I know the
HIS asks you about veteran status, but I don’t know whether anyone has explored
trying to identify who the veterans are that are part of surveys.

Then there is also VA utilization, which probably would be a small piece if
you linked with HIS, I don’t know. But they are among the people many times
with heavy disabilities if you were focusing on individual disabilities.

So it is a question whether any one of you who have been dealing with
health issues have tried to bring the VA into this, either on cancer or any
other issues.

DR. COX: I can sort of answer it. Before the VA event, we were in
negotiations with them to try and link with their population of VA data to the
HIS. I think we had some analysis, about ten to 15 percent of the HIS had
served in the armed forces at some point in their lifetime, and we were going
to add data about the dates of service and whether there was active duty in war
situations.

We were very close. We were actually talking about the amount of money it
would take to do that kind of work, and we stopped. So we are letting VA get
their IT policies into place and recover from the situation they got themselves
in. I think it will come back at some point. But they need to get their house
in order first.

MR. BROWN: We have two experiences. One is, we have a project called
Cancors. It is a large study that is recruiting 5,000 newly diagnosed
colorectal, 5,000 newly diagnosed lung cancer patients, and have been following
them intensely over a one-year period. VA is one of our sites for recruiting
patients.

I don’t know if we have any specific information on their status in regard
to their military experience, but they are a participating site in that study.

The other, over the years we have had several discussions not with the VA,
but with the military health care system, the Tricare system. It has been
disappointing mainly, because of this problem of not being able to identify the
enrollment status in secondary and tertiary payors. So it is hard to set up
anything that is any kind of longitudinal study if you don’t know who your
denominator is or where they are in any given time. But we have some continuing
discussions scheduled on that topic.

DR. STEINWACHS: Just out of curiosity, I assume in the SEER registries
there are veterans that may or may not have been contributed by the VA as a
source. Is there any information collected in SEER about veteran status as
described here as having military service or service during wartime?

MR. BROWN: Not that I know of. But you’re right. Some of the reporting
units to SEER are part of VA, but there have been some issues lately about that
as well, which we are negotiating.

MR. PREVOST: At Census prior to the event, we were having lots of
discussions with the VA as well. If one were to use our surveys in trying to
attribute status in a linked environment, there were many concerns that the VA
had about how well veterans are responding to our questionnaires and saying
that they were veterans, particularly if they were in a reinstatement status.

So we were going to begin to start doing some quality assessment work as
well, to see how well the veterans were responding to those questionnaires and
if they could be used. But anyway, I am hoping that at some point in the near
future we will be able to re-start those talks.

DR. STEINWACHS: There are Americans at any given time, a lot of them
overseas, and some of them are there for periods of time. Do they get captured
at all? A household interview would not, you have to be a resident of the
United States, but where you talk about births and deaths, they have to occur
in the United States, too. But if you are in the military overseas, would that
be back into the U.S. in terms of data if you die?

I was just wondering what that link was, because some people are over there
serving us, working for us. Some people are over there on their own choice.

MR. COHEN: For some of those core surveys they would have had to have been
unfortunately an injury, and they would have had to return to the States.

But I can tell you, issues like this are coming to the table in health
care, although it is very, very rare. You are hearing of people going out of
the country to get health care. There were tradeoffs in terms of quality, but
there were some costs, to several disparate countries. If this becomes a bit
more prevalent, I’m sure we will want to track this and get a sense of this. So
right now, we don’t have the mechanisms.

DR. STEUERLE: I would like to thank each of you, and just remind you that
this committee forum is pretty much of an open forum. We invite you to stay for
other sessions. Also, remember to keep in mind that the goal of our committee
is mainly to advise. We mainly advise the Secretary.

So if there are things you think we should be advising, please let us know.
I have worked many years in government, many years outside. I have often viewed
the role of advisory groups or consultants as finding the information that is
already inside, and just rewriting it and replaying it a little bit. The good
ideas that you have had that haven’t quite made it to the top, our role is to
help them get there. So thank you again for your patience, and thank you for
especially dealing with the fact that we crowded you into this one session.

Howard, I think that leaves you the end game.

Agenda Item: Social Security Administration – Howard
Iams, Research Associate

DR. IAMS: Yes. It is always a challenge to be the last on a long day. I
will try and cover the topic and do it relatively briefly.

I need to alert everybody in truth in advertising that I am an advocate of
using linked survey data. I have been actively doing this since 1984, and I
started using the new beneficiary survey in 1986, SIPP matched survey. The
phrase end user, I have been an end user and co-authored at least a dozen
analyses, a number of which are published in the Social Security Bulletin.

I work for an agency that is a proponent, a strong advocate. The Social
Security Administration in the 1960s started a survey called the retirement
history survey, which had administrative data from benefits and earnings linked
to it, and followed people into retirement over a decade.

The Census matched data was started in the early ‘70s. I believe Fritz
Shurren who is in the audience and Dorothy Projector were heavily involved in
starting that work, and it was continued by Denny Vaughn, and I continued it
after Denny Vaughn left the agency.

We just awarded our ninth retirement research consortium. We awarded
roughly 70 or 80 projects, about $5.5 million to three centers at NBER, the
University of Michigan and Boston College. At least half of those studies are
using the health and retirement survey and a fair portion of them end up using
matched survey data. So my agency not only started it, it continues to fund and
support it, and we have an analytical group that continues working on it.
Kelman Rupp who is in the audience has been active with the financial
eligibility model which we developed with matched data. We use that for
modeling SSI, for doing Quimby, and we recently were doing estimates on
Medicare extra help eligibility and what was the participation rate of people
in Medicare extra help.

I have been active involved in using SIPP matched data to project the
future retiree population, the baby boom, which has been extended through the
21st century. I call it modeling income in the near term. John Sabelhouse at
CBO says it ought to be modeling income in the late term, MILT, but anyway,
this has been providing social security reform information to the White House
in designing alternatives for over two years, and also to the House Ways and
Means Committee, and to the GAO. We are actively supporting it. We think it
does a lot of things.

We currently have linkages to the Census, SIPP and CPS. You have heard a
lot about those linkages and how that works. We have supported data
improvements for SIPP and CPS. We are currently supporting a Census effort to
get more matched data for the 2004 model panel using administrative address
record process, which you heard about this morning. They are in the process of
contacting people, and we are contributing funds to support that.

We also enhance the health and retirement survey and try to promote it as a
data set that is of use to the community. We have funded enhancements. We are
currently funding a projection of lifetime pension wealth and lifetime social
security wealth using administrative records, which will be released to the
user community for people in the early baby boom. A similar activity was done
for people born in the Depression.

We have studied bias from using only matched data. We find there is a bias,
and we are going to try and promote or create something for the health and
retirement survey to allow the analytical community to deal with that bias.

So I think we are very active in pushing this and supporting it. We have
tried to create a user community. I’m not sure I mentioned, but at least 158
HRS studies have been funded by us, and many of them are using this stuff.

What future directions? That was a question that was raised, future
directions I see. We are trying synthetic data. There has been an effort with
Giana Baud in Census to cerate administrative data for the SIPP that would be
released publicly. It involves all the SIPP panels for the ‘90s.

I’m not sure how well it will work. I suspect it will work for some things
and not for others. I am somewhat skeptical about its use for the decision to
retire and take up social security benefits, for example, which can be an
idiosyncratic kind of thing, as opposed to measuring general wealth and
well-being.

I think we are going to see — I mentioned the assignment of linkage from
government address files which Census is doing. That is being tried for the
first time in the SIPP or CPS. That was done on ACS, but I don’t think it was
done on SIPP or CPS.

There is a broader health and retirement survey consent form that allows us
to release more administrative data. Previously the lawyers who negotiated that
kind of stuff made things very narrow, and would only allow release up to the
moment that the consent was signed. We now have an agreement that the consent
will include matching through 2021, which means that we will be able to follow
these people forward into the future, which is what we currently are doing with
the SIPP.

The 1984 SIPP, for example, those folks were interviewed a long time ago.
We have earnings records, yearly earnings records through 2004. We have social
security benefits through December of 2005. We have SSI benefits paid by Social
Security through December of 2005. We keep updating our administrative file.
Each spring we go and pull in another year of data, so we are constantly
creating a very useful longitudinal data set from our administrative records.

Possible future linkages. I would love to see a linkage to the 1040 tax
records, so we can identify the validity of some of the income being reported.
I would love to see a match to the national compensation survey, which is a
survey of employers about their pension plans. Potentially Medicare bills.

I did that once with the new beneficiary surveys. As Gerry said it was a
very painful process because of the dirty data set. So I don’t know, I would
probably have to be pushed to do Medicare bills.

We currently have linked a lot of SSA records. We have a master beneficiary
record, which is our Title 2. That is what most people call social security.
For the disabled, we have the latest disability insurance information,
including the first and second diagnosis of why someone was disabled. This is
close to the ICD-9. Social Security has some slight differences.

I would just point out that in order to get disability from Social Security
you have to have been examined by a doctor and proved that you are unable to
work, that this is a medically determinable condition that lasts 12 months or
will kill you.

It is a growth industry. We now have eight million beneficiaries, 6.2
million of them are disabled workers, 154,000 are spouses and 1.6 million are
children of workers.

So this is a fairly concrete establishment. If somebody has poor health at
this point, roughly a third of the folks are mentally impaired. Back in the old
days when we did the new beneficiary survey it was ten percent. So that is
quite a shift.

The SSI record, Title 16, we have every month the benefits since the
program started. We have seven million SSI people. Four million are working
age, two million are aged, and about a million are kids.

We have the Numident. Census tells you about the Numident for giving you
the social security number. We use it for data death. That is where death is
recorded so we can tell death when people have departed our data set.

Summary earnings record, that is the record of — actually, you never
depart our data set. You always are in our data set. You depart active
updating, shall we say. The summary earnings record is the record of earnings
under social security taxes. That is what these people will receive their
benefits on. That is what they are going to get paid from, so you have the real
thing.

The detailed earnings record is our phrase for what is the W2 tax record.
We have the summary of earnings record from 1951 through 2004; we have the
detailed earnings record from 1982 through 2004. Recently researchers have
discovered that they can pull out the amount of money that has been deferred
for 401K type pensions. It is measurable on the form. They are starting to
measure it, and it is a much more valid source of information on participation
in defined contribution pensions than self reports and retrospective
information might be.

We do have Medicare Part D extra help application information. It is
feasible, but we haven’t actually matched it up. I think that basically is it.

So we have what people are being paid by an agency that pays 46 million
people benefits in Title 2 and in supplemental, seven million. We have what
they really have earned, according to what is filed with the tax authorities.
This is a fairly useful data set, and we run it longitudinally for all of the
surveys we deal with, which is SIPP and CPS and HRS.

Rule for use. You have heard the Census rules for use fairly extensively,
so I won’t mention that.

The HRS is widely used because the researchers can do it in their own
location. They have to have an approved research project. They have to have a
federal grant or use a secured data center at Michigan; most get a federal
grant. They have to have an isolated computer, they have to agree to
restrictions on access and public release. They can’t use geography with the
SSA records except at the Michigan secure site. Social Security funds the HRS
to send out a contractor to evaluate whether on an unannounced visit the use of
the computer and the data are being complied with. They do get visited even at
government employment.

Validity and reliability. I think that it is hard to doubt that our record
of what you got paid for SSI and our record of what the Treasury issued to you
for payments in terms of social security is better, is more reliable than the
self reports. We have done several studies which are in the SIPP working paper
series about the extent of bias and misreporting. It is not extensive, but it
does have an effect, particularly on the SSI.

I think another thing that the data provide is that people are ignorant of
a number of the details of the social security program. For people doing
retirement research, sometimes those details are very, very important. I
highlighted one thing that the man on the street would say is important, how
much money did you contribute to your pension account. I doubt there are that
many people who remember the dollar amount in the last 20 years.

Another reason is, it is a fairly high cost thing to collect longitudinal
data. These are fairly high quality data, earnings 1951 to current year, W2s
1982. The SSA benefits start in 1962, the SSI benefits in 1974.

What is new? We have got this publicly synthesized administrative data
which is somewhere in disclosure review. If the world works right it will be
available in the next couple of months. For the health and retirement survey,
we have a much more complete release of administrative data. A past practice
had been to release a selected set of items. I think it was way too minimal.

We had evolved at Social Security a whole set of files that we developed
for the SIPP and for our modeling. We are now releasing most of that
information with the health and retirement survey. It was sent out to Michigan
for the 2004 interview in the spring. There are some users who are starting to
use it now.

That’s it on that.

We were supposed to talk about barriers. I didn’t put barriers in my thing,
I would just as soon not write it down. I don’t think there are any barriers
internal to my organization. The only issue with my organization is that these
records come from administering a program. They are not created for research,
they are created to administer the program. So in order to use the records very
well, you have to understand the program, and there are not very many people in
the research community who understand the details of the program.

I think if you stick to how much was the payment and how much were the
earnings, and when did the benefits start, everybody is going to be okay, but
they do have their idiosyncracies. We are doing internally an effort to
document all these files and all these data. The last version was yay thick. We
are updating it, and we should have this in the next six months. This effort
has been going on for about nine months, involving specialists in the MBR,
specialists in the SSR, specialists in these different files, and we will make
that available to the research community who wants it.

Another barrier is, there is a public reluctance to provide social security
numbers in these surveys. The last HRS got a 50 percent response rate for this,
the SIPP got a 60 percent. We are going to fund HRS to do a better job and
perhaps that will enhance things, but it is a problem. These are such valuable
data, it would be tragic not to have them.

There is a difficulty getting other agencies to let us use their data in
our matched data records. I think I expressed this morning, I consider our data
security equivalent to breaking into Fort Knox. I really am serious about that.
My agency is obsessed that the public can’t get access to peoples’ earnings and
benefits records. I think that the computer security is incredibly difficult to
break through. We take much more advanced methods. Anyway, it is just very,
very hard.

So I have a hard time understanding why we wouldn’t be able to bring in a
data set and match it up for statistical purposes. For three years we have been
trying to get the Bureau of Labor Statistics to let us use their national
compensation survey. We first tried to get a copy to statistically match, not
an exact match but a statistical match. We did it on our own. We worked through
Census with SIPP-C. That all has failed, so now they are drawing up some
agreement and our lawyers are talking to their lawyers, and I don’t know if
that will ever occur.

For three years we have been trying to get the 1040 records for a year of
the CPS and a year of SIPP, so we can assess the validity of reports of asset
income. Our statistical measures believe that there is an under report. The
percentage reporting asset income is under reported by 15 percentage points.
This is a very big deal to Social Security, because there is a statistic we
calculate about how many people rely entirely upon Social Security for their
income. If you just had one dollar in asset income you wouldn’t be in that
statistic, and we think it would cut our level in half if we had valid
information. But we have been unable to get that, for some reason.

Disclosure you know is a problem. We use secure files. We support
financially the synthetic data. We have made every effort to try and get the
synthetic data to work. It would be nice if it would, I don’t know if it will.
We expect to have a contractor evaluate its utility when it finally is made
available. So I think those are the barriers that we face, and those are the
strengths and weaknesses of where things stand. I will stop.

DR. STEUERLE: Questions, comments?

MR. LOCALIO: You made an interesting comment about, their lawyers are
talking to your lawyers. One of the problems I have felt since I have been on
the committee for the last four years is that the lawyers never talk to the
data people. There are some good reasons for this. Lawyers tend to hate data,
they don’t understand data. On the end of my tag here, what I used to do in my
former life, before I climbed the data mountaintop and saw the promised land.

But I am finding that there are these lawyers out there who make
pronouncements and write rules and regulations and policies and do not touch
base with the people who have to use the data. Then they don’t understand the
practical problems or the implications of what they do. I don’t think we have
any agency lawyers here today, is that correct?

DR. IAMS: I want to defend our agency lawyers. We have been having to go
through the agency lawyers on interagency agreements now for about two years.
We are able to communicate with them fairly well.

This last one with BLS, maybe you are right, because I don’t think they
understand SIPP-C at all. It is partly under SIPP-C and it is using SIPP-C
language. But they were able to call over and talk to each other and come up
with some sort of agreement.

I noticed that ELS wants to review every piece of information that is
created by my agency using their data. If that means that they have to approve
what goes to the White House, I don’t think we will have a deal.

It has been a very long and drawn-out process. For the most part I can
understand what you are saying.

MS. TUREK: When you say synthetic data, I think of something polyester.

DR. IAMS: It’s close.

MS. TUREK: What happens? How do you create synthetic data? Do you try to
maintain the same distribution? If a person is in a program, do they stay in
it? I suspect that we could have a whole conference on what do you mean by
synthetic data.

DR. IAMS: When it was first sold to us four years ago, they were going to
statistically change the survey data and keep all of our administrative records
intact and whole.

What is now being reviewed by the disclosure board has statistically made
up every single one of our administrative records, and all of the survey data
except three data items. So you will have to go review the work of the
statisticians who did the econometric modeling, that says that they think they
retained the relationships.

As a multivariate analyst type, I am dubious and have been dubious, which
is why I wanted what they originally said they were going to do. But in order
to meet disclosure review, they got pushed and pushed and pushed until now it
is all statistically made up, for the most part.

DR. STEUERLE: I should indicate, at Treasury for years we developed — and
still we had to use, I don’t think they called them synthetic data, but they
were statistically matched data sets. So they weren’t exact matches, they were
statistically matched. Tom Petska who will talk to you in a second has been
involved with that somewhat.

The notion was, you preserve both the variance and covariance matrix for
the original administrative data set and say a survey data set you had, but the
link you used some sort of transportation algorithm. You have a cost function,
you minimize the distance between wages or interest, stuff like that. Then you
hope that the things you didn’t minimize on were okay, which is where I think
Howard was becoming quite skeptical.

But sometimes, if you are in the government and you have to make cost
estimates, or you are in perhaps Medicare or some other place, you have to have
a more complete data set. Sometimes we have no other choice.

DR. IAMS: I think it will work very well for certain kinds of research
questions. If you want to know if someone earned a lot of money in their
lifetime and worked a lot of years, these data should be just fine. They are
probably pretty close to the original.

If you are trying to do what a lot of labor economists are doing, which is
predict exactly when someone applies for social security, when do they leave
the labor force, that point of decision is going to be statistically smudged.
I’m not sure it will be as good to research that subject.

DR. STEUERLE: Another way to say this, we have no idea what the standard
deviation on any variable you create is.

DR. PETSKA: I’d just like to say a couple of things. Gene, that is exactly
right. Treasury is still doing statistical matches. Treasury has a very strong
interest in the CPS, and they cannot get the identifiable version of that, so
they have been doing statistical matches for many years, back when Gene was
there in the Tax Reform Act of 1986 and so on.

But in regard to your comments about access to 1040 data, I don’t know if
you were trying three years ago or for three years or whatever, but I would
certainly be willing to talk more about that. As I think you know, I will be
talking tomorrow morning at 9 o’clock in the first presentation. Access to tax
data is closely regulated by Title 26, 6103 of the Internal Revenue Code. It
requires an authorized statutory purpose.

Social Security is clearly in the code, but the question is, what about the
purpose? Even though you get access to certain files, what about other files
and the content? We agonize with Census Bureau repeatedly, and Gerry can vouch
for this, about not only what files, but literally an item by item basis. There
is a regs request to expand the items that is pending right now. The Bureau of
Economic Analysis has the same thing.

So we can certainly talk about that off line and see where it is now. It
has gone from your lawyers to our lawyers, and that is why it hasn’t gone
anywhere. Maybe reason can prevail.

The last comment I just wanted to make very briefly was, for such things as
W2 data which go into earnings histories, there is a joint custodianship
relationship. This is shared by the agencies. So if you appeal to Social
Security to get access to the earnings histories, that has to come to us as
well because those data are co-owned by the two agencies.

DR. IAMS: We have had a very amicable and longstanding relationship in
dealing with those.

MR. RILEY: One additional on the synthetic data. As we try to match
administrative records data to survey data, this is the problem. We can either
release no administrative record, we can force people to come to an RDC or come
to the area or figure out some way to get remote access, or we can create
these synthetic files, which is one of the things I think they were trying
to do with the SIPP.

One of the big problems with creating a synthetic file after the data set
is released is, you haven’t done any top coding to begin with, and you are
trying to add these other data on, and it makes it much more complicated. So I
think this is one of the things we are trying to do with the new system, is
design it such that we know beforehand what is going to be released and what is
not going to be released.

This is also one of the issues the CENSTAT panel, we have asked them to
look at, are synthetic data good, what is going on with them, can we assess the
quality of them in addition to the administrative data. So this is one of the
things that really need to be looked at, which is why we need to get the data
set out, so people can review this and get some information.

DR. STEINWACHS: I was just curious from the researcher side. Are there
examples in which the analyses of synthetic data sets in terms of policy
analysis have been done and published in the peer review literature? Or is
there a pushback from reviewers about, once you indicate it is a synthetic data
set versus quote-unquote a real data set?

MR. RILEY: I think there are some examples. John Ebatt has a number of
papers. They have used both the synthetic data and the actual data.

We currently have a variable that we have looked at, the multi-benefit
analysis, which is from the security records, matched to the 2004 SIPP data to
compare the mean. The key is the distribution in the covariances. You look at
the means by about 20 different characteristics, and they are very similar, the
administrative data versus the actual data.

So this is the stuff we are looking at. But I think if you go to the LEHD
site on the Census website, you will see some papers that have tried to
evaluate the synthetic data.

MR. LOCALIO: I want to follow up on that. I have seen the technical papers
in the field, but I don’t think I have seen an applied paper presented using
synthetic data.

I can tell you right now, if I saw one, I would be open minded about it
only because I have read the technical data. But I would say that it is well
beyond the capability of most journals to review papers like that. It is too
new. It hasn’t been tried out enough.

MR. RILEY: Applied journals.

MR. LOCALIO: Applied journals, that is correct.

DR. STEINWACHS: This may take you in a little bit different direction, but
when we all talk, we always think about the social security number as the thing
to link on if you can. There have been a number of comments about the
difficulty of collecting it and so on.

I am just wondering how good the social security number is. If you say
there is one person for the entire life, I always think of the witness
protection program, where I assume it changes. I know that is a very small
number hopefully of people. But there must be things that look funny when you
look at social security numbers, where people have picked them up, used them,
maybe don’t use them. Some people have two or more of them eventually in their
lifetime. Is that a tenth of a percent that you think are funny numbers, or is
this more pervasive in our system?

DR. IAMS: I’m not in a position to say. I think the Office of the Inspector
General at Social Security has had some statements or studies made about the
validity of social security numbers. You know that it is a concern at the
Homeland Security Administration.

I would say, we have something called the earnings suspense file, which has
millions if not billions of dollars sitting in it, which comes about when
someone’s tax record comes in and something is wrong with the record. It
doesn’t quite match the name with the number, and it can’t get deposited in the
account.

Now, that is not an area I know a great deal about, but this was going on
when I first went to work for the agency back in the 1980s when our agency lost
a quarter of its work force. The suspense file went way up. I recall, at one
point IRS refused to work on it and SSA refused to work on it, because they
were both short of administrative funds in the ‘80s. Anyway, I would just
go anecdotal.

It empirically exists. It is not trivial. If you go to the Inspector
General reports, I know they have had work done on the earnings suspense file.
That might give you a sense.

DR. COX: I can say that when we link the HIS data to SSA data, we did do a
validation step first. While I don’t think it is a huge problem, we see two
areas of concern. One is elderly women who are using their husband’s social
security numbers because they don’t have their own, or they receive benefits
under his work history. We do see lower validation rates for SSNs provided by
Hispanics.

DR. STEUERLE: Howard, turning to an area that you and I have worked on over
the years, one thing you do have in Social Security is an ability to try to
measure lifetime earnings. Although there are gaps, it is probably one of the
best data sources anywhere to look at relative status of populations over long
periods of time.

In fact, the former head of CMS did a study right before he took that job
in which he tried to estimate whether Medicare was distributed in a progressive
manner according to income. I can’t remember the study, but I remember looking
at it and thinking, if anything is a synthetic data set, because that is what
he had, he had that.

You have Social Security which if linked to Medicare could come much closer
to answering that question. So beyond answering that narrow question of that
linkage of Medicare data to lifetime earnings to see studies on the
distribution of Medicare benefits, are there other efforts being made to try to
figure out how to use a lifetime earnings measure of well-being for a whole
variety of health outcomes and measures?

DR. IAMS: I think that our retirement research consortium has a number of
papers and analyses that use lifetime earnings. Because the health and
retirement survey is in the public domain with minimal restrictions, there is a
lot of use.

The economists who study that type of thing were using this when everything
was in the public domain back in the 1960s, before Watergate led to the
restrictions of the Privacy Act. They carried on that type of work.

Now, sometimes they have health as an outcome. Many times they are doing
labor force retirement. Let me ask, Kelman Rupp, do you know of — you do more
in the disability and health area, are some of the people studying the outcome
of disablement or health impairment or something along that line, Medicare,
using a predictor of lifetime earnings?

MR. RUPP: (Comments off mike.)

DR. IAMS: I guess you have tapped a subject for the user research community
to get into.

DR. STEUERLE: Does the SEER data file have lifetime earnings as a
prognosticator of cancer?

DR. IAMS: You haven’t matched it up to earnings records, have you?

DR. BREEN: We actually tried to link it with Social Security Administration
records at one point. Maybe we tried to be too detailed, but not all of the
data were electronic, so they weren’t available for all the years.

Largely, and I think it will change as baby boomers retire, but it was
available pretty much for white males. There weren’t earnings history for most
of the people.

DR. IAMS: That sounds weird. I have been using these data for 20 years, and
that has not been the case when you match it up to SIPP. We have something like
85, 90 percent match to SIPP and 75 percent match to CPS, and we don’t have the
kind of holes you are talking about.

DR. BREEN: What is the age range? Because cancer is a disease of older
people.

DR. IAMS: Our data don’t start until 1951, so we have earnings on people
after 1950. If you were looking at people born in the First World War, there
would be a lot of missing earnings.

DR. STEINWACHS: I am told that our contractual arrangements for the room
end this afternoon at 4:45. You may be thankful for that.

I do want to thank all the speakers this afternoon. This has really been
fantastic, a wonderful learning experience and a great exchange. Thank you very
much.

DR. STEUERLE: Can I just add one thank you to Cynthia Sidney, who helped
organize this?

DR. STEINWACHS: Hear, hear. I hope you will be here at 9 o’clock in the
morning tomorrow morning when we will learn from the IRS how to solve all these
problems. We will move on to the Veterans Administration, education and some
other areas. Thank you very much.

(Whereupon, the meeting was recessed at 4:46 p.m., to reconvene Tuesday,
September 19, 2006 at 9:00 a.m.)