[This Transcript is Unedited]

NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS

Joint Meeting of

Subcommittee on Population Health

Subcommittee on Privacy, Confidentiality, and Security

Subcommittee on Standards

Joint Roundtable on
Health Data Needs for Community Driven Change

April 30, 2013

National Center for Health Statistics
3311 Toledo Road
Hyattsville, MD 20782

Proceedings by:
CASET Associates, Ltd.
Fairfax, Virginia 22030
(703) 266-8402

CONTENTS

PROCEEDINGS

Agenda Item: Welcome and Agenda Review

MS. KLOSS: It is 9 a.m. on April 30. We are very pleased to convene this Joint Roundtable on Health Data Needs for Community-Driven Change. Welcome to NCVHS members and staff and our special invited guest experts. We have been planning for this roundtable for several months. We are really grateful for everyone’s full participation.

The National Committee on Vital and Health Statistics, as you know, is well accustomed to holding hearings, not well accustomed to holding a roundtable in this format, but we had a particular goal of inviting experts to give formal comments, but also to fully engage with us in pushing further on what some of the issues and solutions might be. This is a new agenda for the national committee. We are really pleased to be launching it this morning.

My name is Linda Kloss. I am a member of the full committee and co-chair of the Subcommittee on Privacy, Confidentiality and Security. One of the formalities of the national committee is that we need to introduce ourselves for the public record. I will just say that I am a health information management consultant from Chicago. I have no conflicts of interest. We will ask the other members of the committee to introduce themselves and then I would like to briefly go around and have you introduce yourselves and your affiliation. We will try to keep this moving quickly because we have a very important timeline that we need to adhere here.

I think next I would like to ask our chair of the national committee to introduce himself.

DR. GREEN: I am Larry Green. I am from University of Colorado Denver, member of the committee, chair of the committee. I have no conflicts.

DR. CORNELIUS: I am Llewellyn Cornelius. I am a member of the committee and a member of the Subcommittee on Population Health. I am a professor at the University of Maryland, School of Social Work and I have no conflict.

DR. COHEN: I am Bruce Cohen and a member of the full committee, co-chair of the Population Health Subcommittee. I am from the Commonwealth of Massachusetts, Department of Public Health. I have no conflicts.

MS. MILAM: Good morning. I am Sallie Milam. I am a member of the full committee. No conflicts. Co-chair of Populations, member of Privacy, Confidentiality and Security. I am West Virginia’s chief privacy officer and I am the West Virginia Health Care Authority.

DR. BURKE: I am Jack Burke. I am a member of the full committee and member of the Subcommittee on Population Health, Privacy, Confidentiality and Security. I am the compliance officer at Harvard Pilgrim Health Care at Boston and the privacy officer there as well and I have no conflicts.

DR. FRANCIS: I am Leslie Francis, University of Utah and no conflicts.

MS. GREENBERG: Good morning. I am Marjorie Greenberg from the National Center for Health Statistics at CDC where you are currently located. I want to welcome you here at NCHS on this rainy spring day. I am the executive secretary to the committee.

DR. JEAN PAUL: Hello. I am Tammara Jean Paul. I am staff to the Subcommittee on Population Health. I am also at NCHS, CDC.

DR. FITZMAURICE: Michael Fitzmaurice, Agency for Healthcare Research and Quality, staff to the Subcommittee on Standards and the Subcommittee on Quality.

MS. LOVE: Denise Love, executive director of the National Association of Health Data Organizations and visitor or guest.

DR. CARR: Justine Carr, chief medical officer of Steward Health Care and chair of the workgroup on HHS Data Access and Use.

DR. FULCHER: Good morning. My name is Chris Fulcher. I serve as a co-director of a center at the University of Missouri called CARES or Center for Applied Research and Environmental Systems.

MS. HOFFMAN: Sharona Hofman from Case Western Reserve University in Cleveland and not a member of any committee.

MS. CHRYSLER: Hi. I am Denise Chrysler from the University of Michigan School of Public Health and director of the Mid-States Regional Center for the Network for Public Health Law.

MS. ZAPPIA: Good morning. I am Barbara Zappia from the Greater Rochester Health Foundation in Rochester, New York.

MS. HART: Good morning. Lacey Hart from the Mayo Clinic as a guest today. I am a director of a project management office and I am the program manager of the Population Health Resource Program.

DR. CATLIN: Hi. I am Bridget Catlin with the University of Wisconsin in Madison. I direct the County Health Rankings and Roadmaps program.

DR. BAZEMORE: I am Andrew Bazemore. I am the director of the Robert Graham Center for Policy Studies in Family Medicine and Primary Care.

DR. ZARATE: Good morning. I am Al Zarate, former Confidentiality Officer at NCHS and now a special advisor to Eve Powell-Griner, who is the current Confidentiality Officer.

MR. CROWLEY: Good morning. Kenyon Crowley, director of Health Innovation at University of Maryland College Park and a member of the HHS Working Group for Data Access and Use.

DR. BREEN: Good morning. I am Nancy Breen. I am the National Cancer Institute and I specialize in health disparities and data on local populations and small populations.

MS. KANAAN: I am Susan Kanaan. I am a writer for the committee.

MS. KHAN: Good morning. I am Hetty Khan, CDC, NCHS. I am staff to the Subcommittee on Privacy, Confidentiality and Security.

MS. WEBSTER: I am Kacey Webster with CDC, NCHS and I am staff to the Subcommittee on Population Health.

(Introductions around room)

MS. KLOSS: Thank you everyone. We have a new agenda coming around. There were missing pages in the agenda. That will be distributed to you shortly.

The goals of our meeting are four and you can see these upon on the screen. To advance our understanding of access to and use of data by communities. To refine the work that we have done on a stewardship framework for use of community health data. We are going to provide a briefing on prior committee work on these two points. To think together with you on understanding of the role of government and providing data tools, resources, and promote community-driven change, and understand current state and possible gaps in data content and variable standardization for community data collection and use. We have very broad goals and I think wonderful group of experts to help us chew on these.

Our game plan for this roundtable this morning is to brief on relevant recent NCVHS work. As you will see on the agenda, this roundtable is cosponsored by the Subcommittees on Population Health, Privacy, Confidentiality and Security, and the Subcommittee on Standards. Three of the four subcommittees are participating in this because one of our overarching goals as a committee is to see the issues we are working on converge.

We are going to bring you up to speed on what work the committee has done and then hear from Bruce to frame the roundtable issues, what issues are we specifically bringing forward. And then we will hear from our first panel focusing on the collection and compilation of data as it is now being done throughout the US in communities. And then this afternoon, we will do our first small group work on panel one topics and we will proceed to panel two topics using data for decision making.

All of us are well experienced in group process. Our reason for moving from framing the issues to small group work is obviously to make sure that we all have the opportunity to fully participate. You will be assigned to one of three breakout groups facilitated by our experts here who have introduced themselves. And our plan is to not only have our expert facilitators, but to have a senior staff person help capture ideas and will have a narrative data capture using computers. All discussion will certainly be captured one way or the other. But by breaking this larger group into three hopefully will be all just really fully engaged.

We will report out each of the small group work. And then we will proceed to the next topic. Certainly, there will be some redundancy. We know that. But we are going to be listening very carefully and leading to the work that we will do tomorrow afternoon, which is to move from issues to recommendations. Again, there we will commence with a panel that will help us replay what they have heard through these first breakouts, but also add their own perspective. And then we will have a general discussion on recommendations, which will close out day two.

The morning of day three, May 2, we are going to continue to press for recommendations I think we have found in hearings and other committee work. We do best when we push as far as we can toward understanding what options and recommendations and actions might come out of this while we are here and all together rather than going back to our day jobs and then reconvening three months from now to say now what did we hear. That is why we really made the point of adding that final half day. That is our agenda. It is ambitious. It is well orchestrated. The timing is tight. But I think our subcommittee chairs reserve the right to modify things on the fly as we see how things are working or not working.

We, again, are just really grateful for your participation. We are going to
work you hard. But I think what comes out of this we do not go into with preconceived notions. You are going to help us shape this. It may not be a report. It may be a letter. It may be both. It may be a set of recommendations. It certainly I hope will be new information that will enable and empower the work that you do too. We think this is officially launched.

To move us along, I am going to introduce Sallie Milam who is going to present the work the community as a learning health system and recent work that has been done just this year.

Agenda Item: The Community as a Learning Health System Report and 2013 Environmental Scan

MS. MILAM: Good morning again. Can everybody hear me? We will talk first about the community as a learning health system report. This was an effort that started a couple of years ago to look at how communities can become learning health systems, the resource that they have and the resources that they need. It tells a story of bottom-up solutions tailored to local needs and powered by local talent. We had two very exciting workshops in Washington, DC. One in February of 2011. That was about communities generally. And then one in May that focused on stewardship privacy trust issues.

This is a figure developed by NCVHS over ten years ago. We continued to find
it very useful. As I am looking at it, it is really very hard if not impossible to see the attributes and all the different domains on that chart. But suffice it to say that it reflects all of the different influences on health and it is printed in the community as a learning health system report. I do not know if we have copies today. It is on the table. If anybody is interested in getting a better view of that figure, it is in that report. It is also on the NCVHS website.

We heard from communities talk about the different data that they utilized in their study. When you look at the different domains, we heard that communities use much of that data. They used data off the shelf. They used homegrown data. They used state and national survey data, clinical data, data on food source and physical activity options, public safety, land use, and socioeconomic factors.

We heard that many communities were focusing on factors that were easily amendable to change and factors within their control. We heard that several communities were tackling high rates of chronic disease such as diabetes looking at where there were good food sources and what exercise options were nearby. We heard that a few were linking clinical and population health data.

The presenters also stressed the importance of the qualitative data putting the face on the problem. That helped them prioritize their problems and move their agendas along.

This is a slide representing the learning health system. There are a lot of different attributes that you can see on that slide. You see biodirectional information flow. You see that it is electronic. You see a variety of different partners in a community.

With that, we also identified at least ten common success factors in a
community that has and operates as a learning health system. First, a galvanizing health concern. Second, a comprehensive understanding of health and community health. Three, trust and a collaborative culture, social capital. Four, access to data on local health and its determinants. Five, analytic capabilities. Six, data display and dissemination capacities. Seven, functioning coalitions, community engagement, agreement on priorities. Eight, organizational and technical support. Nine, political and financial support. And ten, processes and systems to translate information and priorities into action, evaluate results and modify as needed.

This slide wraps it up together. We heard how communities were developing data around a very broad definition of health. We saw how they were innovatively displaying their results particularly on the web and dashboards, Healthy Sonoma comes to mind.

We heard from communities how they were building trust, how they were educating the community members, how community members were participating in the data collection, and to some degree analysis. We heard how governance fosters a sense of ownership and control and how it was beneficial to their projects. We heard about the importance of transparency.

We also heard about the needs for an infrastructure in several ways. We heard that we needed a standardized set of community health indicators, training and technical assistance to improve analytic and data management capacities, support our facilitation to strengthen local financial and human resources, better data visualization tools and skills. And we determined a privacy and security framework was needed for communities.

We also identified a number of specific suggestions where the federal government could both harness and seed the energy of community health movements. We have a much longer list in our report, but as you look at this slide, you see a number of areas where we would look to strengthen the local capacity.

We knew that continued work in this area was relevant and we wanted to determine what the sweet spot for NCVHS is. We also knew a lot had happened in the past year and a half. That was at about the December point. We decided it was important to do some reconnaissance. We worked with our partners, Susan Kanaan, to help us really just figure out at a very broad and shallow level what was out there. I think, Susan, you felt that you pretty much identified everything. Although it was not what we would call a scientifically valid research approach.

We were able to get a general feel of the progress that had been made over the past couple of years. We were able to ensure that where we were going was definitely needed and within NCVHS’ mission.

These were Susan’s observation. Susan advised us that it was absolutely a logical priority for NCVHS to help strengthen the bridge between the federal data liberation movements and the data intermediaries, that that was important and that that needed to be done.

As she interviewed folks and researched their projects and their resources online, she identified three themes across these data intermediaries. That they were using data pretty much in essentially three large areas. One was data for social action. Community-driven change. A second area was mapping community problems and assets. A third was fostering a national movement.

Susan also identified that some of the infrastructure gaps that we identified in the community as a learning health system report had been met, but also the gaps that you see on the slide exists today. We are hoping to hear more from you today on those gaps.

Susan also identified that the stewardship framework is still needed and its build out is needed within the communities.

We heard from members that we also needed to do something more. We needed to better understand what else was going on in communities that our colleagues were aware of a lot of exciting initiatives and that at that point and we were early on in 2013. We did not know what we did not know. Again, this is not a scientifically valid sample. This is really a lot of us in our day jobs reaching out to people we knew trying to get a feel for what was going on and where we might focus. We owe a big thank you to Dr. Tammara Jean Paul for leading this project and helping us with the analysis and the slides. Thank you, Tammara.

You will see we had a pretty good response. We have 95 organizations. Mostly
heard from community groups and public health departments. We are not going to be presenting all of the information that we heard. We just picked a number of the slides. I think if anybody is interested — if you responded to our feedback tool and you would like to know what more information is, we can get you that information.

The question is what types of data do you currently use. We asked of our respondents. We heard that government and local resources were the most common and that private sector and electronic health record data were the least common.

With regard to the question, what are the limitations or challenges of the data that you use, the most common limitations were lack of information about population subgroups, lack of enough place or geographic detail, data are not timely, and lack of standardization of the data format.

When asked what data you would use, but you cannot access, the most common response was local/county government data. That was 60 percent. Behind that came federal government and state government data and electronic health record data.

You will see that community members are involved in these three areas of the data life cycle, most predominately in data collection.

Most people collect their own data.

When asked what tools and support you need to better analyze the data, you will see that most people pointed to mapping software. And then the other tools are pretty evenly across and just under the 50 percent range. A lot of different software and federally available tools, web-based tools are needed.

For the folks who collect data, most people feel that they completely protect the identity of the individuals. You will see that that is at 57 percent.

You also see across all of the different controls for protecting identity and confidentiality that overwhelming almost — at 95 percent everybody feels that they protect the identity of individuals and that they also — most people have these other controls in place.

I will pass it to Leslie.

Agenda Item: A Stewardship Framework for the Use of Community Health Data

DR. FRANCIS: Thank you. Before I start, I would like to ask people who have come in since we introduced ourselves. I saw Paul and I think Leah and Len.

DR. TANG: Paul Tang, Palo Alto Medical Foundation. No conflicts.

DR. VAUGHAN: Leah Vaughan, Health Policy Group. No conflicts.

DR. NICHOLS: Len Nichols, George Mason University. No conflicts.

DR. FRANCIS: Are there others who have —

PARTICIPANT: (off mic)

DR. FRANCIS: As we were working on the understanding that we had of how communities collect, analyze, disseminate, use, hope to use data, we were also concerned about the kinds of questions that are raised by issues of data protection and making sure that communities can legitimately trust ways in which data are used.

The hearings that you heard Sallie described, they were really twinned. On the one hand, there were the efforts of the population subcommittee and on the other hand the efforts of the privacy, confidentiality and security subcommittee. Basically, the underlying theory is that it is really important to use this data, but if you run into a train wreck because people are afraid of the collection and use of data, you do not do anybody any good. How can we appropriately enable communities to use data was the question on the stewardship side.

This is the report back to the report. The initial report that we produced had a promissory note in it. This is the quote from page 19 of the report. Further efforts and leadership are needed to define a privacy and security framework to guide the innovative uses of local data emerging in communities across the country. Our subcommittee was working on this side of the effort.

The hearing that we had in 2012, this is the second of the hearing. This is the one that led to the stewardship letter. We heard from some of the communities we had heard from at the beginning, others many involved in data use. We heard from groups in all sorts of different contexts. We heard from academic researchers. We heard from community, academy partnerships. We heard from tribal groups. We heard a vast number of thoughts from people expressing both the importance of using data and certain concerns that had led to notorious cases like the litigation from the Havasupai Tribe against Arizona State University or the destruction of new born blood spots, tremendously important public health resource in Minnesota and in Texas.

Here are some of the critical points that emerged from that hearing. The importance of educating community members and community leaders about data use and benefits. The needs for involvement of community members in decisions about data collection use, and communication of the results of data analysis.

We heard one story where there were risks of stigmatization concerned with how the analytic results were conveyed. We heard about the critical nature of building trust among organizations and agencies that are sources of data. We heard that having governance mechanisms in place foster a sense of ownership and control.

More of what we heard. That privacy is not just an issue for individuals. It is an issue for society, for groups, for family members. What is done with one person’s information may actually affect others in ways the others have not had an opportunity to participate in.

We learned that the chain of trust must involve data from birth to death from the time at which it is originally collected to the point at which it is archived or destroyed. We learned that data stewards have special obligations to the communities of concern.

Then, we spent some of the summer looking at the existing stewardship and frameworks for data. What we found was that they were aimed not at much of the data use by communities, but they were aimed at the use of individually identifiable information. If you think about the privacy act from the 1970s through HIPAA, through the federal regulations, governing research with human subjects, it is not even human subjects’ research if you do not collect information that could identify individuals. The protective frameworks, the stewardship frameworks that were in place were models designed to deal with individually identified data.

Typically, what they did was use an individual informed consent model. This is not a good fit for many types of community uses of data. We determined that there was a need for a new stewardship framework for community data use that we did not have adequate frameworks in place. We thought perhaps rashly, perhaps imaginatively that it was time to get to work on one.

The result was our stewardship framework not at all meant to be exclusive, but meant to be a framework start, a guidance for what some of the most important places to get to work might be.

I should say that one of the ways that the committee — for those of you who are unfamiliar with it, one of the ways that the committee works is it writes letters to the secretary of HHS. A letter to the secretary is not just a letter to the secretary though. It is a public document. It is a way of signaling more generally what do we think our recommendations for HHS, but how do we think that people ought to be thinking about some of these kinds of questions. When we talk about issuing a letter, we are not just writing a letter for the secretary. We are writing a public letter.

What were the eight features that we identified as critical for community uses of data? First of all, communities ought to be open about it and transparent about it. They ought to at least think about what kind of individual and group choice is relevant. It might be that for some kinds of data use, opt in. For others kinds, opt out. For some kinds, no sort of individual choice, but public notice, transparency, openness are the relevant ways to make sure that people are aware.

The concern that we heard that led to this was that if community members get surprised by the data that is being collected, how that data is being used, that can significantly undermine trust.

A second aspect. It is important to say what you are going to use data for. Purpose specification. That is one of the old, fair information practice principles. We thought it is still applied here. We also thought that it is important for purposes change to say that.

A third piece of the framework was community engagement and participation. You saw on the slides from Sally that the highest percentage of community engagement was on data collection at least in the feedback tool. Much less on data analysis. And a fairly high percentage on how results are communicated. But still I think it was in the 80s with respect to data collection and lower for the others.

Data integrity and security. It is not good for communities if problematic data are used. It is not good for communities if data security is inadequately protected.

Accountability. There needs to be someone who is or some designated entity who is responsible for what is going on with data use, somebody who can be asked, somebody who can be the responsible source for protection and so on.

We learned that there are serious concerns. We know that it is not all that likely that a de-identified data set will be used in ways that can re-identify individuals, but there are concerns in that area. I was struck in listening to, again, the feedback tool that while I believe that it was 95 percent are confident — somewhere in the high 90s are confident that individuals cannot be re-identified from their data. Only 57 percent had processes in place to be sure of that, which is a pretty interesting question for us I hope be thinking about today.

We learned that there are very important issues for communities to think
about when data sets are enhanced, when they are combined, when different types
of data are added together; for example, geographic data and health data.

With regard to this questions about small groups, communities. If you noticed in those slides, one of the kinds of concerns that communities have is that they do not have granular enough data. They do not have enough about subgroups. They do not have enough about smaller locations. How to think about trying to enhance that and in the same way not undermining trust.

The last aspect was risks of stigma and discrimination that can attend data use and the results of data use.

These were really put forward not as mandates in a framework in our letter, but as areas for communities to consider, issues that communities need to attend to and try to figure out what to do about resulting in a set of recommendations in the stewardship letter. Letters do carry recommendations with them. The recommendations that this letter carried was to facilitate the development and promulgation of models for stewardship. Help people figure out how to put some meat on the framework, on the skeletal. Support the development of dynamic guidance resources that compile best practices for experts, communities, and other data users that they are learning about stewardship. Compile case studies of results that communities achieve through their uses of data so that other communities might learn and be inspired. And promote the creation of training materials for researchers who collect and use community health data. In other words, there is a lot to be done to figure out where to go with the framework and that is why we are here.

MS. KLOSS: We are doing so well on time. Thank you, Sallie and Leslie. I think we have about five minutes for questions or discussion for either of them on these background projects that have been briefly reported on.

MS. HOFFMAN: Is there a particular set of questions that we are supposed to address in this meeting? We are covering a lot of ground here.

DR. FRANCIS: Next piece. We have tried to lay out a set of question for the groups in each of the sessions. It might just be helpful for a minute to go back here. We are not entirely sure what we anticipate to have come out of this. We are hoping and there are a variety of possibilities. One possibility would be a series of letters. Another possibility would be the recommendations for guidance documents, background materials, reports. Another possibility would be that we develop some models. There are a lot of options.

DR. GREEN: I really appreciate you asking that question and I want to make just two quick comments. I just realized it might also help if we just stated the obvious. The NCVHS is a statutory committee. It has been around since before Marjorie.

(Laughter)

MS. GREENBERG: I was five years old.

DR. GREEN: Just as Leslie said, there is not a predetermined formatting of the products, but there will be products. The way you can think about it if you wish is the committee produces letters and reports. We are likely to produce both in some manner or another based on your work and the presentations today. But there will be product. That is what I want to underscore. Every now and then things that the committee recommends actually happen.

The other thing I want to underscore that has been implied, but I want to make explicit is the committee has learned that virtually everything about data has changed except the way we think about it. It is very much like the Einstein quote. But this is a new era and a new world and the stewardship framework that was laid out very nicely is an example of where the committee has arrived. We are a data committee. We care deeply about data and their use to improve health for individuals and populations. I will just add that for presenters and discussants particularly.

You want to be careful what you say because you may see it. We so appreciate you saying it.

DR. BREEN: Just to kind of build on what Larry and the others have said. We also as a National Committee on Vital and Health Statistics I would not say it recently came to our attention, but we have recognized in the last few years how important local data is. That is something that federal government really does not attend to. But we can see how important it is. It is really critical for public health action in a way that federal government, data or national data really cannot be used for because it is too broad. It is not narrow enough in its scope.

What we are trying to think about and I think this pertains to our old thinking versus new ways of thinking is how can the federal government help local agencies. We are probably not going to be collecting local data at least not in the near term. The national committee does not collect data. We just make recommendations. But the National Center for Health Statistics also does not collect data at the national level. The sample sizes are not that large.

What can we do to leverage, to help, to promote and also to maybe standardize? We laid out a framework for standardizing privacy, but also maybe data collection so they can be compared among the different localities. And a lot of that is going on and we have identified a lot of it. You all have helped us. But those are the kinds of things that we are trying to figure out in this committee is how can we make recommendations to HHS that will help promote local data collection use dissemination because we see how important it is.

Agenda Item: Framing the Major Issues for This Roundtable

DR. COHEN: I am Bruce Cohen. My job is to frame the issues, but you have done my job. We can just skip my slides. Since I have already prepared them, maybe we will go through them. Also, this is the last presentation. We have organized this roundtable more as an interactive conversation. Mine is the last talking head you will see. The rest of the three days will be a conversation within experts and feedback among expert panels and then really rolling up our sleeves and getting down to the work in small conversations.

When I think about what we are doing, we are really on a journey to explore data security, data access, data use, and data dissemination. I thought since we are on a journey, it would be appropriate if we traveled the seven C’s to frame our conversation. Here they are.

The seven C’s are context, convergence. We have begun discussing some of these. Consistency, collaboration, our conceptual framework, course of action, and ultimately community-driven change.

The context. Again, none of this is rocket science, but the whole idea the
next three days is to pull this conversation together to make these threads more explicit. Certainly, communities are becoming much more proactive in planning and priority setting. More and more data are being generated by everyone: the electronic health records, the access to all payer claims databases, online surveys, more qualitative information and certainly social media data are feeding into this enormous pile of information. The question is how we sort through this and how we use it and how we use this data to help communities make better decisions. The data do not make the decisions. The communities do. Sometimes, I think, that gets lost in this rush to collect more information.

Also, the landscape is changing with respect to policies and laws and regulations. Certainly, the Affordable Care Act, health information exchanges, meaningful use, accountable care organizations. There is a lot going on that is powering the engine of data collection.

The Affordable Care Act actually requires nonprofit hospitals to do community health assessments. It will be interesting to see how hospitals partner with communities in these efforts.

Hopefully, as we move forward the key for context for me is really an expansion of the notion of what public health is broadening the focus of public health really to think about the quality of life in our communities, moving away from the disease-based model to consider social determinants, the built-in environment, and certainly the social environment. All of these are key parts to understanding population health in our communities. The context is rich and changing. It is a really challenging time, but a time of many opportunities.

We mentioned convergence. Certainly, there is a movement amongst not only the federal government, but state government as well to liberate our data to make it more available and more useful. There is more demand and use of these data. As we have mentioned before as Linda introduced, the national committee is trying to integrate our work to combine our traditional themes around population health, security, standards, and quality of data to weave into this convergence to help improve community health.

Consistency is also a key. I work for the government and I know we need to send consistent messages to communities. We need to understand and agree on what our shared values are around collecting information and using these data.

We have mentioned standards several times and standardizing variables. There are broad concepts and there are variables that are used to measure these concepts. There are definitions for these measures. But they are all over the map. How do we develop or understand the differences? If we want to operationalize, let’s — first of all, is socioeconomic status important to public health? I think most of us would agree. What are the appropriate measures for socioeconomic status? Let’s say one of the measures we are interested in is related to income. Should it be individual income? Should it be family income? Should it be wealth? Should it be assets? Once we determine one of these variables, how do we go about collecting data in a consistent fashion so that it is available for communities to use and understand how it affects their health? Consistency is one of the C’s.

Collaborations and partnerships. Certainly, here there is an opportunity for the whole to be much greater than the sum of the individual parts. The key is to put all of our perspectives together to help us develop the best strategies. Essentially, as part of the frame for our discussion, our implicit goal was to put the right people in the room to promote this conversation. Today, we have community members, community-based organizations, different levels of government, data providers, health care providers, researchers, policy makers, data guardians. This is an opportunity to exchange our different perspectives in order to reach some common goals.

I will not go through the conceptual framework. Linda covered it earlier. But here are the four broad areas. Leslie talked about some of the work that we did before the roundtable. Sharona asked the key question. What are the questions?

You will see in the agenda as we go through these next several days, each of the four panels and each of the four breakouts have four broad questions. They are, the first, how do communities collect and compile data. Within each of these broad questions, we have listed some very detailed and specific questions. As you go through these next two days, you will be able to see these questions with respect to compiling and collecting data. How do communities find and use data? Where are the gaps? With the proliferation of all this information, how do communities choose data and know what methodologies are sound? What analytic tools are there available to help communities use these data? How do communities partner with governments and local health care providers and data providers and other resources to use these data?

The second question is around incorporating data into priority setting and decision making. Is there a basic set of standard measures? Actually, there are several basic sets of standard measures? The issue is how to choose among them. And the issue that was raised early is that these data — many of these measures are available at the county level. I live in Massachusetts. People do not know what counties they live in. The data needs to be at the community level or actually at the neighborhood level for actionable change. What can we do primarily the national committee and government to help promote the collection and use and understanding of small area data?

Ultimately, as I have been a data collector all of my life and I am in many community settings, what always I am struck by is some people think data have the answers. It is people who make decisions who have the answers. And decisions will be made. How can we incorporate the proliferating information in data that we are generating to help communities make better decisions? And better means decisions that are priorities for them.

I have covered the third bullet here about information-enabled community-driven change. We have been talking about developing standards around harmonizing and data collection.

This is essentially the frame for these next two days. What we hope to accomplish during this roundtable. After the roundtable, we have begun discussing it. Where do we go from here? I think the entire committee wants to continue this conversation. I would be foolish to think in three days we will have final answers. This is part, I think, of a continuing engagement between the national committee and I think the federal government with data users and data collectors at the community level. I think our goal here in the national committee to help facilitate and promote this conversation and to the extent that we can use our resources help move the federal government into an area where it can help support some of these developments.

My final slide is we need to keep our eyes on the prize. The prize for us is the quality of life in our communities. And to the extent that we can help provide information, protect that information, and help communities make better decisions, we will have moved closer to our goals. That is hopefully the target that we are all shooting for. Those are the simple things we are going to do in the next couple of days. We put the right people here in this room and the adjoining rooms to be able to accomplish this very simple mission. That is our frame. Thank you.

We have some time if people want to comment about additional things that they would like to see discussed in the next several days. We will certainly entertain those suggestions.

DR. EDWARDS: Good morning. I am Carladenise Edwards. I am senior eHealth advisor for Zerodivide located in San Francisco. I apologize profusely for being late, but I see my California peer came in two seconds before me. Our brains have not quite adapted despite the fact I spend more time in DC than I do home.

I was really intrigued by the first presentation. And the thing that stood out for me was two things. One was the incongruence between the confidence in masking people’s identity and the processes for masking their identity. I would like for us to spend some time looking at that.

But the thing that really struck me was the need or the desire for data from electronic medical record systems and the inability to capture it. I am wondering if this group has any influence or ways in which we can effectuate change in that area if that is something we should look at from a federal government’s perspective or is that why you have the community here in a way that we can perhaps effectually change in ways that the feds cannot.

DR. COHEN: The answer is yes. I think as Larry, our distinguished chair mentioned, we can — the goal here is the National Committee on Vital and Health Statistics can make recommendations to the federal government about how we can help serve communities. We firmly believe that our recommendations need to be guided by communities to help us figure out what we can do best to support communities. I think it is a two-way street and it is an ongoing interaction that will achieve recommendations that hopefully will be beneficial.

Paul, do you want to comment as well?

DR. TANG: Actually, I am not commenting on this so much as raising another — we talked a lot about getting data at the community level and using it in policy decision at the community level and trying to draw some data from EHR. I am not sure we spend enough time looking at how do we get community data back into the EHR. What I mean by that is back into the care of an individual person that lives in that particular community. We probably are not thinking about that. That is probably another area where it could play a big role in that.

MS. GREENBERG: Marjorie Greenberg. Just in response to that, I think this is clearly part of the convergence theme of being able to collect data once, use it for multiple purposes and have more convergence among clinical, administrative, and public health data. There are few ways specifically. Of course, Paul, who just spoke, chairs one of the ONC committees, the policy committee. This committee makes recommendations on meaningful use, which is primarily right now focused on individual health care, but does have some public health and components to it. We have membership across those ONC committees and this committee and also try to collaborate with them.

I think we are really interested in — this is just getting started now in a way because the incentive payments are being made. It looks like there is considerable uptake of the records. But even exchanging them within the health care system is not completely obviously worked out let alone to other sources. But at the same time, we are really looking to hear what barriers people are experiencing if you are in a community that has good penetration of electronic health care records. What kind of information you would need. I think we really feel that so far we have not heard that much about efforts to use electronic health records. We have some presentations. I remember particularly one from the Bronx so that we can make more informed recommendations or the committee can make more informed recommendations to the department. We really appreciate that kind of feedback.

DR. CARR: Following on what Marjorie was saying, I am curious what in particular in the EHR you would have an interest in knowing.

DR. EDWARDS: I was commenting on the presentation that community members interviewed that getting data out of EHR was one of the top three priorities. I do not know being on the clinical side how relevant. I think people have a mystification of this data in information that at some level is not relevant in areas people think it may be. I do not really understand the context of that question, but the fact that it was one of the top three things people wanted that they could not get says a lot. To Marjorie’s point, I think there is some work to drill down into what is it folks actually need and want and is the EHR the right place to get it. If we are actually successfully in building the health information exchanges through public/private partnerships, perhaps there is a way to leverage a “database” or system of aggregating the appropriate information that can be distributed to the right people.

DR. CARR: Just following on that, I am struck today by the expansiveness of our ambition. World peace. I wanted to add that. I think that our success is going to be to hone in on some of the themes that have come in. We will not solve everything. If we could solve one thing or a few things and think in terms of use cases because when I think of EHR, yes, there is a lot of data. But actually, there is administrative data that can be used to answer many of the issues. I guess it is just — imposing on ourselves a discipline to choose a direction that is use cases or whatever, but that it may not address everything that we have outlined today, but that we have a tangible building block from which to move forward.

DR. COHEN: As some of you can see, the tents going up. This has been an NCVHS tradition to identify those who wish to speak. You are welcome to follow it or raise your hand if that works best for you. Thanks for that. Justine is always great at bringing us back to reality here. Thank you. Paul.

DR. TANG: I wanted to build on the past three comments. Thanks for raising the point, but to build on what Marjorie said about the relationship between the activity here and what is going on with HIT Policy, which has the HITECH incentive program behind it and what Justine just said. I will point out that public and population health is one of four categories in meaningful use. It has been limited. The work there has been limited by a couple of things. One is the public health department did not get funding from HITECH. But the other is really all the barriers that Bruce brought up.

If we were to work on some of those, we would move both programs forward in a very synergistic way. We cannot move without the things, the infrastructure that HITECH is trying to put in. And yet the HIT Policy Committee is not working on some of the questions that we were discussing today and have been discussing.

But I really like Justine’s point about which piece of the ocean do we want to boil and do we get anything to make steam. Maybe the use case. If we come up with just a very few, less than a handful of use cases. And the words we have been using in the HIT Policy Committee around public and population health is bidirectional. Can we get information that would feed and fuel the policy decisions that are being made at the community level? But by golly wouldn’t it be nice to know about the community that an individual lives in? That is incredibly important. Could we design a few use cases that illustrate that and then just work on those data that fulfill that need? But of course around that are the policies, everything from the privacy to the protected, et cetera. Can’t we get something to work because that will start moving an engine very quickly down this track?

The other piece of moving it quickly is if you do not operate in the timeframe that is needed when people have the bandwidth to think about it or are thinking about it then it also does not matter. If we come out with this ten years from now, it just will not matter. We do not have the attention. Right now, we have the attention and we have a complementarity of FACA groups working to provide some information. It is a wonderful time if we focus enough in getting work done.

PARTICIPANT: Last comment from Nancy.

DR. BREEN: Thanks, Bruce. I wanted to build on what Paul just said because I think it is really important and of course building on Justine’s question how can we focus this. I think the idea of trying to understand where people live, the context in which people live, how to get that information to primary care providers and into the location where they get health services is really important. I think that that is a really good focus of this committee. I would support that.

I also though would not want to lose sight of doing things within the community in order to improve the community conditions, not just use the data and accept the conditions in which people are living because sometimes they are unacceptable. In that case, we should also be thinking about how can we improve that at the community level because not everybody goes to the doctor and often times the people who do not are the people who need the services most. I hope that would be a friendly amendment to what Paul was suggesting.

DR. CARR: Can I make one more contemporaneous comment please? It makes me
think about the events of two weeks ago where a bomb went off at the marathon. There were five level one trauma centers visible from the site. A fertilizer factory blew up in West, Texas with many fewer level one trauma centers. I think that is a great example as we generalize about health care let’s say in one place versus another, the key fact of how many level one trauma centers are within one mile of the bomb site are incredibly important.

DR. COHEN: We used to be ahead of time, but I knew that would not last. Thirty seconds each and then a break and then we are going to reconvene here for the first panel.

DR. TANG: We are making progress though I think because we used to think of community and community data is like this thing. But really what Justine just said is it makes a difference. I am internist. I do not think of treating an individual’s diabetes or an individual’s weight — you can only do it — an exaggeration, but you can mostly do it through community. It goes back to what Nancy says. We really have to — we do not change communities, but we give communities an ability to change and make change. We really have to make that possible. This is very exciting, but we have to think of it in a broader scope of not just population data. It is really data for action.

DR. COHEN: Thank you all. A great opening. I look forward to the rest of the session. We are on break for 15 minutes.

(Break)

Agenda Item: Panel 1: Collection and Compilation of Data

DR. TANG: We are going to begin our panels that are each going to be followed by some breakout groups. My understanding is that we have a panel discussion to infuse with knowledge and experience. And then we go to breakout groups where we can explore some of the questions that are listed for you.

I have the distinct pleasure to introduce a few of our panel members here. I only have three. Am I correct? The first is Andrew Bazemore, who is the director of the Robert Graham Center for Policy Studies in Primary Care at the American Academy of Family Physicians. We are talking about collection and compilation of data, by the way. Next will be Bridget Catlin, who is a senior scientist and program director of the MATCH program, which is Mobilizing Action Toward Community Health at the University of Wisconsin. Barbara Zappia, who is the senior program officer at the Greater Rochester Health Foundation. Three representatives of different kinds of communities and all discussing the collection and compilation of data. I think we will begin with Andrew please.

DR. FULCHER: I do have a quick thing to make. Apologies. My name is Chris Fulcher. I just wanted to let you know that I am quite hard of hearing. I am deaf in one ear and quite bad of hearing in this one. I have this FM system. If you see me pointing it, do not be afraid. But I am looking at your really intensively. I am reading your lips. And if I am getting up it is because — Andrew, I am going to give you this when you get up there so I can hear. I just wanted to set the tone so you are saying, who is that crazy guy doing all this stuff.

DR. COHEN: This is also a good reminder for everyone to speak clearly and make sure they are speaking into their microphones so that we can all hear. Thanks.

DR. BAZEMORE: I have known Chris for seven years. I have always found his gaze intense. I am Andrew Bazemore. I am a family physician, as mentioned. I work for a center call the Robert Graham Center. The Robert Graham Center is a bit of a unique animal in the world of association of workshop shops. It was actually born and gifted an editorial independence, which allows me I think to be here today in many ways because I might spend the rest of my days talking about family physicians and a pretty narrow perspective on primary health care. Fortunately, we have been blessed with a team, a wide array of social scientists who actually were able to take on primary health care as originally envisioned, which means bringing together the elements of health care delivery on the first contact front line with the social determinants and those in public health.

And as a center, we really have had a considerable interest in our 14 years of existence in how can we inform integration and resist the tide of fragmentation. We have long hoped to put our work into peer-reviewed publication. But we have had an equal aspiration to be informants and tool developers and realizers of what Todd Park has clearly inspired us all to call liberacion. We see incredible value in providing information to really inform the process, which we agree has really been galvanized in the last three or four years of public health primary care integration, of comprehensive primary health care that brings behavioral and physical elements together and really that means and requires for us population and clinical data integration. I think of it inspired.

I suspect given that it was the chair, Larry Green, who turned me on to the fact that there were some really remarkable notions around the time of the last great health reform in 1967 born in a report on an equally esteemed committee headed by Marion Folsom that we have colloquially called the Folsom Report where he really dreamed of the notion that there needed to be communities of solution forming to address problem sheds, small areas that do not necessarily fit political boundaries and small problems that do not necessarily fit conventional disease associations. And communities that come together perhaps led by a social worker, perhaps a clinician, perhaps someone in the public health field to address these. We have really tried to dedicate ourselves through tool building and data integration efforts to enabling these communities of solution.

We say that liberacion has been — it has been refreshing to see Todd Park and others really drive the notion that we have to open federal, state, local, and community data sets. But obviously, this is only the first step for an effective primary health care and community health improvement process. Really to empower communities, once you get passed liberating, you have to think hard about how you integrate that data and you translate it into meaningful tools.

How to integrate this information, translate it into tools, and as importantly if not the most importantly engage the community stakeholder. And then in return turn this into a 360-degree process. How do we improve the actual data sets themselves based on community input because how many people hear from someone when they have created a table, a map, a description that speaks to a community member? You know you have that wrong. They scratch a little note that says I need to improve this. But the fundamental basis of that map, table, or data or projection of data, the data set itself never changes.

When we really have tried to take this on, we started working with community health centers. And for those familiar with the federally qualified health centers, they are really born of the notion that communities have to be in charge of their own primary care safety net. These are driven by boards that have a 51 percent community constituency. They are forced to apply and then reapply for federal grants that force them to say how they are going to serve specific geographies or populations that are in need and then defend that they are actually serving these. And yet they face a tremendous dearth of information about how they are doing.

The medically underserved areas, again, the data driven scores that allow
them to get the grants in the first place are not challenge necessarily or have not been challenged by their utilization outputs, by the footprint that they actually leave on a community and as someone who helped to start a community health center, I can tell you that I was told I had a nice oval with a high medically underserved area, an IMU, or a score that said you need to be here. And you are going to have about 50 percent black and 50 percent white patients from this small corner of Baltimore. Because I spoke enough Spanish to be dangerous and was partnered up with a bicultural, bilingual, Argentine pediatrician, my service area when we tested this about six months later, it spanned 20 miles, folks passing hospitals, community health centers and ERs to come get into our clinic. This information was not making it back. If anything, this was suppressed at the organizational level at first because it looked like failure on our part rather than success and demonstrated additional need.

We spent years trying to help with this translation process, this integration process, pull together clinical data sets and population measures into tools, not just maps and tables, but tools that can inform better decision making about where you sent your outreach in your next clinic. And ran at all kinds of hurdles at bringing these together just in the Baltimore region, competition amongst otherwise very friendly community health center CEOs being one of them.

We really enjoyed, again, trying to be part of an enabling community safety net development from the ground up by finally achieving about four years ago a liberation of the uniform data system. This is one of the first pearls that I have certainly learned in the process of getting information back to the communities is how do you help federal or state agencies that hold data sets that are required reporting elements, turn those back into useful not burdensome, again, information drivers for the communities or in this case the grantees who bear that burden and bear that requirement. There was a loud screaming for why do we turn this in every year and we never see anything in return.

What we have done with uniform data system and building an online mapping tool and ultimately being able to produce it publicly is try to help the federal policy drivers of the decision about where the next health center. The state drivers who have to create plans — we have had two straight administrations, Bush and Obama administrations say we are going to double the size of community health center network to adapt to the increasing demands for access in vulnerable populations. How do you get the states who are trying to create a plan of action for where you are going to put the next and the next data set and then the grantees themselves who want to put in these applications or even better those who have never thought of becoming a grantee to step up and say I can do this? And I can afford to pull this information together. Or more importantly, let someone else do it for me.

You start by taking these reporting systems, which are reported back in a zip code level, and show you how many patients are coming from any small area, a zip code from community health centers. How many are showing up in areas of greatest need? And greatest need can be measured in all sorts of ways. Currently, the HRSA definition is areas with high-density poverty. Where can we find or in this case highlight areas where you seem to have a strong penetration or a very weak penetration of community health centers and a strong density of poverty?

In building this, we also learned that you could not just turn on a mapper, allows someone to go in, click on areas and demonstrate yes I think there is need here. You have to be able to liberate the data set itself. Ironically, because it was a duplicate recording system, you are collecting all this information through your practice management, your EHR software, and then you have to duplicate it for the electronic handbook, give it to HRSA, and you never actually see your outputs in many cases. How can you hand it back to the community constituent, those who would want to on the ground perhaps apply for a new grant or consider expanding their existing grantee site? You had to create downloadable, accessible points where you can see tables and start to see summary statistics, but also extract the information.

How do you liberate that community, again, agents who need information, integrated information, but they also are forced to have it in order to get resources they need? Again, they are up against $10,000 to $50,000 per contract vendors saying I can help you pull this together for your grant, but it is sitting right there in the hands of HRSA if you can bring the census in HRSA and the CDC’s behavioral risk factor surveillance survey and any number of other sets together. You do not need $10,000 to $50,000 vendor. You just need someone to try to tie these together in one place.

Given that the number one research question under the Bush administration expansion was where do I put the next 1, 10, 50, or 500 community health centers since we are going to double the size, and they did. You need some way to start to look when three applications come in to one project officer that look awfully similar except they have pinned in three different locations within five miles of each other the spot that you absolutely have to pin the next health center. You need a place that you can overlap all that information. You have to create the tools to help you get there.

When the data is absent because I can tell you when this tool is turned on — when the data is absent, you really have to create things to fill those gaps. As soon as we turn on the UDS mapper, we alarmed and then enlightened a lot of community health centers. And the next step was the alarming of the free clinics and the rural health centers who said we do not have a federal reporting data set. How can you help us get one, number one? But in the meantime, how can you help us avoid having a bunch of community health centers move into our backyard when we have a perfectly adequate community-driven safety net that works? You have to build tools and create best estimates. In this case, ways of making sure that you can take network data sets and look at measures of approximate service area or footprint to help drive you forward.

Finally, how can you start moving past poverty as a singular measure of need and actually bring in, again, through BRFSS and other sources bring in other ways that you can evaluate small area need based on what the community determines as its top priority?

I should say that one of the biggest delights in working for a center that is trying to liberate more and more information comes from my past in global health. When I do work in rural Honduras with a comprehensive determinants project in about five communities and have to strap on duck tape on my ball cap and a GPS unit and go door to door knocking to create a primary data set, I come home and I am always relieved to know all we really need to do is liberate and integrate and translate. These are small problems by comparison. How can we help the community which says it is high and low birthrate and where it interacts with high and low income that helps us decide where we want to put our resources, make those decisions.

And then finally, once you start to build a federal tool, as soon as you build it, you get a call from the New Orleans mayor’s office that says that is great, but you missed our question and you cannot answer it. And we do not really want you to build it on top of the national tool. Can you come in and help us adapt this to our needs?

In our case, we had a big hurricane called Katrina. We had billions of dollars in federal state foundation and other revenue pour in. We still have clinics sitting side by side and in Algiers, no clinics whatsoever. We have a simultaneous, service area overlap and deficiency sitting across the river from one another. Can you come in and help us use local information, change the question, and pull in local data sets that you do not have and never will?

Or in North Carolina, can you help us get to — again, in the CCNC, the revolutionary reshaping of Medicaid delivery, which divides the states into 14 districts, has folks actually detailing practices about their quality indicators at the practice level, but still just managing panels. Can you help us move towards real population management? Can you bring together not only cost and claims information, but show us where asthma density is highest, where the smoking penetration is the highest. Put the two on top of each other and actually build something that sits behind our firewall because we cannot make some of this public. But the decision makers desperately need the tools and bring these together.

Again, I will not say finally, but near finally we have learned certainly
that when you capture the power of public data, you sometimes are forced to bend its purpose. The national provider identifier is just one example. This is what CMS uses to make sure that we have a uniform number attached to any type of provider, behavioralist, oral health, NP, PA, physician or otherwise if they are going to be billing CMS and really in 2013 if they are billing anyone because everyone has adopted this standard. How can you take the downloadable and freely downloadable MPI and turn it into a workforce data set? And more importantly five years after its birth since it was not born as a workforce data set as it gets more and more polluted, how can you clean it up so it can continue to serve that function?

We are really in the end we found that in all these processes stakeholder engagement was the essential element. Everything we have done has led to one-on-one conversations. The tools invariably create ten questions for every one they answer. Most of what our center is doing now with any tool we are building is trying to get out as Chris Fulcher has taught as well really get out on the ground and make sure you are at every conference and opportunity to sit down one on one, start an hour long scheduled conversation maybe with the tool you came to talk about, but really try to listen and build solutions.

I think the final piece is that we see is continuing challenging is for these learning communities to really grow. How do we continue to take advantage of integration opportunities? The health center controlled networks are sitting on a mountain of information, the RHIOs, the health information exchanges. How do we pull them together with population data so that more and more people can share in this wealth of information? How do we translate these into more tools? How do we make quality dashboards increasingly meet the data portals that Bridget will tell you about that Chris has been building in abundance?

How do we move the physicians towards thinking about community vital signs? When they open their chart on a one-on-one encounter, they know what a walkability index is. They know what the food desert scores for the region are and they actually have the translational ability to see what that means and how it will change their behavioral counseling.

How do we engage the stakeholder? With the ACA as opportunity to really grab the hospitals that are forced by the IRS to do community health needs assessments in a three-year iterative process. How do we gather the incredible wealth of information and dollars there and come out of this starting this year and pull it into the public space? Or follow agriculture’s example from 100 years ago and create extension agents just as they did to take knowledge from the land grant universities and make them reach the frontline farmer. How does an NCVHS really drive the information down to the smallest area level by enabling either through extension agency or other techniques knowledge?

Finally, how do we improve that data based on community input? How do we make sure that any online portal for data upload also has a way for data correction? I will stop there.

DR. TANG: I think what we will do is take questions at the end. Bridget is going to be going next about the work going on at the University of Wisconsin.

DR. CATLIN: Thank you for inviting me to be here today. And thank you, Andrew, for mostly showing me your slide on this so I did not have to turn around and look over my shoulder. But to make the others of us who are sitting up front more comfortably, I am not going to have any slides today. I am just going to talk from some notes.

First, I thought I would talk about why I think I was invited here to talk to you. In Wisconsin, we have been measuring and yes ranking the health of Wisconsin’s counties for ten years. In 2009, the Robert Wood Johnson Foundation funded us to take that work from Wisconsin and take it to all 50 states, which was quite a shock because we thought they were going to let us do it incrementally and add five states a year for three years, but they said no. Do all 50 the first year out.

In February 2010 — we would have made it in December in 2009 within the year, but that is not really a good time to release anything in December. We waited until after the holiday period. And in February, we released these first state-by-state rankings looking at health outcomes, which are a proxy for the health of communities today and then a number of health factors that cover the multiple determinants that predict future health. All of this is available at countyhealthrankings.org, but I am not here to do a sales pitch on our project or anything like that. I am here to share with you what we have been learning over the past ten years of working with communities and in this case, counties. We do what we can. County is the lowest level that we can get to cover the nation.

Even in Massachusetts, we have found that there has been a lot of conversation that has been started by our work. I am going to share some of our thoughts and things that we have learned from that.

Early on, as we were talking to a number of public health organizations, warning them that we are going to be producing these rankings, we got a rather stern warning from somebody who I will not name, but he was very upset that we were going to rank him and spank him. We heard that and we never had that intention, but it really has helped guide us and the fact that we aggregating data collected by other people, making it available to counties, but with the sole purpose of a call to action. Data by itself is no use. It is turning into action, which is what is really key.

Over the years since we started the Robert Wood Johnson Foundation who have been much more than just our funders, they have been totally our collaborators and partners in every way. They very soon after the first release realized that they as an organization really had to do more to support communities in using this data and improving health. They added a new component to our — it was then new, the roadmap. It became the county health rankings and road maps program.

For those of you who see the MATCH acronym on the listing on the agenda, the very first go around of the county health rankings program did have the MATCH acronym assigned to it. But the communication folks and the Robert Wood Johnson Foundation decided that that was much too close to match.com and that we needed — move away from that. It took a long time for Pat Remington and me, my colleague, to come up with that MATCH acronym. We have retained it to describe the work of our group. MATCH still lives on.

By the way, before I started talking, who has heard of the county health rankings? Pretty much everybody. Something is working a little anyway. Those of you who do not, I apologize. I am not going to do rankings 101. Feel free to browse our website or listen in on what of our webinars or something to get more of an understanding. I am just going to give a broad brush.

When we first released in 2010, I can tell you that we got feedback from many smaller rural counties across the nation that this was the first time that they had seen county-level data that addressed the many factors that influenced their health. They were so grateful. They were not in states like Florida and California and New York, which have fantastic county-based population health information systems for their communities, but a lot of states do not have that. The initial feedback for this was very important and that we should keep doing this.

As I said, we do not actually, even though I have been asked to talk on the panel about collecting and compiling data, we ourselves we gather data already. We gather county-level data. We do not do anything with individual-level data. It has already been compiled for us to the county level. We use data from a number of different sources.

In particular, as I am here in the National Center for Health Statistics building I have to acknowledge the work of that organization and the staff that work on the health indicators warehouse because without them we would not be able to do what we do.

Secondly, the folks in Atlanta and particularly the folks that run the behavioral risk factor surveillance system, they are also very key. I would say that of our 30 indicators by which we rank, about half of them come from those two sources. They are very key. I cannot stress enough how important it is for the BRFSS system be continued and at all possible sample size be increased because there is a lot of angst that goes into how to produce county-level estimates from a system that was really only designed to develop state-level estimates. We will not get into the details here. If anybody wants to hear me talk about this, catch me at a break or something like that. It is very important. Folks on the ground wanting to monitor health behaviors in their communities cannot do this without that data.

We also use other federal data, CMS data. The Dartmouth Institute partners with us and they compile CMS data for us. And then we rely a lot on the American Community Survey too coming out of the Census Bureau. And then a number of other different sources.

But our guiding principles by which we do our work are that what we are trying to do is take all this data, put it into one place, sort of a one-stop shopping for communities and display it in an easy to use manner with lots of links to how to move to action. Beyond that, as I said our other guiding principles are that we are working with a very broad definition of health and our overall purpose like you all I heard it mentioned today is really to help communities improve health for everyone within their community.

When we decided what measures we actually wanted to rank on, we focus in on actionable measures. Yes, we do happen to rank them. We rank them as a starting point because they get attention, not because we think the rank themselves are necessarily that helpful.

We believe in transparency. We like to clearly identify our data sources, the year’s limitations, and things like that. We also do not want to overwhelm people. Parsimony is one of our criteria too. We make our information available at no cost to users and then freely downloadable as Andrew pointed out. That is very important.

As I said, I am not here to do a plug on the county health rankings. I am going to just talk to you a little bit about the things that communities are grappling with. I am going to talk about questions that communities are trying to answer that we have been hearing. I will mention some current sources of local data, some questions that communities have in finding and using data, and then I will talk about some gaps in the data. I think a lot of what I have said have already been covered, but maybe it will help to put it all in one place like this.

We spend a lot of time talking to folks in communities that are trying to answer where are the areas of greatest need as part of their community health planning processes. They want to know where to direct resources that are increasingly limited. They need data to do that. A lot of attention now with the IRS regulations and hospitals as well are getting in on this community health needs assessment area. It has become of even greater importance.

But one part of the community assessment and planning process that gets missed out quite often is the need to be able to evaluate progress. It is one thing just to be able to identify needs, but then when communities actually start trying to do something, they are very interested in knowing if they are actually making a difference. And some of the measures that we use that are at the higher level are going to take years and years to show change. We need to do more to be able to help communities there.

Then of course, there are lots and lots of different advocate organizations who are looking for very specific data to meet their needs. And then there are — really I am just focusing on three key areas that we see communities wanting data for. Those who have begun to identify problems need evidence to show potential funders where there really is a problem and they show that my community is in much worse shape than this community. We have grad students who like to do Google searches for us periodically. Aside from community health needs assessment, we often see county health rankings mentioned in terms of cited and grant applications as a source of data to attract funding.

In terms of what sources of local data are available and what are people using, clearly the federal government sources several of which I have already mentioned are very key. In addition to the health indicators warehouse and the BRFSS system, CDC wonder that allows communities to go in and be able to look at their vital statistics data as much as possible at the community level and then cut in different ways is used. And then the National Cancer Institute also produces local data that community use.

The next level of data is state government data is very important. It is not just state data from departments of health. It is also data from education departments, from transportation, from child welfare systems, things like the WIC program, states that have the youth risk behavior surveys. That data is used. People rely on information that states put together for their health plans and then of course health care utilization data as well.

Locally, the amount of data collection that communities do varies quite widely. Smaller, rural communities do not have the resources to do surveys. They may run a focus group or two. Larger communities with a few more resources do their own health surveys and collect their own data.

And then there are several other sources of data and some of us are represented here such as the county health ranking, such as Chris’ work with Community Commons, Healthy Communities Institute. Some of these systems cost money and the ones that cost money are beyond the reach of many of these smaller counties that have as much need if not greater need than other better resource locations.

We get a lot of questions when we release our rankings. I have been a research scientist nearly my whole career in health services research. When I moved to the Population Health Institute eight years ago and then began doing this work, I have now a whole new set of terms that come out of me. Customer service and communications is key and all kinds of things like this. We do annual phone training with the people who — I have to remind them, yes, they are a customer. They are not buying anything, but let’s treat them like that. Be as nice as possible and as helpful as we can. We get email questions. We get phone questions.

The most common question we get is I have my data. Is this really true? Does this really capture reality? And of course and not surprisingly it is rural counties where the numbers are small and they can fluctuate from year to year. It is not really surprising that they ask that.

A lot of questions go into who does the data cover. We have a large prison here in town. We have a college. How does that help us or hurt us? What does that mean for our data?

How to actually interpret the data. A lot of us take things like confidence intervals for granted. People do not know what that is. They need help understanding that. Really, what they are trying to get at is what is a big deal and what is not. If I am different from my neighboring county, is this something that is worth pursuing or not? We have a tool on our website called areas to explore that flags for each county. We have a lot of algorithms behind how we do this. We do not show that when we first bring up this site because we do not want to influence people. We want them to look and come to their own conclusions. But for those who need some help and a lot of them do, we do make that available.

People have questions about how to present data to particular audiences. Everybody loves maps. Maps are fascinating and cool. Andrew and Chris do great jobs with maps. You cannot answer every question with a map. You also cannot answer every question with a pie chart, which is what some communities think too. There is a lot in between highly complex maps and pie charts and communities need help with that.

And then they get to more important big questions. What will it take? I see these indicators. What will it take and how long will it take to move that needle? And then other questions just get into questioning. We did our own survey within the state of smoking levels. Ours are not the same as yours, which is right. We always tell people. The more local it is to you, the better it is.

Where are the gaps? As I have already mentioned, communities need data that will allow them to show progress in the short term. Measures like obesity and premature death that we rely on are very good for giving you a 30,000-foot level. But moving those is going to take time and being able to measure that is really important so that people do not lose faith.

And then as we have already heard, people want data in a more granular disaggregated manner whether it be by geography or focusing on specific populations. We go back and forth as a program as to what extent we should try to move towards more data drill down. The current thinking with the foundation is that is a huge leap to go from us compiling data for 3100 counties to then being able to do the disaggregation.

What we are doing wherever we can is identifying more local resources primarily state resources that provide these opportunities. But when you do an assessment of this of what is available across the nation, it really does vary by state.

There is a lot of discussion about the potential as an electronic health record as a source for us in the future. I think as Nancy pointed out. Even if that were ever to become available and we could find out how many people are smoking from EHRs, the people that are not going to the doctor are not going to be captured in that. I think that is a lot of promise, but it will not be the answer to everything.

And then the last gap that we see that a need for in particular is more data on the built-in environment. Within the county health rankings, we try to measure this, but we quite honestly our measures are not the greatest. We do it to start the conversation going, but people really need to understand things and look at things. You would think it would be easy to find out about bike paths and working trails and sidewalk prevalence and things like that, but that stuff is just not available.

Some quick conclusions. We really have noticed and realized the educational value of data and not just the data. Having a model or a framework. Our model of population health on which we base the county health rankings. It brings a lot of people to the table beyond the health care and the public health people that are pretty much represented in this room. Our model encompasses education, government officials, philanthropy, businesses, and bringing those people can be done if you frame your data appropriately. If we just produced a report for all 50 states and had not taken the time to frame it with the model and do the communications work around it, we would have lost that opportunity.

Actionable data is key. There is always going to be cause for more data. But I know that the committee has talked about the need for core sets of indicators. We try to align our work with Healthy People 2020 with their key indicators, but not all of those are available on a local level.

Something else that we have to remember though is that local communities do not have the technology that we all assume and take for granted. One of the problems we have with our website is that 25 percent of our users are still using Internet Explorer Version 8, which does not allow a lot of the things that more current versions of browsers do. And on top of that, local health departments in particular because of all the importance around privacy and issues like that have some really strong firewalls set up and actually have difficulty getting out of their organizations to find this data. That is frustrating to us. We try to do what we can on the browser side, but we cannot do much about the technology side.

But beyond technology and data support, what communities really need help with is understanding and using the data and moving to action. Through the Robert Wood Johnson Foundation, we have actually been able to hire. We have three full time what we call community coaches that are available to help people with no charge for their services. They are very busy people I will tell you that.

What gets measured really is what gets improved. We do need to just to continue and enhance federal and state data collection activities. Providing these local estimates is key. We have to combine many years of data to be able to do this. There are some promising work with modeling that may help us more with deriving these local estimates.

And then we would actually like to see more cross federal agency collection and linkages between BRFSS and ACS. There is more potential there.

And finally, I would just like to close and tell you about one community in the State of Washington, Mason County. In 2010, when we released our rankings and within the State of Washington they went were down at the bottom. They did not get depressed by this. They might have been a little and they did not tell us that part to start with. But they took a look at their data. They compared themselves to others in their state and they realized that there were other counties that were doing much better in terms of their health outcomes and actually had similar levels of income. People always ask. Isn’t it all about income? But they found another county that has very similar income levels. But in turns out that in Mason County, far fewer of their residents were going on to higher education. They were graduating them out of high school at the same rate as other places, but far fewer of them were going on to further education.

Mason County has spent time working on this. They put together a grant proposal, two out of RWJF and now through our roadmaps to health community grants, they are being funded to do a project to really work on increasing college readiness starting in the young years and really moving forward to changing the culture towards the value of higher education. That is the kind of story, which is why I do what I do. Thank you for letting me be here.

DR. TANG: Well, that is fabulous, informative, and inspirational. I really like the idea of the community coaches, something we sort of put in, but that is really a concrete example of how to help.

I also appreciate the acronyms MATCH. It takes two people. One to come up with an acronym and one to make it make sense. We had that with Empower. I totally sympathize with that.

Barbara is from Rochester in New York. I had visited there. It is one of the premier community-oriented cities. It is really amazing how the businesses have come together to improve health. Barbara Zappia is going to be our final panelist.

MS. ZAPPIA: Thank you. I am Barbara Zappia from the Greater Rochester Health Foundation in Rochester, New York. I am very pleased to share with you some information about how groups in the City of Rochester are collecting and using data at the most the smallest community level, at the neighborhood level, and even at the block level.

What I would like to share with you today is a little bit about the initiative that funds these grantees and their work. I will talk to you a little about where they started collecting data in their first year. Introduce you to the evaluation framework that has guided how our grantees are collecting and using data now and the secondary data sources that we have struggled to find and with our challenges and opportunities.

This map shows where three of our grantees are located in the City of Rochester. We have a fourth grantee that is located in a rural area in Yates County. These geographies that you see are inter-city center, high-poverty rates, low-level of educational attainment, high rates of crime and violence. It is the area of Mineral County where most of our communities of color reside.

A little bit about the initiative that funds these groups. It is predicated on the notion that our opportunities for better health begin where we live, work, and play. The grantees are working on in this initiative is they are attending to the physical, social, and economic environments of the neighborhoods. It is really important to understand the boundaries for these neighborhoods were created by the folks who live in these communities. They are authentic to folks who live in these regions. Hence, they do not correspond with any sort of administrative boundaries, census tract zip codes.

This initiative is asset based. We have used the technical assistance provided by the Asset-Based Community Development Institute at Northwestern University. Probably the most important thing about this initiative to understand is that it is resident driven and grass roots. When I talk about data collection, this is residents collecting data from other residents. This is the residents of these communities using the data to tell their story to share information about their communities with city council, with police department.

The icons on the bottom are four current grantees in the City of Rochester. The grantees are community organizations. The Settlement House that has been around for 100 years. A community development corporation. A federally qualified health center. And in our rural area, a rural health network. Again, this is all about residents becoming co-producers of health that residents are not advisors or participants. They are social actors in this initiative, producers of health, and hopefully change advocates.

When these groups receive funding from the Greater Rochester Health Foundation in 2008, they receive $65,000 and a year’s worth of time to get to know their neighborhood. They turn to available data, census data, data on housing, crime, and land use from the City of Rochester. Community health studies were very valuable, in particular, the Latino and African American health studies that were completed by our local health-planning agency. County health data was available. But because none of this was available at the neighborhood level, they collected their own data beginning with the health conditions of concerns to residents.

Health is in quotes here for a reason because the conditions of concerns to residents were the things that were outside of their front window. Rats in the street were a big health concern. Trash in the street was a big health concern. The drug dealers on the corner were the health concerns. They were not at least in the early years that concerned with diabetes and obesity as we were.

Residents collected data on health services that were needed, social conditions of concern, what residents liked least and most about their neighborhood, and because this was asset based, the existing neighborhood assets. Who were the strong individuals, institutions, agencies that they could partner with to create a healthier community.

This schematic here is the framework for the evaluation for this initiative. Also, in 2008, we commissioned an evaluation. Our evaluators were assigned to do two things to assess the efficacy of the overall initiative, but to also serve as evaluation coaches and work with grantees and residents in their communities to help them develop the data collection tools, the systems, to help make sense of their data, but really provide hand holding and guidance every step of the way.

Just to walk you through a little bit of this framework, on the far right hand side, those are the long-term changes that we hope to be able to see at the neighborhood level, changes in health status 10 to 15 years from now. Changes in heart disease, diabetes, depression, anxiety.

Our grantees for the past three years have been concerned with that left hand red box, the changes in the environment, exposures, and experiences or individuals who live in the neighborhood. Things like the social environment, levels of social cohesion, civic engagement, collective efficacy, physical environment. Is the neighborhood cleaner, safer? Is it a more healthy place to live?

As we move to the right looking towards more mid-term changes that we are looking for in the second column changes in attitudes, feelings, and understandings of folks who live in these neighborhoods. Do they have greater hope for the future? Do they feel safer in the environment? Do they feel more cohesive and connected to their neighbors?

In the third column, looking for changes in behavior, alcohol use, tobacco use, physical activity, nutrition. And then moving down to changes in medical conditions that precede disease. High levels of stress, obesity, hypertension.

And the boxes below are the data sources that grantees, residents, and evaluators are looking at. For our short-term indicators, mostly data collected at the neighborhood level through surveys. As we move to midterm and longer term outcomes, looking to both primary and secondary data sources to help us understand that.

This is just a slide with some examples of the data collection tools that one grantee has used. A lot of survey questionnaires. These have been implemented by mail door to door at meetings. Discussion group guides can help with small groups that are participating in community walks or seniors in a housing complex. Lots of observation tools to understand the changes to the physical environment and then ongoing monitoring tools.

And then secondary data reports. We have turned to the Monroe County Adult Health Survey and the Yates Community Health Assessment, the Rochester City School District reports and the Dundee school district reports. Again, none of these data sources is available at the local level.

Rochester Police Department Crime Data is available at the street level. Probably the most promising data source we found so far is the New York State Department of Health SPARCS database, which looks at inpatient hospitalization and emergency room visits and working with our local health-planning agency, the Finger Lakes Health Systems Agency, we have been able to get some data that corresponds with our neighborhood geographies.

Our challenges. The geography as you have already mentioned is a real challenge for us to overcome. But also reaching residents for primary data collection is always a challenge. These are folks who live in tough neighborhoods and are challenged by a lot of things in their lives. We have tried a number of different ways to reach them. Door to door in our urban centers has been the most effective way, but very labor intensive. Our evaluation coaches have trained residents to partner with sometimes college students or other residents to knock door to door to implement a survey. One group knocked on 800 doors this time last year.

In our urban centers, mail surveys have been minimally effective. We try to piggyback on the work of our Monroe County Department of Health when they were doing their adult health survey and we contracted with the agency that was completing the phone interviews, but were unable to find either cell phones or landlines for their neighborhood geographies. Mail surveys have been very useful and a good response rate in our rural communities.

Small numbers are definitely a problem as you have mentioned. Finding data sources for health as residents define health vary broadly has been a challenge. Mental health data is pretty sparse.

We have found that residents in these neighborhoods certainly have the capacity to do this work to understand the data, but they need the resources and support every step of the way.

It is also something that is of concern to residents is how they tell their story. They want to share the data with the people they want to share it with. What they do not want is data to paint a picture or perpetuate a picture of their neighborhood as a bad place to live. That is very important to them.

Finally, this is just a schematic that our evaluators came up with to try to make the most of what we have as far as knowing whether or not our efforts or the grantees’ efforts have made a difference looking at data that we do have at the neighborhood level comparing to other data sources and looking to larger geographic data when we do not have local data.

DR. TANG: Thank you. As anticipated, it is definitely a rich set of panelist presentations. I think we have just a few moments for clarifying questions.

DR. COHEN: I think it has been such a rich panel. Why don’t we take five to seven minutes to ask some basic questions if that is okay?

DR. GREEN: I want to build off of what Justine and Paul said just before you started about the value of use cases. You gave us a lot and I want to say thank to that. Andrew, one of the things you said was just putting a tool out there and letting people see it does not mean that they will understand — and I represent that. Some of those maps went by. Could you put one of those up or just explain what they showed and how it affected local action and decision making as a very specific example?

PARTICIPANT: I will do my best.

PARTICIPANT: Why don’t we go to another question while that is being set up?

MS. KLOSS: My question actually was for all three speakers. I was interested in who is the recipient. Who is the primary user of the data? For example, who are your grantees? Who might be working with the county data? Who is calling with the questions? I just wondered.

MS. ZAPPIA: The residents of these neighborhoods are the primary users of the data. They are using the data to define their priorities for improving their neighborhood, but also going to city council or talking to the chief of police to get the support.

MS. KLOSS: But they have organized themselves in some way to be a grantee.

MS. ZAPPIA: Yes. They are organized by these geographies of the neighborhood. They have an identity there.

MS. KLOSS: To the extent that they have a nonprofit to set up.

MS. ZAPPIA: The grantees each are a 501c3 organization. They are the fiduciary and provide the structure for organizing the residents.

MS: KLOSS: Who might be a typical user of the county data?

DR. CATLIN: Well, we started thinking that local health departments were our
primary users. They are frequent users, but it has gone much beyond that. We have private citizens who hear something on the media and contact us and want to get involved and want to do something. We have chambers of commerce who are contacting us with questions. We have newspapers not just for their stories, but we have had editors who actually — they have seen where their community is and the editor of the newspaper wants to do something within the community. Some of it is university folks, nonprofit organizations. It is a broad spectrum.

DR. BAZEMORE: Well, we have built a number of tools and the answer could be different for each. Since I focused on UDS mapper, it was initially intended as a federal planning tool. It migrated to a state planning tool for primary care organizations and associations and a grantee-level tool. But as Bridget mentioned, it is amazing how quickly you get calls from media outlets from folks who say I do not even know what the federal qualified health center program is. Can you tell me more from rural health constituents who have great concerns about this and want to know how to build these? I think once transparent information is available, the constituents grow in number and in scope.

DR. GREEN: What is that map and what decision did it affect?

DR. BAZEMORE: And, again, I think that is the beauty of a dynamic tool or system. I will echo Bridget in saying we have tried to make sure it is not just a map. It is tables and histograms and tools all wrapped up in one. The map just seems to be as we have seen in usability testing, the way you get people engaged. They understand communities through geographies. In this case, we are sitting down with a pair of grantees in Petaluma, California who said we are concerned over expansion and we are worried about investing the money and the time. Can you help us understand? Looking at the various shades of green going from light to dark where you have increasing penetration of low-income population.

And around Nevada, we are thinking about putting a center up, but it certainly seems that your darkest green area suggest there is an awful lot of health center patients sitting on top of a pretty high density poverty area. That may not be the place you want to target. Your hash mark starts to say how many grantees are actually working in this area. To the project officer who then joined us in the conversation said we are getting as I mentioned earlier multiple applications to serve this one region. Where can we find the sweet spot? Least number of grantees so you have less — you have more dependence on a single grantee, high densities of poverty, and really low penetrations, low rates of utilization of community health centers, which they knew in the area other than the ER because that is the other piece, Larry.

You sit at the table as we spend a lot of our budget in doing is making sure we are getting out to regional meetings. You sit at the table and you discover all the deficiencies and your theoretically transparent data projections. When they say you know what you did not account for? The private practice here or what the hospital is doing or what really we have been able to accommodate with nontraditional, primary care sources of care delivery in this area. It was a conversation starter about where you might put the next grant for community health center service delivery.

DR. GREEN: Was a decision made?

DR. BAZEMORE: In this particular case, our follow up is that they decided — that the leading grantee said yes. We are going to go ahead and take our center and double it. The one I worked for in Highlandtown used a similar map. Having been nervous for five years with a 20,000 square foot footprint in the middle of urban Baltimore about its big neighbors, Johns Hopkins and Johns Hopkins Bayview said we could not possibly compete with their ERs. We showed them no. Your penetration rate. The number of people walking through your doors on a daily basis says you can not only compete, but you are going to lose the patients you have to them because you are not currently staffed or foot printed adequately to serve the community in the degree of dependence they are showing on you.

DR. COHEN: A couple of observations and questions. One that I would like all three panelists to think about for their small group discussion. What can the federal government be doing to help promote the issues and address some of the issues that you brought up this morning? The second comment related to something Andrew said and Carla and Denise said earlier. You mentioned the agricultural extension services. I think the health information exchanges. We have not really explored the potential. There will be these entities created ubiquitously throughout the country as potential repositories of data to get to communities when we think about how communities might use information that is being generated from EHRs. I would like to think along those lines.

Something Bridget mentioned. I know BRFSS is expanding its focus from state to county. The new BRFSS directive and initiative is to focus more on county. There is a lot of activity going on in small area estimation. My general question, again, not for discussion now, but is how acceptable do we think modeling approaches will be for communities to use around some of these issues and whether the federal government should be promoting these kinds of activities.

Finally, just a quick anecdote. One of the communities I work in. We did our needs assessments we spent a lot of time collecting secondary data. We did about 100 key informant interviews and essentially everyone was telling us mental health is an issue. We had no secondary data that could confirm it at all. But it is clear that the stress on youth, the sandwich generation and issues of aging were all connected to what we are calling mental well being, community mental well being and mental health issues. That is an example of needing to go at the smallest level and perhaps focus on qualitative data as much as quantitative data in order to really understand dynamics in a community.

DR. TANG: Mr. Chair, you used the five to seven minutes you gave us. I have three more cards. Do you want us to continue or what?

PARTICIPANT: Let’s ask Debbie. How long do you need for our —

MS. JACKSON: Ten minutes.

PARTICIPANT: Can we do five more minutes?

DR. FRANCIS: It was clear from all of your discussions that smaller group information or smaller location information is really important. At least Barbara mentioned one important issue for communities is that they not be portrayed in a way that disadvantages them. I thought I heard at least two other important features for communities that the data be free and that the data be usable by them.

What I would love to ask is whether any of you have any further thoughts for the whole group about what you are hearing from communities about more local data and how to protect it. Are you hearing anything when that data is sensitive like mental health data? Anything more we should all know?

DR. BAZEMORE: I did not show today where we really got started in moving from one off relationships with, for example, community health centers or delivery agents into an online tool. But our first step was to build health landscape. And the first things we heard from the community users was the questions we are most interested in answering are some of the most dangerous. We likewise heard from our IRB that was responsible for governing our exchange of information, which we hope to publish one day was you are going to have to be careful. You are walking into arenas for which there are no rules and guidelines. Absolutely with only HIPAA and a few other sources as our guide, I think you are forced to help communities where they really do not know what they are capable of risking to do things like limit extents. If you are going to show a map or even a table, what is the smallest unit or smallest area at which you can avoid risk?

There is big debate over whether if you get smaller than five of something or is it 10 or is it 15. Do you reveal the risk of Ms. Smith’s HIV diagnosis being known? Can you deliberately and randomly reassign a location? Does that help you or does that actually make Ms. Jones suddenly appear to have HIV when she really does not? Can you build in ways that community data users do not have to be working in a public space? Can you build those tools or help them with processes by which they can upload their own information, which is infinitely more powerful than anything that we are able to aggregate. I think Chris and Bridget would agree. We are always stressed by how do we get to more and more granular areas. We know we cannot. We know the user holds that information. Can we give them spaces where they can interact their information with ours and not let anyone else see it to protect it and make good decisions?

MS. MILAM: I am thinking about a few things each of you have said, but particularly, Barbara, about your comments around the evaluation component and some of the data that was collected. Leslie spoke to areas of data sensitivity. Typically, I guess from a legal standpoint when we think about sensitive data we look to state law and the traditional categories, your mental health, your HIV, et cetera. And each state has its own laws. And the risks that those laws were designed to protect against would risks to reputation, job loss, financial identity, that sort of thing.

But you also talked about some of the data that is collected has to do with what community residents see going on in their neighborhoods like crime on the streets. I am thinking that a risk to protect against could be injury to life or to limb, injury to an individual directly. That is a different kind of risk than you typically think about in data.

I am wondering. Were there models that you all utilized to protect against that risk to deal with data collected by the community? Did you all look to state models or other models? Did you all identify those sorts of risks? How did you think about that?

MS. ZAPPIA: I am not sure I understand your question. The risk of actually going door to door to collect the data. Is that what you are referring to?

MS. MILAM: No. I was thinking if you are identifying that drug deals are happening on a certain corner and that is in a community plan as something to be addressed, there could be people unhappy with that. Want to know who identified them. If the community members were collecting the data, were they educated about protecting confidentiality? Who maintained the results? Not just in that data point, but sort of the bigger question. That is what peaked my interest. There could be a lot of very sensitive data gathered by people who do not normally collected data. I am wondering. Did you guys have anything in place to train on confidentiality, to protect it ongoing? What models perhaps were utilized to ensure confidentiality if it was insured?

MS. ZAPPIA: The grantees working in the neighborhoods had in place systems so that if people were uncomfortable sharing data that they felt vulnerable, they had systems to do so confidentially. But we found in most of these neighborhoods that people were so fed up with the activity that is happening in their neighborhoods that in public spaces in community meetings they would stand up and say if you do not stop the drug dealing on the corner — until you do that, we cannot do anything about the health of people who live here.

DR. NICHOLS: I am acutely aware that I am the only person between us and learning what we are going to do the rest of the day so I am going to be really quick. Those were great presentations and I would like to ask 30 questions. But the two things you said, Barbara, that really are pounding in my head is the importance of counting rats per capita and the notion of an authentic community not dovetailing with the jurisdictions God has given us. I guess I wanted to really just ask this specific question. Is it the case that the residents did not think the county data were accurate about diabetes and obesity or whatever or the simple case that the variables that were collected at the county level are the wrong ones because we haven’t gotten to the rats?

MS. ZAPPIA: The rats were the things to bring people out of their houses and start talking about health. The prevalence of diabetes in their neighborhood was not something that they could get motivated to do something about. You talked to folks in the neighborhood and they do not discount the data that is available at the county level, but they feel like they know what is going on in their neighborhood. They want a chance to work on their priorities first.

MS. KLOSS: The great thing about our format is that these discussions will just continue.

DR. TANG: I will try to lump a little bit of what we heard today. One, very inspiring uses of data. Two, that the best way it seems that we can help communities make use of data is to offer them convenient, one-stop shopping for data particularly if you supplement it by local data on things that matter locally.

And finally, the notion that if communities would appreciate the help we talked about in our report, but the coaches that help them make use of data to improve health in their local communities.

But I thought this was an excellent panel. Thank you so much for the panelists.

(Applause)

Agenda Item: Review Logistics

MS. JACKSON: Debbie Jackson here. I want to welcome all of you. This is an amazing opening for this show. I feel like it has been a production. The curtain has gone up and the show is going on.

We are in decent time here. Mainly I wanted to do some people moving information for you since this is a very strategic gathering and a lot of moving pieces. All of you at the table have folders of certain colors and you might also have dots on your nametags because we placed you strategically in one of three panels.

If you will notice also within your folder is your name identifies which panel you are in. Each of the speakers is one of these three panels. Red, yellow, blue. I have been saying that as a mantra. That was done not as an accident, but thanks to Maya and others to help spread the types of expertise in these panels in these groups. We have these small groups where you will do a discussion. All the groups are the same in terms of what you are going to cover. You are all going to cover the same questions that are on the agenda. As you can see, we have a certain amount of time. We are trying to stay on schedule and on time. The panelists were wonderful in keeping your information at the time requested. Thanks for that.

The yellow group will be in this room. This is the larger of the three groups. This is what also has the video of the teleconference, the telephone access. We will have a telephone conference coming in later on this afternoon. Those who are in yellow will be in here. The other two groups are across the halls in either room 1406, 1404. Their identified as well. If the participants of the table would identify where you are going to go. When you come back from lunch, you will not all come back together in plenary. You will go directly to your breakout rooms.

We have a nice number of assistance and staff and experts to help organize the discussion starting with the facilitators, very skilled facilitators and moderators. These folks will help you stay on a schedule. Mainly the facilitators will look at the timing. The moderators are subject matter experts. You might see that term in your list as subject matter experts to help crunch your information. The facilitators will look at your timing and say this is a great topic and I know you are high powered on this, but we need to move forward especially with the time we have. Listen carefully for that.

The conference and this roundtable is being broadcast on the web, but only the plenary sessions. This section that is here in this room. When you go into your breakout, the web folks see a page that says thank you for joining and follow us at the next time. The plenary session is the only one that is broadcast on the web. But there is capability for — as long as it is working for them to submit questions, comments as information like that comes up to be as inclusive as possible. I will bring that to your attention.

Also, at the table are ground rules for the breakouts. The facilitators will be the main keepers of that. But just keep those in mind so that the information can keep going and flowing and covering as much information as possible. Bruce made it clear that the questions are for priming the pump. It is not like every question has to be answered. But we were thinking as things get so exciting and talking especially with this first breakout this afternoon. We have two breakouts this afternoon. Tomorrow we have two, but one in the morning and one in the afternoon. This time we have two in the afternoon. That first breakout when you come back from lunch is going to be a lot of meeting each other because you are going to be together four times.

With that, you may not be able to cover all of those questions. Take a look at the one that Bruce mentioned about the federal role and maybe the red group. We looked at blue, yellow, and red. Maybe red can start from the bottom up to start thinking about questions. Blue bottom down. And then yellow in the middle. Just to get an idea so that all of these are covered as much as possible. Those are the ground rules.

We will have recorders so that your information can be transmitted at the time of report out this afternoon. But I think as I mentioned, we do need a reporter for each of the groups: blue, yellow, and red. Especially just for this first one to see who might want to help report out in the plenary session what your group has covered. You are going to have the information included on Power Point. Your report is going to do that kind of work, but just kind of identify who — the recorder is going to help you pull the material together, but figure out who will be reporting out for that afternoon session after you get back from the breakout.

The main thing now is lunch. We do not have food capability in this building, but across the street, across the boulevard, there is a small deli that you can get food and that is out to the right. Out to the left there is a garage. There is no food over there. To the right, you will see a whole section of restaurants, your burgers, fries, and your pizza. There is a Chinese restaurant as well for quick foods. Otherwise, we just have the things just for drinks and stuff here.

I think we are on schedule. Maybe ten minutes. If we are looking at getting back together as far as the time element, what are we looking at?

DR. COHEN: That is fine. When folks come back from lunch, if you have a yellow folder, you will come back here. If you have a red folder, you will go to room 1404 across the hall. If you have a blue folder, you will go 1406. If you have no folder, come here. Everyone is free to enjoy themselves and have lunch. We are returning at 12:45. We have 55 minutes.

(Lunch Break)

Agenda Item: Report out from Small Group Discussion 1

MS. KLOSS: We are going to start with the red group. Who is recording on behalf of the red group?

DR. CARR: Well, we had a very robust conversation. Then proceeded to tried to — we do sit down to three main concepts and some cross cutting issues. We talked about the data and we were very granular in some cases talking about the EHR data and the challenges with that as well as big data. We have lumped it all together to say there is a place out there where there is data. We are called that data commons. One of the things that is needed is an inventory of that available data. In many cases, data is out there and people do not know that it exists. That includes federal, state, county, EHR, and locally collected data.

The second overarching category is access to data. It is evidenced by the questions. A key challenge with access to data has to do with the expertise of the workforce. There are people who are trained informaticians who are very nimble with this and then there are people who know what they want to do, but are not familiar with the data. That is, you think about the different job descriptions along the data continuum, the variability or the mismatch of the expertise with the skill of the individual and the expertise needed to use the data creates problems.

The third was a very large discussion although it is simplified here in one line in terms of the use of the data. A key issue is obviously validity of assumptions. One can pull data and draw conclusions, which may or may not be true.

But actually the fourth issue was what a lot of the conversation was about, the cross cutting issues.

In each of these areas, the data, access to the data, and use of the data, the spectrum of protection that is provided for that data.

Second cross cutting issue is the spectrum of data integrity, data that is collected in a precise manner, data that could be tweeted data or crowd sourcing data has variability in terms of the integrity.

Third cross cutting issue are the issues of standards, interoperability, and harmonization. And then finally fourth cross cutting in each of these areas is governance.

I will open it to the group to say if you wanted to elaborate on any of those. We did not go question by question as outlined in the agenda. But as we look back at this, each of these issues is addressed. How communities find and use data gets to the data common. Do people know about it and where do they find it? With the proliferation of data and data sources, how does a committee know how to choose the data? I think this is where the skill set comes into play, people who are proficient and people who are novice. And how to best promote openness, transparency, and choice when collecting data. A key point that we discussed is transparency when the data is collected and maybe the transparency is this will be identifiable and it may be used for a number of applications and just making sure that folks know that.

Fourth, what analytic and technical support can local, state, and federal government provide? And, again, this gets to the expertise, trying to co-locate the expertise so the people know where to find it and how to access it. We did not get to how you partner with health care providers and academia except to say that EHRs have not achieved the potential that they might have in terms of enhancing as data commons. And then how do communities deal with the lack of data content, standards, and variability across data sources? What can be done? I think, again, it gets to looking at the data integrity. Some data that can be very semi-quantitative is good enough. Other times you need very precise data. I think it gets back to the expertise of the workforce so that the knowledge necessary for using that data is possessed by the person that is putting the data together.

MS. KLOSS: Thank you. Let’s move on to the blue group.

DR. LOVE: Caving to peer pressure, I am the reporter for the blue group. We had quite a bit of robust discussion about how communities operate and find data. I will just rely on my group to fill in the gaps because I was either talking or thinking about something else.

Some of the insights we brought away from our discussion are up here. That communities we felt are doing a lot of discovery over and over and duplicative and spend a lot of their resources on the front end that could be better spent maybe with improvements just finding where data reside.

We felt that any community-level initiative really suffers from a lack of infrastructure be it data or analytic so that challenges them in setting priorities.

We talked a little bit about some of the shifting terrain around us be it the requirements and the ACA. We have social media coming on. We did not really get into EHRs and all of the implications around there, but they are huge. We did talk a little bit because a couple of us are involved with gathering cost data. That is changing a lot of the discussions around community collaborative. And big data. What does it mean? Big data mining, big data initiatives kind of change maybe how we have done business in the past for measurement and community assessment.

We talked a little bit about patient-generated data. I do not know a lot about it, but that sounds really interesting. And identifiers is my big thing. That won’t go away. And access to tools. All of these are part of this shifting terrain that are bundled into any community initiative.

And then we talked about data intermediaries and perhaps playing a larger role in accountability of the data and monitoring that.

What is that catalog or the essential data sources minimally that every community in theory has access to or should have access to as a starting place? And then the resources on the planning end of a community initiative as I remember it could be used to filling those gaps of data that do not exist instead of chasing data and spending two months finding out where SPARCS data live and who the contact person might be. That might take three months if someone is coming in and not familiar with data and the lay of the land.

And then we need models and this I might need help with, but communities that do have infrastructure are sustainable and use community coaches. What do they do right? What are the best practices and what can we document for others to model their own community initiative? And then we talked about the levels of data access, open controls to the spectrum that we just heard about to restricted and we have tools for those.

But I would open it up to my group to fill in the missing pieces of this robust discussion we had. I know we talked about APCDs and identifiers and cross jurisdictional data exchange and some of the silo data issues. Again, we did not really have solutions for these, but I think that is just reflective of some of the challenges we all face and some of the terrain that community initiatives find themselves having to navigate.

MS. KLOSS: The good thing about this first exercise is we did not need to have solutions, which is a good thing. We will work up to that. Yellow.

MS. CHRYSLER: We had a little bit of difficulty getting going. I think it just demonstrates how hard it is to get your arms around the kind of big issues we are dealing with and how you can drill down enough to even have a way to look at issues about data collection compilation meaningfully. After a false starts, we saw that a good place you organize and build your thinking is about a use case. We chose obesity as our use case because it affects everybody at every level. The sources of data can be moddable. It was a great example of how you drilled down. You look at this is the broad problem and then what are all the factors that go into the broad problem and then how do you measure those and where does data come from and how do you use it. Those sorts of things.

We really wanted to just not look at what is the current status of finding and using data, but also where do we want to be. What is our potential desired state?

We understand data collection to cover multiple ways of acquiring and accessing data. We spent a lot of time talking about data acquiring and accessing data from the neighborhood level. We spent a lot of time on a neighborhood level. Where do you find data? How do you use like data and what are the lessons learned from the Rochester example? That certainly was starting with the galvanizing health and well being issue. In that case, it was rats. From there, people go into all the other issues that affect their community.

The need for locally, meaningful boundaries, not just geographically jurisdictional boundaries, but what makes sense to the people in the community. The goal of organized change. We repeatedly reminded ourselves. Data is not just data. What is data that is we can use for action. That data comes from moddable streams and how do we integrate data.

We spent some time on electronic health records, for example, and health information exchange and how do we include that as well as population health data. And how can data be not just bidirectional. That is an important piece. But also how do we return data to the community and help the community benefit from the data.

And then the co-production of data and purpose, which is defined by the stakeholders. How can we at a neighborhood level, for example? How are the people within that neighborhood part of the collection of data and the use of data and the analysis?

Certainly, not to stop with the federal data and I will extend that to government data period and be creative about finding, collecting data from numerous sources.

And then I had already mentioned the electronic health records and feedback loops of data back to EHR. I had already mentioned marrying data from multiple streams and integrating that data.

I believe this focus on function and looking at models. Don’t try to duplicate the actors as these vary from place to place. My memory is and I will ask my group to join in is that in looking at data collectors and we talked, for example, about school nurses and BMI that it is important not to look at a title or a specific position, but to look at the function of that kind of purpose because a lot of school districts have two nurses if they are lucky for thousands of children. How do we look at the function of those people? That is going to differ from community to community, place to place.

The whole notion that everybody in the community needs to be. They can function as a data collector. This goes back to the co-production. And that, again, data is the enabler. Data is for purposes of action. It is not the end in of itself.

I would like to ask everybody from the group. Do you have anything to add there?

Our experiment was who could sit still the longest or shortest time before they could not stand it and nobody would volunteer and I just could not take everybody sitting there like that so I finally went okay.

MS. KLOSS: We appreciate that. And the good news is that now you are done. Any observations from these three report outs? Any consistency, convergence? I heard some descriptors of the current state, chaotic, plenty of gaps, but some identification of cross cutting issues. And then a focus on some descriptors of what the future state might be using use case or some other mechanism to move from where we are at today to where we think we want to be. Any ideas that came out in your groups that did not get reported?

DR. GREEN: It is a small aha. I am struck with how the three groups — I am sitting here taking my own notes and I am organizing my notes the way I want to organize my notes, which is around our goals for the four things. Our first two goals related to data acquisition and use and the next one about stewardship. Most of what we just heard really fits very nicely into our goals. I find that reassuring.

MS. KLOSS: How about any feedback on process? We are going to —

PARTICIPANT: Do we have to?

MS. KLOSS: Yes. I felt our group in the yellow group we probably had ten minutes of storming and norming, but you always do when you start out with a small group and a very big question. I think our breakthrough came when we said let’s not try to boil the ocean. Let’s just think this through using one example. We had consensus around that being a point to give us a little more comfort. Any other observations? We are going to have this perfect by the end of the day tomorrow.

MS. HOFFMAN: Are the questions for all the breakout sessions or different? They look different to me. I think Justine has pointed this out too. It is a lot of questions and we had trouble which question do we address. Do we have address all of them? If we want results, it might be useful at least to prioritize because in two days we cannot answer 20 questions.

MS. KLOSS: I will speak for the yellow group. We ended up pretty much focusing on the overarching question. How do communities collect and compile data? We did not drill down methodically although we certainly touched on points in the questions. We felt okay with that. We gave ourselves permission to do that.

DR. COHEN: I think we went back and forth about whether it was appropriate to assign the questions to different groups or try to go serially to answer them. We decided we would leave it up to the groups about how they wanted to address the issues. Our intention is not to prescriptively answer all of these questions, but to have these sub-questions as a guide and a stimulus for discussions around the broad theme. However each group wants to use these questions is fine. Don’t feel that you need to go through all these questions. Don’t feel you need to select one or two, but whatever works for your group.

MS. BERNSTEIN: I am noticing that what is going to happen on day three is that Leslie and Sallie are going to be trying to coalesce the ideas that have come from all these discussion groups. There are a lot of points on each of these. I think we probably have to get ourselves to force ourselves maybe to just pick those three, those four things that are the most important and just focus on those. We do not really necessarily want a report out of the entire discussion that happened in the discussion group, but just really try to report to us those things that you really want the committee to capture when they are thinking about their report or recommendations or whatever next action they are going to take that is going to speak to the department. Does that make sense? This is a lot for them to try to put together at the last day.

DR. FRANCIS: We will work on it.

MS. KANAAN: My sense of the first session was that it was very much about establishing a thought process that was going to work for the group and that the content was almost less important than the way we were agreeing to think about things. That will inform the way we go forward hopefully in each group.

MS. KLOSS: All right. Are we ready to move to our panel two?

Agenda Item: Panel 2: Using Data for Decision Making

MR. BURKE: I am Jack Burke. I think I have met almost everyone in the room. Our second panel is devoted apropos of where the first panel ended, which was data is an enabler, not the end. The second panel is devoted to relying on data to support decision making. We have three panelists who have teed up their slides — passed out to you I think about 20 minutes ago in hard copy.

First is Lacey Hart. Lacey is the director of Health Sciences Research Project Management Office at the Mayo Clinic. She has an MBA. She is certified in project management by the Project Management Institute. At Mayo, she is the program manager for a variety of research programs including those awarded by HHS, the national coordinator, secondary uses of EHR information, the National Institutes of Health, and the National Cancer Institute, and perhaps most significantly and I believe what she is going to focusing her remarks on is the Southeast Minnesota Beacon Community project.

MS. HART: Thank you. Good afternoon. How many are familiar with the Beacon communities? You are familiar with the Beacon communities really taking the HITECH in their communities that really leverage adoption of EHRs, meaningful use, health exchange, and then there is the research component. Just kind of a culmination of all of those activities. Now, anyone who is familiar, one caveat is these communities were funded at the same time as these other activities, which makes it a little difficult, but nonetheless we have made some achievements.

There are 17 across the country. We are smack in the middle of this geographic map. Then of course this is the aims of all the beacon communities to build and strength health IT, improve cost, quality, population health, and of course test innovative approaches to both technology or care delivery. And each beacon approach this a little bit differently. That is the basic foundation. But I will talk very specifically about how our beacon community applied that.

In my discussion group, we had a talk about what is community. This is my geographical representation of my community. We do represent the Southeast region. There are 11 counties. We picked this when we wrote the grant based on ease. This was a state-recognized region, if you will.

Now, we are comprised of all of the local public health departments are actually funded through the grant to participate with us. Along with all of the major medical centers, we have in our region integrated medical centers. Hospitals, clinics are jointly operated. We do not have with the exception of two although they are connected to a health system, we do not have private practitioners necessarily in the region.

And then we do cover 47 school districts. Now from a patient population, we cover both those that reside in this regional area as well as those who receive care in our regional area.

We have a continuum of care here. When we approached our beacon, we really had a community around the table from the get go and we started looking at how did we wanted really tackle this challenge of health information exchange in particular. One of the things we noted is that around the table we have different varying levels of engagement, cost ability to participate, and just different use cases. You guys kind of talked about a lot of this in the previous session.

We have those that are health care systems that have health data. They have a robust infrastructure. We can debate how robust, but they have a robust infrastructure. And they of course can get meaningful use funding.

Then we have those in local public health departments who have, again, valuable health data, but limited infrastructure capacity and no funding to work towards this goal with the exception of in our community we had some beacon dollars.

And then we have school districts, health facilities. We have just home care, lots of different day cares, et cetera in the community, but again, have valuable health data. But again, no infrastructure capacity. We say limited. Pretty much they have none. And they have no meaningful use funding.

We said how in the heck are we going to do this and what is the guiding point. We talked about this in our breakout session as well. What kind of data are we going to focus on? How do we exchange it and what is the end goal?

But before we did that, we started with actually asking our community. Actually asking members that resided or received care. We did what is called the deliberative democracy. You first bring in and we went to various regions of the community. There is a whole way of selecting how people can be brought in so all walks of life can be represented. But you bring in experts. Bring in the IT expert. Bring in the data privacy expert. Bring in the clinical care expert, public health, et cetera and have them speak to what is possible and maybe what some of the constraints are. And then ask the community what are your hopes and what are your fears. At that point, the experts leave and it is a true qualitative focus group session where they can share openly and freely what their hopes and goals are.

This turned out to be the most valuable thing. We were the only beacon that did it. I am convinced that this why we actually did things quite differently than many of the other beacons because our community spoke. Really everything we did was from a provider rather than from a provider view point, which was where the funding was coming from, but rather from the patient and the community viewpoint.

Just to note real quick from an HIE perspective, we picked a peer-to-peer network. Some of you are familiar with the jargon. But essentially instead of all of us health care providers sending our data to a third party and having to exchange it for us, we exchange it with each party we want to exchange with on a peer-to-peer basis. Why is this important? This is hugely important when you look at health exchange, not from a provider view, but from a community view. If I want to exchange information with local public health, they cannot afford the infrastructure to join a typical HIE. This is how we are able to get our partners to participate in health exchange.

We were also lucky in our region just to also point out that this while there were skeptics at first, this is becoming a trend nationally of how do you start connecting. We did have the privilege of joining some other large organizations with the same infrastructure. Showing how this infrastructure can exchange information at a local level all the way up to a national level.

Some of what this infrastructure allows us to do. Here is a use case of what is important in our community. Transitions of care. Simple things. When you look at it from a community view, one of the things you start finding out is who does have the data and who does the work. We found overlaps and we found some serious gaps. One of them is when a client of public health is admitted to the hospital. They get admitted and they get discharged. Eventually their case manager finds out about it, usually a week up to three weeks later. They then check in on them. At that point, they have already been readmitted if there was going to be an issue.

Second, public health is on the hook for notifying the primary care physician after billing has gone through that their patient has been admitted to the hospital. Again, we have health care joined in network. That primary care doctor already knew. The public health was feeling a little sheepish sending them documentation. We took care of that with our health exchange.

At the point a patient gets admitted to a hospital, we are already know that they are a local public health client. We can notify the nurse manager at the point of admission, which means they can be at the hospital helping with the factors of that patient that then influence discharge. For example, do they have Alzheimer’s? Do they need to go to a special nursing home? Do they really have the support at home that they need, et cetera? As well as then we alert their primary care doc at the same time. It closes the loop on who all needs to know about this event.

We also have in parallel and there is a reason it is in parallel. We have a community data repository. It started as a clinical data repository. But again, with the public view, we have turned it into a community data repository and I will talk about what that means.

We do host it at Regenstrief Institute in Indiana. That was just out of the very need to have something quickly. They have been doing it for a long time. Also, the fact that they offer a repository with technology to silo data and protect data that many groups do not have. That was the reason we chose that.

The community data repository. We get clinical data, but we also get things like patient reported data. We get outcomes data, quality of life data. Right now, we currently get that from public health. We get medication reconciliation. That is a really cool one. With our health exchange, we do exchange data that can be parsed and consumed. If you are into technology, you know what that means. It is not just a PDF, but it is actually the data elements.

What we do in our community is now we know what has been prescribed. We may or may not know what is filled depending on how much you are going to pay a pharmacy for that data. And then the third is public health. If they are in the home, they can do a reconciliation of what was actually taken. What did the patient actually take? We can share that information as a community. Public health nurse can see what all the clinicians think this person should be on versus what they are actually taking when they come back into the clinical setting. What did they actually physically take? A little problem with that. There is no data standards for medication taken. We are having to make that up as we go.

And then social determinants of health as well.

In our state, we do have Minnesota Research Authorization. We enabled all of our community partners, not just the traditional large research shops in our region, but everybody to set up their data collection to be compliant with Minnesota Research Authorization. A patient can therefore choose how their data is used whether it is just for clinical care or whether it is going to be for research and that is all stored in the repository at a data level so that we can understand when we can and cannot use particular data elements.

Another aspect of how we use this data is in-reach social worker. Now, everyone swore at the beginning of this project our patients did not cross over these large health care systems. They swore up and down that our patients did have access or they had the means to ensure that their patients had access.

But we did a little experiment. We started talking about how do we reduce the cost of health care. We thought maybe we could reduce some of the cost with the assistance of an in-reach social worker. When we looked at the data of how we could use that, we found out very quickly that our patients do cross over and they do have access issues. And in fact they actually blew it out of the water here. We said if we could look for patients who had more than three ED visits over four months. If we actually tackled all of those patients on a month that we are doing that, we figured we would have to hire 19 to 20 in-reach social workers to help get them access. We have had to actually increase that number in order to fill the three social workers that we have. A much bigger case load.

But it was eye opening to everybody that really what they thought about their patients or their community or population it really was not the case. Looking at the data was very valuable. And then of course that allowed us to really hone in on where do we need to provide services. What do we need to do as a community?

Public health surveillance. It can be used for that. We have just a heat index here. It was one example that we looked at. But just understanding of the source of the data when it is happening across that there might be an issue as opposed to waiting for each individual organization to report up.

Now one of the big efforts in our community looking at this from a community view, we did pick two chronic conditions to focus our measurements on. We did not stick exclusively of those. Those were just the ones we looked at to see if there was improvement.

With children, we looked at asthma. We had experience in our community in a particular county that by sharing the information that this child has asthma particularly when they spend most of their day at school, we could impact how their care whether it was — if there were incidents that were happening or whether or not just recognizing they have asthma in the school and what to do about that. It turned out you can do this. It is pretty valuable, but how do you exchange the information.

This exploded a whole host of questions. We started out with how do you just exchange an asthma action plan, but the reality is how do you just talk about health issues in school. How do you address the fact that schools — this is not their number one priority, but it has a direct impact on their number one priority. Are kids in school? Are they testing well? And just generally doing well at their academics. This really challenged how do you do it.

Now this gets to who owns the data, who stewards the data, who governs the data. We were able to figure out how to export the data from the health records. We figured out how we could send it to school nurses. Parents didn’t just want us doing that. How do you consent? How do you make sure just that data goes and no other data? They did not want everyone at the school to have it available to them. How do you make sure that there are protocols in the school? We were successful in doing that and able to come up with what we called cocoon of care, really kind of taking care of that child. Now we did reduce ED visits. I did write that down.

At the start of Beacon, our children had about — of those who had identified asthma, we of course found a lot more kids that had asthma than we thought. But we had about an 8 percent of those kids were hitting the ED regularly on a monthly basis. After we have been doing this for two years, a percent, one and a half percent approximately were hitting the ED.

Essentially what was happening is with this communication loop being closed, the school was not just calling the parent and sending him to the ED if something happened. That was the biggest reduction. It was during the day.

The second thing that happened is when things were happening or going wrong in the school, which that information may or may not have made it to the parent, it was getting reported. We can now do an instant report back to the parent and to the provider. Johnny is in the office three times a day with his inhaler. Is that expected or not? How do you tackle that plan of care?

The other aspect is just shared decision making. Once we found we had data from a community view, we had data from nontraditional providers or people caring about the health of their community, we still found gaps and it is usually at the patient level. It is usually the patient themselves that were missing data. We are missing information about their lifestyle and their preferences and how they want their care delivered. Looking at diabetes as our kind of condition, we started tackling how do you bring that into the conversation. How do you bring in information about you as a person into your chronic care?

This is just patient-driven decision aids. You know you have all of the clinician decision aids that are out there based on data. And they then drive what is the course of action.

No different here. Here it is telling the clinician that this patient needs a statin. But instead of just prescribing what they normally prescribed, they actually asked the patient what is important to you. And in this case, this patient cost was an issue to them. Maybe they cannot afford to buy their drugs. If they cannot afford to do that then they are likely not to adhere to their plan.

Correspondingly, they get to make two choices. Maybe I find one that is cheap, but unfortunately maybe it makes me gain weight. I am not going to stick to it. It really brings in their preference.

Now, again, no standards for this. We are doing our best to capture this with the best way we know how, but how we would exchange that outside of our community I am not sure. There is not a standard for doing that. We do exchange it though. We have a lot of patients that gets transferred from primary care to specialty care. At that point in time, our data was showing that a lot of things like medications were changing. But the preferences needed to be captured. If an endocrinologist was making a change to that patient’s medication, they still had the preferences brought along with them.

The other data gap. This, I know, was on the agenda, but we have not talked much about it. Quality of life. What is your quality of life? How is this impacting you? When you are talking about health issues for us, we really tackle chronic conditions. A lot of times people are really focused on the condition, not over the long haul. This is something we argued with ONC quite a bit. When you are looking for improvements in a little grant period, they are always looking for this great peak of improvement. That is actually detrimental for a chronic condition over time. We tackled that measurement issue actually by tackling quality of life. Because if we could show over the short period of time that we are impacting their quality of life then we could project what their long term success might be if there are other clinical outcomes.

We have some smart researchers. They are actually working across the nation at what are the standards to measure this. Unfortunately, when you get into quality of life or patient-reported outcomes, the number of data elements that people try to capture are so large. They are usually from a research perspective. That really does not help you a whole lot in a clinical setting. There has to be a compromise.

We went from 45 questions that were recommended nationally down to just 10 questions. It really gets down to one actually of what is your biggest concern. From a community standpoint, how do we address the big concern of what maybe is impacting your quality of life at this moment in time and maybe making it difficult to adhere to your plan?

This is how the patient responded. Essentially it is about time. You asked me something other than my clinical measure. Actually if you know — that is Jason. He was our project officer. He was out guinea pig.

For the lawyers in the room, this was not easy. You will notice I said a few things. When I talked about data, I talked about it at the data level. I talked about it in the use case level. We did have to take consideration to work with the laws. We had state laws. We had national laws. We had conflicting laws when we went to the different settings.

What we challenge ourselves to do was really come up with where was the commonality and let the voice of our community drive us. This does still have hiccups. It has a lot of excess work. It is not a lean process in any way and that is something we want to strive to achieve. For example, if I want to exchange health information with a school, I still have to do it on a case-by-case basis, a plan-by-plan. And that parent has to sign consent at both the clinic settings as well as at the school in order to exchange. That is very cumbersome if you start thinking about how much time a school nurse has to spend just processing paperwork to make sure it is compliant when really she just wants to deal with the clinical issues at hand or the health issues at hand. That is something we are looking at. How do we streamline that? How do we make that better?

That is where you can find more information. Thank you.

DR. COHEN: We have time for a couple of questions.

DR. TANG: You mentioned the Minnesota Research Authorization. It is interesting. Minnesota made the news because of the opt-in. I am reading between the lines. You tackled that by having a centralized authorization database.

MS. HART: We actually have every possible consent, opt-in/opt-out and mixed modalities you can imagine. When we tackle this, we had to — that is why I said we did tackle it separately and that was for legal reasons. We do not have a repository tied to our HIE, which means we are not using our repository to exchange health information per se. It does not allow us to then achieve meaningful use. There is a caveat there. Health information exchange is separate.

Then when we look at our data repository, we had to build the consent model around clinical care use, which is a different consent. Research is a different model. And then of course just some of the nontraditional reasons like exchanging it with schools, which actually is not even covered. We are just trying to cover it from all the basis. We have every possible iteration you could have.

DR. TANG: I guess it was not as easy as I thought you were saying. Where I was going with the question is could you do — if you had that construct of a central repository of standardized consents, could you move there to address your school issue or other places where health data are?

MS. HART: Yes. That is exactly. The one beauty of the repository as we have it now because it is stored in data. The data for every contributing group is siloed, which is one protection, as well as security as set at every data element. It does allow. If we could solve the actual paper and pen process, and make it more lean, the technology can catch up with that.

DR. COHEN: One more question.

DR. EDWARDS: That was a great presentation. The quality of life questions that you have asked. Have you all published that or is the results of that information available publicly?

P>MS. HART: Yes. There is actually the prior publications. The key researcher is Jeff Sloan from the Mayo Clinic. There is a patient report outcomes research area at Mayo. We work very closely with Dartmouth Institute and PRMs measures. These are aligned with PRMs measures if you are familiar with that. It is published. We first did it with Cancer. We are now publishing right now the diabetes aspect. And then what is soon to come is the look at just chronic conditions and how we apply that tool in public health, which is actually different than the clinical setting.

DR. COHEN: Thank you. Next up is Dr. Ninez Ponce who is joining us by voice. I think we have her slides. Dr. Ponce is professor at UCLA’s Fielding School of Public Health and the Department of Health Policy and Management. She is a principal investigator with the California Health Interview Survey conducted at the UCLA Center for Health Policy Research. She is an expert on health insurance, economics, and racial/ethnic health disparities. Dr. Ponce serves on a California Health Benefits Review Program and is an expert at measuring race ethnicity, physician-patient communications, and discrimination. Dr. Ponce, can you hear us?

DR. PONCE: I can hear you.

DR. COHEN: Welcome.

DR. PONCE: Thanks, Jack. Will somebody be advancing my slides for me?

DR. COHEN: Yes.

DR. PONCE: What I will do is I will just say next to advance my slides. Are my slides up on a projector or do you folks have handouts?

MS. WEBSTER: We have them up. If you will just say next whenever you are ready.

DR. PONCE: Thank you. Thank you for inviting me. I am at UCLA. I am here to talk about two examples. When I say the proven, a survey that I basically grew up with here at UCLA, the California Health Interview Survey. It is a durable proof of concept for communities transforming survey data, population-based survey data into policy action.

Then I just wanted to touch on the possible. This is a project I am currently engaged in with a collaboration of community health centers where these new models of engagement of employing community-based participatory research or CBPR where the community health centers are the ones that are driving what the research questions are because they know what problems need to be solved. They are the ones that own their registering clinical and administrative data. They are the ones that set the rules of engagement.

First, I am going to talk about the California Health Interview Survey. It is the source of state and local population-based health data. I emphasize state and local. It is designed from the ground up for communities to use the data. To support decision making at the local level largely at the county level and for some of the larger counties like Los Angeles and San Diego at the sub-county level. For example, in Los Angeles, it would be service planning area.

It is also designed to measure health needs and disparities in California, race, ethnicity, being one of the premier concerns, but also by geography, by age, by gender, social class, and also by sexual orientation.

It is a telephone survey. It is a random digit dial survey. There is a landline component and a cell phone component. It has been conducted since 2001. We have completed five waves of data collection. We are in the field for our sixth wave. We collect information up to 50,000 or more adults, teenagers, and children. We conduct the survey in English, Spanish, Mandarin, Cantonese, Korean, Vietnamese and starting this year in Tagalog.

It is funded by several entities, federal and state health agencies as well as foundations both in California and nationally. Also some NGOs also fundus.

I also want to acknowledge because it is stakeholder driven. There is a lot of in-kind uncompensated support from members of work groups, technical advisory groups, and advisory boards.

The formula is simple. We make data meaningful for counties. We make data meaningful for racial and ethnic groups. We free the data, of course, with all the protections and privacy consideration, but we make the data available as soon as possible, typically about six months post-data collection towards the end of data collection so that the data is fresh to inform policy action.

This is a slide from an article by the founding CHIS principal investigator, Dr. Richard Brown. This was a model on how we had envision CHIS at the drawing board to be a model for participatory research and that we wanted to achieve this even though it was a very large survey.

The model is a hybrid approach. It is still very much research led with stakeholder and community input and the PI, the governing boards, the director and the team. They are in the middle of this conceptual framework. But it is informed. The content, the kinds of questions we ask, the sampling. It is informed by the communities. When we first designed CHIS back in 1999, actually even earlier, that we went up and down the state making presentations to several communities in the state and informing these several work groups and technical advisory committees.

We want to make sure that the survey is relevant to the communities that planned it. And that it appropriately measured factors related to their community needs. I think a hallmark of CHIS is that, again, this freeing or democratization of data is the data results are available and accessible to all the communities and their advocates.

Here is an example of cutting and slicing the data so it has some policy relevance. We have CHIS data, health statistics by legislative districts. This is available through a web query. We recognize the need to examine health statistics are bound by political jurisdictions. There is a resource to look at health profiles by assembly districts, state assemblies, state senate, and US congressional district.

What is in these types of health profiles are a sampling of the health profiles that were informed, again, also by community input, but also since CHIS has been around since 2001 and we have something called AskCHIS, which is a web query system. We use top hits of the queries of what consumers of CHIS are looking up in terms of what types of health statistics they want. This is a list here of the profiles: smoking, diabetes, obesity, sedentary behavior, psychological distress, asthma, food consumption, poverty. Towards the bottom are coverage concerns: uninsured anytime and then also — we have also constructed something in anticipation of the ACA enrollment, those who are exchange eligible under the Affordable Care Act and those who would qualify for the Medicaid called Medi-Cal in California expansion.

Easy access to CHIS data and findings. There are multiple formats. We have the publication on the web, as I have just demonstrated with the health profiles by legislative districts. We also have an AskCHIS online query tool. I will show you the screenshot in the next slide. We have data files for use for researchers. Some files are public use so just readily downloadable. And then some are confidential. The confidential files include variables that are sensitive. For example, sexual orientation. But also we suppress any kind of sub-county level geo-coding.

This is the screenshot of the AskCHIS web query system. You do have to log in. I won’t go through the demonstration. In addition to having this available, we also have a team called health data that are trainers of the use of AskCHIS and they train community organizations, schools, media, legislative staff, on how to use this web query system. The California Endowment has been a major supporter of CHIS especially the dissemination mission of CHIS.

Two examples in use of CHIS for policy. Examining the food environment in Los Angeles and predicting enrollment in ACA once the ACA program rolls out in California.

Just review of this. The food environment in LA. This is some work that one of our researchers at the Center for Health Policy Research, Dr. Babbi had conducted with some community groups. She basically linked BMI information with external data source on retail food outlets that were associated with fast-food establishments and showed that the more fast-food establishments in an area is a greater association with obesity among adults. In this analysis, it was designed and presented in a nonprofit group called PolicyLink in their report design for disease. And as a result of that, in 2008, the Los Angeles City Council enacted a moratorium on fast food establishments. In an area that had a lot of fast food establishments and it is associated with higher obesity rates in South Los Angeles.

Moving forward and being prospective, CHIS is also used to power the population source in a simulation model called CalSIM. It is a product of UC Berkeley Labor Center as well as researchers at the UCLA Center for Health Policy Research. It basically uses assumptions on health insurance and enrollment using the Medical Expenditure Panel Survey and then it is applied to CHIS, the California Population Data.

It is used by Covered California, which is the name of our new — first state health insurance exchange to understand what we would expect on who is going to enroll in the exchange in Covered California and who might enroll in the expanded Medi-Cal program.

It is also used by a community-based organization, for example, California Pan-Ethnic Health Network, which is a collaboration of several race-specific advocacy groups. They use CalSIM to predict that language barriers could deter more than 100,000 Californians from enrolling in Covered California and therefore the need for language assistance services during the open enrollment period.

To summarize with the proven with CHIS, some insights are that the stakeholder engagement occurred at the design stage of CHIS. I know when I started in 1999, I know it began even before 1999, with Rick Brown and Drew Holby(?) and folks at the Public Health Institute. The stakeholder engagement occurred very much at the design stage.

We really do believe in freeing the data of course without compromising confidentiality, but that the data is accessible and I think — with query system is a signature product in demonstrating democratization.

We are moving into having done this over the five cycles over the past ten years that we do believe that there are gains to sharing and banking, sharing some of the lessons, the ups and downs of what we are learning with the telephone survey administered to a large population-based sample. There are gains to sharing and banking some of the survey items that work and do not work particularly in different languages and for certain population subgroups.

We are still around. Community use induces more demand and also more innovation. For example, we want to innovate this AskCHIS system, which is right now at the county level to have small area estimation getting at the zip code level. We do get a lot of requests. LA County is so huge. I want to know more about what is going on in Korea Town — we are actually prepping for a roll out of unused small area estimation called AskCHIS neighborhood addition at the zip code level. This community demands then results in longevity of this data.

I wanted to end with the possible. CHIS is again proof of concept. It can happen. But it is definitely centered around researchers. The possible is the new models of engagement. I want to talk about a partnership. I am an academic partner for AAPCHO, which is the Association of Asian Pacific Community Health Organizations.

Beyond local quality initiatives, beyond wanting to improve the care given to their patients, I really think that CHCs — we are at a time now and maybe it wasn’t 10 years ago or 20 years ago. I was an advocate before I went back and got my PhD. But these — they are ripe for wanting to use their data to inform policy debate. They are very keen on understanding whatever comparative effectiveness means and they use their registry and claims data for comparative effectiveness that is not measured in the policy discussions where we use basically the large employer-based data sets and claims.

They want to be involved in risk adjustment and payment reform especially in the ACA. In particular, this risk adjustment includes or accounts for the notion of social determinants of health. I think that is very palatable right now. They want to know that they are compensated for care for complex populations and also for taking care of the nonclinical aspects of patient care such as transportation, language bridging services, ensuring that instructions are readable.

They are also interested in the validated measures of patient satisfaction. I think some clinics have come to me and said they do not really think that some of the CAHPS measures account — truly measure what satisfaction/dissatisfaction is. They are very interested in creating measures that jump off from national standards, but also account for local culture and local variability’s.

I am the academic partner and the contract with AAPCHO is bound by community-based participatory research principles. I need to have a track record with working with communities. There was explicit contractual language that states the plans on ownership of data and the scientific capital of the data.

The CHCs provide administrative data that is informative, but it is limited. This isn’t unique to the CHC data space, but for all health care delivery data space where it could be enhanced by linkages, for example, through social determinants that would be measured either through population-based surveys like CHIS or through a census, American Community Survey, or other surveys that get at notions of a social capital or residential segregation or income inequality.

Administrative data is observational and it is not experimental. There is also definitely ethical considerations of running interventions that are close to randomized control trials. But we can also help by examining with empirical modeling techniques like propensity scores and variables to model quasi-experimental conditions. This is I think what academic partners bring in this collaboration.

I want to underscore that at least with this group I am working with, this community is very sophisticated with framing the problem. I am not there to frame the problem. It is authentic engagement I think thrives because it requires constant contact, not just occasional signoff. I am not just drop in this academic and try to do the modeling for these. It is several meetings. I think I have more than one meeting a week with this community partnership.

It is democratizing the data. It is also of course much in demand. The linkages, quasi-experimental designs meaningful data has to be accessible. We are in negotiation. We are in talks in trying to develop an AskCHIS query system for their clinical data.

There is a lot of learning and understanding and explanation on why do we need standardized measures. I think there is more and more of a grasp, but there is a utility of standardized measures and especially if it is understood as widening the policy reach and impact. Again, beyond just a local quality initiative.

And of course patient protection is utmost. The group I am working with has a community IRB. There are data use agreements in place. There is secure data access in place.

To conclude, there are two models I am familiar with. The research led, with strong community input and the community led, with strong research input. I think both models operate on the shared vision that in order to make data meaningful for communities that you have to free the data and that this data then is pretty strong for policy action.

MR. BURKE: Thank you, Ninez. Any questions? It was a very powerful demonstration of the use of data for public policy making.

DR. COHEN: How much does this all cost and can this model be replicated in other jurisdictions? CHIS.

DR. PONCE: Well, we have over 50,000 observations. CHIS is expensive. It is between $16 to $18 million for a cycle, which is two years. So about $9 million per year.

DR. COHEN: This is Bruce Cohen. It is phenomenal. It really is a wonderful vision. I wish I could afford it, but I understand and there might be opportunities to leverage resources to do something like this in a variety of jurisdictions. This is very powerful. Thank you.

DR. GREEN: That was beautiful. Thank you very much. Could you say more about what constitutes a community IRB and how it operates?

DR. PONCE: The community IRB that AAPCHO has members from their member community health centers. AAPCHO is a collaboration of several community health centers. There would be somebody who is — ideally somebody who is a former patient, somebody who actually has a lot of direct service. And then it also includes — I am on it, but there is some academic presence in it. That is the community IRB.

DR. GREEN: How does it work?

DR. PONCE: How does it work? If a researcher goes to — wants to do an intervention at a community health center then that proposed project would be put forth in a community IRB. A community IRB would then review it. It works like a regular IRB would review it and then go through all of the checks on privacy and confidentiality.

DR. GREEN: I am sorry to be annoying.

DR. PONCE: How does it work? We meet by telephone. We get the materials beforehand. We have a checklist. We meet by telephone and then we discuss each project as a committee.

DR. GREEN: Does that IRB have all the authority embedded in any other approved IRB to review a research project?

DR. PONCE: You mean if the researcher already has an approved IRB safe from their university. If the authority from the community IRB would trump the university IRB. Is that the question?

DR. GREEN: Yes. That is part of the question.

DR. PONCE: One of the requirements would be it would be desirable if that research project has a university or some other state IRB, but it still has to go through the community IRB. It still has to go there as a check.

MR. BURKE: Thank you, Ninez. Can you stay with us for the rest of the day?

DR. PONCE: I have to teach class. If there is more interest in this community IRB, I could provide the context and resources on the set up.

MR. BURKE: I think there is and we have your contact information. Thank you very much.

Our third speaker is Dr. Eve Powell-Griner who is a confidentiality officer here at CDC’s National Center for Health Statistics. She is certified by the US government as an information privacy professional and has her doctorate in philosophy and sociology demography. Dr. Powell-Griner overseas NCHS’ restricted data access program and public dissemination of micro-data.

DR. POWELL-GRINER: My discussion sort of departs from what you are hearing from the other two before me in that what I really spend my days working on are upholding the promises that we make when we collect the data. One of the questions that we very often get is why can’t you just give us the data. We are going to use it for very valid purposes. Of course we are going to treat it carefully. There is a lot of answers to that.

One of the things that I want to talk about briefly is data distribution use and protection requirements, issues in data granularity, which I know all of us are very interested in since we are focusing on community level health, and then some of the steps that we might recommend to local health departments or local data collectors to consider in producing statistics that are safe that meet their needs, but also honor promises made to people providing the information.

Data distribution and protection requirements. Obviously, data are collected to be used. But the key question becomes used by whom and for what purposes. That ideally would be identified prior to the collection of the statistics, but in fact it is usually identified after data set exists.

Distribution is often affected by federal laws, which you are all familiar with, by state laws, which vary from state to state, by policies or ownership. For example, some of the data that resides here at the center are administrative records, which do not actually belong to NCHS, but belong to CMS. And then also by promises made to the person or institution providing the information. This can vary widely and actually local communities may be in a better position than some of the federal agencies to be able to craft language around the purposes and the use to suit their needs better than we frequently are able to do at the federal level.

NCHS, as I am sure all of you know, has two competing mandates. First of all, we are expected to disseminate our data as widely as possible. But we also have to protect the confidentiality of the information. We have various laws under which we operate: the Privacy Act, the Public Health Service Act, and then CIPSEA. And basically, the Public Health Service Act and CIPSEA say that you may only use the data for the purpose that you have explained at the time you collect it. You may not release the data in any way in which an institution or person providing it can be identified. That can be very restrictive.

In addition, as I mentioned earlier, we do sometimes have other data owners involved. For example, vital statistics does not belong to NCHS. It belongs to the states. We use it with their permission and we use it under the constraints of 308(d). All of these laws really affect us in terms of the level of granularity that we can release the data at and also the purposes for which it can be used.

The requirements for much data that is collected is that it can be use for a statistical purpose only and statistical activities of course are pretty broad. I think it would cover most of the uses that we have talked about here.  The data cannot be released in identifiable form. That can be a little
trickier as I will talk about later on.

The distribution limits the parties who may have access to detailed information. We have public use files, the free data that we have heard mentioned that everyone has access to. But then we have more restricted files that may have direct identifiers where that would be available only to our agents, typically our collecting agents or agents who we are engaged in linking record with, or indirect identifiers, which are a much thornier issue because in and of themselves they are not particularly identifying. But when combined together, they can create blocks that relate to a unique identify of an individual. I have given you some examples there of what those blocks might be.

The preliminary protection considerations directing the constraints imposed are unique to a particular — the question you would ask is are the data unique to this file. In other words, if you are collecting information that is not going to be shared by anybody else, is not collected by anyone else, you are in a much safer position. Is the collection process replicable? If it is that also increases risk. Are there external files with comparable data? This is very difficult to determine because the amount of information that is available is growing every day. As technology has changed, it has become far more accessible to many members of the public.

Are there unique cases on key variables and combinations? Has the data been enriched? In other words, has it be linked to something else? Have you added something about the area characteristics? Have you linked it to CMS files?

The resulting constraints on the publicly disseminated data here mean there are no direct identifiers. Some of the things that we engage in are broader coding structures so that we remove our geography. We show only region and state. We remove a lot of coding details by using top and bottom coding. And we also engage in case suppression and variable suppression. These have profound impacts on our users and we are well aware of that.

We know, for example, that most of our users want state-level data. They would much prefer to have county-level data. They would like it for not just the major race ethnic groups, but they would like it for the detailed groups. It is simply not possible for us to release it in that format because of the combination of characteristics due to the richness of the data sets that we are working with.

How does that impact what we can do? For example, the CDC WONDER was mentioned as a source of information that states use and counties use in their data collection activities. CDC WONDER started out with a suppression criteria of three. It had to move to five. Last year we had to move it to ten. This was on the advice of the state advisory board. The reason that we did that was we became aware that the data was more susceptible than we had thought it to be. This was the measure to protect the confidentiality of persons that were in the vital record files.

In general, although we all want a great deal of granularity, the smaller the population, the more easily an individual or an institution is going to be identified as a unique. The greater the number of variables that you have pertaining to a particular individual, the greater the likelihood the combination of them is going to increase identifiability.

Some examples that I thought I would mention is the administrative records. I think all of you are probably aware that a few years back there was a Harvard researcher who identified the records of a Massachusetts governor using some information from voter registration and just health record data.

In vital statistics, we know that if you have a very rare event, you can do a web search and probably come up with a name of the person who was involved.

And then something that actually came to my attention just yesterday is the Genome Project where, again, a Harvard researcher showed that 42 percent of a sample of anonymous participants in a high-profile DNA study could be re-identified using some pretty simple things: zip code, data of birth, and gender, and then the information from a DNA study.

That same researcher, Dr. Sweeney, showed the impact of three simple variables: the effect of having a zip code, of having gender, and of having date of birth. If you look in the lower left box, you will see that having the full five-digit zip code, the gender, and the exact date of birth enables identification of about 87 percent of people. She offers a little application on her website where you can enter your gender and your exact date of birth and your zip code. I came up as a population unique. It is kind of a fun thing to do. It drives home the importance of really thinking through the variables that we allow to exist on our files and to realize that what we may think of as de-identified data really can probably be easily re-identified if someone chooses to do that.

I have listed a few possible strategies that local data collectors might consider. I think these are fairly common sense ones. But I think the key is to really focus on what your requirements are for your statistics. You do not necessarily have to have information about every subject. You probably have some priorities. If you focus on those, that is one way of limiting the vulnerability of the data.

Understanding the key characteristics of data. Are you collecting it for individuals? Are you collecting it for households? Is it an event based? Is it residence based? How old is it? How good is it in terms of quality? Looking at whether or not there might be some circumstances where disclosure would be likely to occur. And considering whether or not such disclosure would be a breach of public trust or of law or policy. If so, then selecting a controlled method to manage that disclosure risk and then going ahead and implementing that and releasing your statistics.

One of the things I wanted to mention that came from the last presentation is that NCHS is actually in the process of trying to develop an equivalent to CHIS query system. They are using the National Health Interview Survey to do that to produce state-level data.

MR. BURKE: Thank you, Dr. Griner. Questions? Time for one or two questions.

DR. GREEN: Let’s adopt a posture in saying we really want to use and reuse data for multiple purposes. And then notice the importance of designating ahead of time what the data are going to be used for in order for them to be used for that purpose. And then look at the rules and regs and their continued expansion and growing elaboration. Help us understand where this goes.

DR. POWELL-GRINER: I am not sure I have the solution to that. What I would say is that I think that with respect to purpose, generally the purposes are statistical in nature. That is, you are trying to understand health conditions. You are not using the information to impact the benefits of the persons that you are studying. In other words, you are not using information to deny them coverage of some type or a benefit of some type. I think you could use general language with respect to purpose.

With respect to the changes in the laws, that is something that we just have to adapt to as those changes are made, again, using the NCHS example. There are files that NCHS has released in the past that we would never release today. All you can do is as change occurs, move forward with your security requirements. Again, who would have thought ten years ago that it would be possible to enter a few bits of information and do a web search and identify an individual. I certainly would not have thought that would have been the case, but it is very easily done today.

I think that as you have seen in the case of CCHIS, one possible solution is to not to think of all data as being equal, but of recognizing that there are going to be some data that can be released safely to everyone and other types of data that will need different levels of control. Just as CCHIS is doing, they have their query system, which is safe for anyone to use. They have two types of researcher data, one that is public use and then one that requires data use agreements. I think if we think more about those models, what are the models that are needed for the different types of data rather than as uniform entity, it would serve us better.

MR. BURKE: I think we are out of time. Thank you very much.

DR. COHEN: We have a ten-minute break and reconvene in your small groups. Let’s make it a five-minute break. Reconvene in your small groups in five minutes. Thank you.

MS. BERNSTEIN: I did manage to put together bio sketches for everybody so that you can read about all the people in the room in a little more detail. Janine has those and she will hand them out to us as we go to our next room.

PARTICIPANT: Are the slides up on the web because we only got half of them.

MS. BERNSTEIN: We are also remaking e-slides, which I apologize. We inadvertently copied them not double sided when they needed to be double sided. We only have half of them. But we will get you another set of those on paper. All the slides are available on the web that we have electronically.

(Break)

DR. COHEN: From the yellow group and then the blue group and ending with the red group. I am turning it over to Lee who is going to moderate this session.

Agenda Item: Report of Small Group Discussion 2

DR. CORNELIUS: I thought my job was to summarize the whole day. Could we have the yellow group up first? We are going to be quick and expeditious because I actually have been thinking all day long about you.

DR. BREEN: The last time if you recall the yellow group organized our thinking around obesity. We wanted to use an example. We selected obesity again this time. We focused on that first question, which asked about standardization. We agreed that standardized definitions and methods are a good thing, but we wanted to ensure that — we also thought flexibility is a good thing. We spoke about two examples in particular. The OMB recommendations are standards for race and ethnicity. But we talked about those as a floor, not a ceiling. In other words, you are welcome to collect additional information though they have some procedures that they have learned over the years work best and they are recommending those procedures. In fact, if you get money from the federal government, you need to use those procedures to collect your data.

And then BMI is another one, which has a standardized approach. But more recent research has shown that there are specific populations for which the BMI standards do not work so well.

Standardization yes, but flexibility is always important particularly the closer you get to the local level.

Then we had an interesting conversation, which we had to recognize after a while that we were talking about two different types of data. One is electronic health records, which is a new and emerging kind of data. Another is survey data, which is quite a well-established kind of data. We started discussing the whole issue of having the data cycle and that the survey data has gone through many data cycles. At this point, we have most of the questions on the shelf and cognitively tested by the National Center for Health Statistics and other places whereas with electronic health records that is still an emerging kind of data. It is probably going to need to go through a number of rounds before that will be — will have off the shelf elements because that is not quite ready.

And then there was a side conversation, but it was so interesting and amusing we had to share it. That is that the information that people get particularly for epidemiological of public health purposes can be kind of strange to clinicians once they realize what is happening. And the example that came up was how do you collect smoking data. Bruce said that is pretty standard. You ask people whether they have smoked 100 cigarettes in their life. And if not, then they are nonsmokers and you don’t ask them the rest of the smoking questions. Paul said oh my God. You think that people who smoked 100 cigarettes in their lives are smokers. We could all sort of see that calculation of what kind of clinical relevance would 100 cigarettes in your lifetime have. But that was interesting too.

I think it is important because of the back and forth that it shows we are going to need between using electronic health records and survey data in the same gulp. We are going to really need to think about how we are going to explain and discuss these data so that they are meaningful in both realms to both the population health community and the clinical community and of course we want those two groups to be communicating well in important ways.

Then we were fortunate to have Barbara in our group. We really picked her brain about the Rochester case. The process there seemed to be to start with a meaningful incident, which then you would get some information on and that is where the data collection came in. Whatever the results were would allow you to identify the community priorities and then you would seek more data and possibly develop interventions or whatever the next step would be. Sometimes it can be as simple as having a charrette or a group like this where you would have tables with different topics and just let people go to the table where they have the most interest. She said in the one case, everybody, practically 90 percent of the people in the room, just walked over to the table, which was about drugs and alcohol. There are various ways to do this.

And then we came up with two emerging recommendations. We have emerging because we are not really supposed to be giving recommendations yet, but they just seemed so generic out of what we had talked about. One is that the federal government can play a crucial role in promoting training around data collection, documentation, standardization, et cetera because one of the other things we learned about Rochester was that people who are collecting the data there did not know off the shelf questions. They were not thinking along those lines. They did not know any of the planning tools that are available that CDC, NCI, and other government entities put out. This is an opportunity to provide information to a wide range of the public in order to be able to use those things because that is what they were created for.

And then secondly that collecting qualitative data is really important, but we have no standards at all for that. That is a wide-open area for the federal government to participate in. We have a recommendation for that as well.

Any other things that I have missed from group two?

DR. COHEN: Thank you. Blue.

DR. EDWARDS: Good afternoon. We also had a very engaged group. Lots of discussion. I do not even think we put a dent in the summary document. But for the sake of being short and succinct, we will go through the bulleted areas that we thought were important to highlight.

One is that my namesake Denise Love made a quotable or tweetable statement and that was public health needs to get sexy with its data. She qualified that by saying but not too sexy. And what that really means is that we need to release the information, but with limits and with real cautionary measures around who we release what to. That is what you would say to your kids. Our goal is to figure out ways in which we can make the data attractive and appealing. People get what they need when they need it, but there are some limits and constraints on that.

The other was that technology has changed significantly. And our capacity to capture information that can be used in repurposed has changed. In the rules, in the stewardship work and the frameworks were developed, we did not have the capacity to collect and retain information the way we do now. It actually kind of made sense that you would say you collected it for this purpose. You can only use it for that purpose. But the potential of being able to repurpose information is so great. Is the greater good more important than the risk that we put limitations on? We think we need to think about how technology has changed our capacity, and then what are the policies that need to change as a consequence of that. I do not think our scribe can keep up with our thinking and talking.

The stewardship framework really needs to be reframed and rethought about in terms of how we empower communities across different spectrums to be participatory in what information is captured and used for what purpose. We started the conversation around what are communities doing with data and what kind of parameters and standards need to be placed. It led us into a conversation about we are really on the granular what communities need. One of the things that communities need is to be a participatory in a process of establishing the standards and what is happening. My group can jump in if I am not fully representing all the comments.

I thought this was really important. I cannot remember whom in the group to attribute it to because everybody was so thoughtful and insightful. The community is a learning system and is a place where we can establish processes. Community as a learning system is a process in that it is real important for organizations like this government and otherwise to help communities assert the ownership over the processes and the priorities. We had a long discussion about did we want to use ownership because there is a big debate over who owns the data. We are not talking about who owns the data. We are really talking about the leadership that is required to facilitate who owns what happens to their data. Who is responsible and accountable for what happens to that data in the information?

I will go to number six before number five. That really led us to talk about the fact that data can be used for good and data can be used for evil. The good should outweigh the evil. But the only way for that data to be released as Denise said in the first statement is for people to understand the value. We have done a great job of educating ourselves about the value and the importance, but does the person down on the street in the road, in the community, in the trenches see the value when they are confronted with discrimination every day, when they are looking at how information has been used adversely. How do we educate consumers, engage communities, and understanding the value, the importance of information? That is going to be crucial to having community participatory research.

And then lastly with all that said, we are still going to need standards. We cannot get away from having basic standards, but standards that are balanced and consistent across different domains whether it is geography or demography or some way in which we standardize. There is not one shoe that fits all. But there might be ten shoes. How do we make sure the right people get the right show and that the standards do not confine us, but they help us open up and release information as it is needed.

To my group, anything I missed, misrepresented, did not include that is important.

Issues for further consideration or study. Basic measures. We talked about. Is there just some core, basic measures that should be publicly available in any form whether identified or de-identified that everyone needs and should have access to without any limitation? We did not have an answer to that question, but we thought it was an important question to raise. That we think there needs to be further study about infrastructure and how we would support our new model for openness of data given how technology has transformed our thinking. Does that mean there are new agencies that need to help us monitor? How do we create infrastructure that enables us to do that?

Proprietary interest in replacing public health. Do you want to speak to that quickly, Marjorie, because I missed that one?

MS. GREENBERG: Is Denise here? Denise raised the point, she described a situation where an all payers claim database was established and that it was locked up and they threw away the key. The insurance company said we would just do it ourselves on our own rules. It was part of the whole discussion about if we cannot get agreement about using data for the public good, just the proprietary interest will follow their needs and public good would go by the way side.

DR. EDWARDS: We talked about the need for the collaboration between the two so that the proprietary interest do not outweigh or circumvent the public good.

And then we liked how — I do not remember which group it was that used use cases. We thought in order for this to be meaningful to communities, we really do need use cases. It is such a big issue going to a community and saying, what do you need data for or what kind of data do you need. They are going to look at you like what. If you ask them the question about, we noticed that there are challenges in your community to walking. Why can’t people reduce their blood pressure by walking? They can say there are no sidewalks. There is this and that. Putting it in a framework that then helps you get the data that then the other folks can use. We thought use casing was really important if we are going to engage communities. Bring it to a level that makes sense to them.

I see my group shaking their heads so I think I nailed it. I am being punished because I missed the earlier session.

MS. HOFFMAN: We wanted to distill ours to just three or four points. I pretty much randomly selected three or four points from among those we discussed for an hour. One point that we made is that it is really — if we are trying to benefit the community, it is the community that should set the priorities and that we should have deliberatively democracy. We should not superimpose priorities and initiatives on the community. We really need to hear from the community.

Another point that we discussed and I think this is a consistent with some of the other points that we have heard is that we need to determine what the relevant unit is. In other words, is it meaningful to say that the community is the state or is it regional or is it local? Is it a city? And those can be very different especially in some states. We need to define the community.

We also talked about the fact that vendors are to a large extent driving the process these days. We do not have good interoperability and we do not have data harmonization because it is not in the vendor’s interest. They want to keep their trade secrets. They want to capture customers so the customers cannot go to another vendor. They want to market themselves as unique and so on. We may need to have some disincentives for bad behavior and some incentives for the types of features and capabilities that we want keeping in mind that everybody has a different agenda and those agendas can very much be in conflict.

Finally, we had a very animated discussion about the extent to which data should be publicly available. Again, this is consistent with some of the other points that were made. We brought up the example of a study, which somehow was published in a peer review journal that found a causal link between abortion and later subsequent mental health problems including suicide. And that study was debunked a few years later. They relooked at the data. The study had been terribly designed. But it was not debunked before quite a number of states passed actual legislation that is still in place that requires physicians to warn women who are seeking abortion that they are likely or that they may suffer very significant mental health problems. There is a danger to data. If it is publicly available, it might be even more dangerous because people can post things on blogs and on websites and so on that look scientifically valid.

We had a discussion about whether we need some gate keeping. Do we need consequences for misuse? Lacy told us that within the Beacon system actually those safeguards are in place. And what kind of governance do we need? But of course, there was robust disagreement in the group because we had some advocates for publicly available data no matter what and for the political process are for people to do damage control rather than restricting access.

I think I will sit down now. That is our contribution.

DR. COHEN: Rather than have Lee provide the summary for today so much has happened. Are you going to make a few comments?

DR. CORNELIOUS: I am just going to say quickly where my brain was and what you will get tomorrow morning. Actually, you took the charge to both think about the key points out of both plenaries and the common points that came across all three groups. What I will produce tonight for our consumption first thing in the morning is an abstract so that when I do the presentation, we can start the morning off fresh.

DR. FRANCIS: And hopefully that will set the work plan for tomorrow because I will not try to set a work plan. I will simply leave it up to Lee.

DR. COHEN: Thank you all. It has been a wonderful day. I look forward to more tomorrow.

(Whereupon, at 5:40 p.m., the meeting was adjourned.)