[This Transcript is Unedited]
Department of Health and Human Services
Working Group on Data Access and Use
September 17, 2013
National Center for Health Statistics
Auditorium A and B
3311 Toledo Road
Hyattsville, MD 20782
P R O C E E D I N G S (1:05 p.m.)
Agenda Item: Review Agenda and Introductions
DR. CARR: I will call this meeting to order of the Work Group on HHS Data Access and Use. I am Justine Carr, Steward Healthcare, chair of the Work Group.
MS. BRADLEY: Lilly Bradley, staff to the committee.
DR. COHEN: Bruce Cohen, Massachusetts Department of Public Health, member of the Data Work Group. No conflict.
DR. FRANCIS: Leslie Francis, University of Utah, member of NCVHS and member of the Data Work Group. No conflicts.
MS. KLOSS: Linda Kloss, HIM consultant, member of the full committee. No conflicts.
MS. GOSS: Alexandra Goss, program director for the Pennsylvania E-Health Partnership Authority. I am a member of the full committee and I have no conflicts.
DR. CROWLEY: Kenyon Crowley, University of Maryland, Center for Health Information Decision Systems and a member of the Working Group on Data Access and Use.
DR. CHANDERRAJ: Raj Chanderraj, practicing cardiologist. Member of NCVHS and no conflict.
DR. MAYS: Vickie Mays, University of California, Los Angeles. I am a member of the full committee, Population and Privacy Subcommittees. No conflict.
DR. VAUGHAN: Leah Vaughan, director of The Health Policy Group. Member of the Working Group and no conflicts.
MR. SOONTHORNSIMA: Ob Soonthornsima, Blue Cross and Blue Shield of Louisiana. Member of the full committee and no conflicts.
DR. SUAREZ: I am Walter Suarez with Kaiser Permanente. Member of the full committee and member of the Working Group on Data Access and Use and no conflicts.
DR. KAUSHAL: Mo Kaushal, member of the Working Group.
MS. GREENBERG: I am Marjorie Greenberg, National Center for Health Statistics, CDC, and executive secretary of the Committee.
MS. JEAN PAUL: I am Tammara Jean Paul, member of the Population Health Subcommittee.
DR. CARR: I would like to invite everybody to come to the table. We have lots of room, and it’s conducive to a lot of dialogue.
(Introductions around the room.)
DR. CARR: Welcome everyone. As you can see, we have an agenda. Let me just take you through what the plans are. I think we’re going to work today to continue to refine the plan for the Solve-a-thon. To do that, a couple of things, I’ll kind of recap where we are. I’ll ask Leslie to give us an update on the privacy policies.
DR. FRANCIS: It is in the works. Actually, I can just tell you right now figuring out what they say and what they don’t say is actually proving to be a pretty complicated process, which is informative in its own right. I’ll have a full report to send around in about a week.
DR. CARR: So we are going to see where we are with the Solve-a-thon. Michele Plorde from King County is joining us at 2:00. This is someone interested in partnering with us for the Solve-a-thon. After we have had a chance, we’ll take a break, Lilly has put together an innovation map. We want to come away today with some firm plans and next steps on how to proceed. I believe Josh is delayed.
To recap where we were when we last met, we talked about taking our charge of innovative uses of HHS data and came up with the idea that if we could get a model where we have HHS data plus some other innovative data such as social media, Google, Twitter, Facebook, whatever, that with that model we would look for a community that had an issue they wanted to address, had leadership and resources that would be capable of partnering with us. As was pointed out, although we’re trying to be very innovative, we sent an email to a handful of people to see if they were interested. We didn’t Tweet, and we didn’t put it on Facebook or anything else. We heard from a group out in Seattle, as I said Michele Plorde came today. We’ll hear from her later today.
What I think we want to talk about before we hear from Michele is to kind of nail down a bit of more details on what a Solve-a-thon would look like. What would we need, who would we need, who are the customers, what are the products, and how does that tie in with the work that we have already done as part of the full committee?
MS. BRADLEY: If we have time, we could provide an update on some of the open data activities beyond what was discussed yesterday, which is less open data and more data.
DR. CARR: I just want to make sure we’re on the same page before we hear from the person from King’s County, and then I do definitely, because we want to be a part of this. So as I said, let’s start with our goals. Let me ask you, if we are to do the Solve-a-thon, what are some of the goals we want to achieve in this?
DR. FRANCIS: Can I ask about the Solve-a-thon design? Some of the “–thon” type things, contests and so on, pose a problem and invite lots and lots of people to solve it. Their aim is to get solutions from people who might not normally participate. I take it that’s not what this is about. This is about working with the community to figure out a solution to some problem they have using data and then seeing how they did it.
DR. CARR: Our number one customer is HHS. Our partner is a community. As we take one problem and look at all the aspects of the challenges and the benefits potentially of enhancing HHS data with other data, we want to document what are the issues, what are the opportunities, what are the challenges, where do we need after the fact to look at it? In addition, by working with the community we’re hoping that together we learn something that adds value for the community. I would say we can’t guarantee that at this juncture because this is something new and different.
MR. CROWLEY: If you think about this open innovation concept, there’s a whole continuum from two people sitting in an office to solve it to the public at large solving it. I think what we’re looking for is somewhere in the middle where you have these HHS experts and community experts and community profiles that can come together and work on this problem together. Somewhere along that continuum there’s probably the right mix of expertise and people involved. Once the design is fleshed out that we want to pursue, there are various design elements you can use to decide how to vet different ideas in the community, who’s vetting them, whether it’s the public vetting. There are a number of elements we can discuss and choose from.
DR. CARR: So this is the kind of things we want to think about. I think we have access to anybody that we need at HHS to help us with this. We want a community person who understands the issue. I think when we speak with Michele today that will be that. Let’s think through more what you were saying in terms of a design expert. Say more about that. Who else do we need besides the community and HHS?
MR. CROWLEY: Well, depending on the problem, it takes any number of people who might be a stakeholder to that problem. If we talk about the context of King County, which I believe has a held communication problem where they’re trying to reach out to certain sub-populations within their community to improve immunization rates–
DR. CARR: I would just briefly say their EMS group has been focused on—they did an initiative on faster activity to cardiac arrests patients by EMS to reduce mortality. They now are looking more broadly at who’s not using 911 and why, and in particular they’re interested in the elderly and if they use 911, also individuals for whom English is not their first language and potential other at-risk groups who may or may not be doing it.
MR. CROWLEY: So within that community we can envision a number of people who may have value to contribute to that type of problem, whether it’s the gerontologists who are understanding the perceptions of the older adults in that community, people who may be experts at messaging for other ethnic groups might be appropriate. As we’re looking to leverage different data sources that may or may not be available in that context, those types of experts will prove helpful.
Some of those are within the community, and some of those may be external. The public health leadership and administration who would be rolling out these types of programs in the community would be important stakeholders. EMS experts, as they’re one of the key pieces, would be important. Then also given that this is building on some existing efforts, I would say users of the existing efforts to get some feedback from them, both the users being the community users as well as those who are implementing the program. We could go on, but I would say those would be some of the important parties.
DR. CHANDERRAJ: I would add volunteer clinics are traditionally excluded from these groups because they don’t use meaningful use or EHR systems, and they form a part of community healthcare. They’re delivering healthcare, and people in volunteer groups should be included.
DR. CARR: I know Josh has some ideas on this, too, but we need expertise in what social media data and what we might look for—what would marry up, once you have the problem, what HHS data and what kind of social media data. I think that’s part of the innovative side of it. Again, showing that we could say one thing, we may not say everything, we may not solve everything, but I think that it would be—the example that we talked about somewhere along the line is you have CDC flu data and Google inquiries about stomach ache, headache, fever, nausea, whatever, and now suddenly you have two different pieces of information about flu that enhances what you would have known with just the HHS data alone.
DR. COHEN: So if the focus—I think who needs to be at the table depends on what the specific topic is. I think that’s essentially what people are saying. If the topic is 911 data, you’ll need—you might want to talk to the phone company or the public health departments who control 911 systems, the EMS services and transport services who are in charge of emergency response. There are a variety of local players.
I’m trying to reach into my mind to think about what federal data might be relevant to this, and the answer is not very much. I’m having difficulty thinking how the Feds affect this particular local problem at all. I can’t actually think of a federal data source that touches this particular issue. If that’s the case, is this what we want to be working on, or do we want something where there’s a possibility that there would be federal data?
DR. CARR: So I think that is the decision tree we have to walk through. We had said the fundamentals were social media data plus HHS data, those two were fundamental to be able to meet the charge of better use of HHS data.
DR. ROSENTHAL: I think one way to think about it is depending on how you carve up the population, I think there is significant HHS data on a number of fronts. The way we were initially thinking about it was problem, intervention, solution. If you think about it, starting with CMS or Medicaid, you have a variety of chronic conditions looking at STAR METRICS as well as hospital utilization data and provider data, patterns of car, from readmission rates, for benchmark conditions that CMS has just released, AMI, CHF, pneumonia, et cetera, they have a kind of post-discharge at line item level.
The idea basically is you have a community that might want to look at a health topic, and part of our task is to figure out where is there HHS data that can speak to it. If we define it around CHF, around diabetes or something along those lines, we can basically look at an HRR or a zip code, look at the sum total of the populations, starting with EMMA, a traditional fee-for-service data, supply data, and utilization data, and then be able to say they did this intervention, was there an impact, using a CMS benchmark for impact? Did the population go up? Did the population go down? Did you see any change in the patterns of the care around that? That’s the traditional HHS data you could pull up.
Then obviously the social media data, the idea behind that is not only is part of the charter, but using that a) to define the problem, b) as part of the intervention itself, and c) for part of the outcomes assessment. Given that kind of framework where you have problem, a pre-existing data set for a subset of the population, and then outcome, you can basically look through those 58 conditions, part C, part D, and be able to use CMS benchmarking for outcomes out of the gate.
DR. CARR: A key part of this is what was new and innovative. What was new and innovative that we had set out was that we would use social media data as part of that. Alex?
MS. GOSS: The work I have been participating with the Office of the National Coordinator seems to me they need some baseline information about social media uses, generational thoughts about it, how they tap into it, that would have been developed to support the consumer engagement initiatives they have going on. So as far as a starting point, why not pilfer the good research that they’ve done and then fuse that to the other data sources?
DR. FRANCIS: So we also need to pick a problem that makes sense in terms of social media data, and I’m not so clear that trying to figure out why impoverished populations do or do not use an ER yields itself to that—
DR. CARR: It’s not an ER, it is EMS.
DR. FRANCIS: Well, for one thing I doubt that there’s going to be information about your decision to use EMS on Facebook or Twitter or Google. You might Google search, but if you’re suffering a lot of chest pain, I don’t see it. Maybe somebody else would.
DR. ROSENTHAL: I think that is part of the exact thing we should look at. HHS has done a decent amount. They have active Twitter scraping campaigns on a number of different fronts using vendors right now as we speak. Some of the folks that we’ve spoken with—actually, there is a surprising amount of data, and that’s, as I understand it, one of the project’s charges, to have HHS wrestle with this a little bit and see what is their data that speaks to certain conditions and interventions as well as where are there gaps that might be filled and just general kinds of tools and techniques. Where is there statistical validity and where is there not?
Just by way of example, HHS has a number of initiatives right now using—so scraping, semantic mining of Twitter around that. You can’t go into it as a standard research project and say we’re going to pre-define the hypothesis and see what’s in there. You’re going to have kind of an evolving hypothesis, natural generation. There’s a decent amount of data in there, whether it speaks to a particular condition, personally I’m agnostic on whatever the intervention is.
I think if you start with those parameters around Medicare and Medicaid, you have a pre-defined taxonomy for outcomes, both of the payer population all the way down to the provider population. The social media stuff you can scrap, that’s going to be the semantic exercise itself, to say, what are the loci that you can actually pull out of that? In a nutshell, there’s pretty good data, and that’s part of the project.
DR. FRANCIS: What I just wanted us to be aware of is there may be a whole heck of a lot of selection bias in that data.
DR. CARR: We will do an exercise and then we will analyze where is there value, what are the challenges, what are the vulnerabilities, all of that. We shouldn’t be constrained by, is this great. It can be just as much about is this possible, is it feasible, what does it take to do it, who do you need to do it, and is this an approach that could be further expanded.
MR. KAUSHAL: So assuming we agree on the strategy, what are the tactics here? Who’s going actually parse through the data? How are we going to organize implementation of this?
DR. CARR: That’s part of the discussion today. You remember in June we thought this would be a great idea, but we didn’t get into the details. I think in many ways to do this we need a sophisticated partner. I think this group we’re going to speak with at 2:00 is sophisticated, but we’re on a group learning curve here. I am not the person who has the final product in mind. I think it’s a work in progress. I think we need to land the plane on something that could be an achievable work product, and what would that look like. Is it something you do in an afternoon, a day, a week, an ongoing process? For our purposes, we’d like to have a small project that we then do a lot of analysis on. How did we get it? Was it valid? What else would we bookend? As Leslie said, we can’t even figure out what the privacy policy is for Google, Facebook, and Twitter. It is oblique to some of us.
MS. BRADLEY: Josh, we were going to have Ted Smith from Louisville, Kentucky join us, but he ended up having a conflict. I know that you got to speak with him about some opportunities that might be in Kentucky, if you would be willing to speak to that.
DR. ROSENTHAL: Sure, so Ted Smith is an interesting individual. He used to be an OMC employee and now runs Louisville Metro Innovation and Health and is particularly concerned with economic issues. They have a number of different interventions, and he’s also kind of PhD statistically-trained and has a startup background so knows how to work with the art of the possible and ambiguity, and he’s basically open to participating as well as Louisville in any way we need him to. He has some significant resources, including using the same social media scraping, projects that HHS does currently as well as a number of statistical interventions they have in mind and are actively running.
One would be—I forget the name of it, but it’s an intervention that won some Robert Wood Johnson support. Basically they take fresh veggies out into the inner city. They’ve been able to collect data as a survey-based intervention, but obviously that relates to a number of provider, cost, quality utilization as well as clinical conditions, and the nice thing about Louisville is it’s a county, it’s a contract, and it’s a metro area. When you think of data density and ability to link up different types of data, it fits very nicely. He’d be willing to dedicate resources to that, and it’s a very sophisticated partner.
The interventions on one hand sort of don’t matter. It’s figuring out whether there’s data to support whatever the intervention is that you want to look at, and then going into it open minded and saying let’s say if we are looking at this Twitter scraping that HHS does and Louisville Metro is doing, where do you see correlation and potential causation? You might see it with HCAPS, hospital customer satisfaction metrics. You definitely see it with a number of the Medicare metrics, not only satisfaction, but et cetera.
He’s more than willing. He’s sophisticated. It’s a good partnership. They’re doing a number of other things. He was honored out here by the White House a month or so ago. As far as intervention, the initial one we were looking at was fresh food delivery access, which is an already an intervention up and going. You could sort of handle it two different ways. We could figure out what a problem is and then try to create an intervention, which has a longer time horizon. If we start doing that from scratch, it’s a bigger lift. It’s tougher to get that up and going. Or you could start with something that’s pre-existing, and that has the advantage of forcing us to work around the art of the possible.
DR. CARR: Pre-existing meaning scraped data?
DR. ROSENTHAL: Meaning and intervention that is ongoing. They have this thesis that particularly within a certain pattern of wage striation, the most impoverished, the most health at-risk populations, that food access will have a positive impact on Triple Aim features. They measure that in classic MPH style, taking survey, seeing what’s going on, et cetera, but there is extant HHS data as well as social media data, which can be brought to bear on that right now. That would be something that has a pre-defined horizon. You can see before, you can see after, and there’s historic and current HHS data that could speak to that pretty clearly.
DR. CARR: So again framing how we would proceed, if the finish line is to be able to deliver to HHS some social media data, some HHS data, and it was able to be put together and “x” was the outcome or finding, I think that is one state. The second work that I think is the work that will go on after that is where there are constraints on the data from a data integrity site. How do you account for data integrity, for security, for privacy, for all of that?
I think Josh makes a good point that if we start with two sets of data that go together, some HHS with some social media data that is already out there, and we work potentially with Ted or someone like that to help understand that, we can then pull out what are the issues that this type of mash-up presents? What do we learn? What do we have to learn more about?
DR. SUAREZ: I just want to go back to the original Solve-a-thon concept. I was trying to understand how would this work. Most other “-thons”, vote-a-thons and other things, involve teams that develop solutions. In this case, we were going to identify one community and then bring that community and bring other people to work with that community to identify a series of areas that they can explain, or the group can come up with some ideas on how to address–
DR. CARR: So one thought was go to a community, see a problem they solve, think to ourselves—see, when Bruce and I took a cab home after the meeting in June, it was so simple to us. We thought about there was an issue in Brookline where we live, about adolescents and at risk behaviors, drinking or whatever it was. We began to think, wow, Facebook and all these different things you could get, independent, for the moment, of the privacy issues, and so in that moment, it all seemed so simple. For various reasons, we’re not going that route.
Then, building on that, we thought there must be another community with an obvious, easy problem. What we’re finding is that’s not exactly true. It’s not as easy– finding it and engaging people and getting the expertise might be a bigger scope project than what we had in mind. The alternative would be there is some HHS data plus social media data already being used together, and why don’t we just jump on that and see what are the lessons learned? We don’t want to spend a lot of time—we want to get some answers. We may go back and revisit it, go back to a community, but I think we want to begin to start that white paper or whatever saying here are all these kinds of issues that you have to think about when you do this idea that also seems so easy with flu and Google. We’ve not landed on our plane yet.
DR. MAYS: First I just want to comment on something Bruce said about what data sets. There are a bunch of data sets that the Feds maintain such as the National Emergency Department Inventory, National Emergency Department Sample, I could go on and on, that may be things that would be helpful to have this data with. In terms of adding some other outcomes, it might be that this should be a question of can or should this data be linked to the HHS data? It might be the question of whether to evaluate whether doing that would enhance our science or enhance community knowledge.
It may be to push the envelope a little further to see if once we discover this whether it would be good for HHS to actually promote this. It doesn’t have to be the linker, it could be that the information is there for people who are using the data to know that you can get this if you also do social media data.
MR. KAUSHAL: What does success look like for us? If x, y, and z happened, then what do we do with it? I have full belief that the mashing of this data will result in value creation. It’s already happening, so what’s our role and how do we perpetuate it?
DR. ROSENTHAL: I was going to say, Walter in answer to the question, originally there are these hack-a-thons, which are pretty standard, and they’re in the tech world, and now HHS is sponsoring them. That’s great. At the end of the day, those largely fail. They aren’t really being utilized, creating value, and stuff that people put out there. It definitely is largely not connected to the communities. It’s just at the beginning of that curve.
As we were discussing saying could communities use this new data in any sort of meaningful way we came up with an idea of a solve-a-thon, and then went out and explored that a little bit. The difficulty with that is the communities aren’t sophisticated and aren’t even there yet. Having this group in HHS on the forefront of that, giving some guidance and coming out with some collateral or a tool kit, in order to use that what would a team look like, what are the sources, what are some techniques, what are some pitfalls, what are the privacy issues, et cetera, is the output we were walking around.
Because they’re so far back from where you’d imagined that they would be, this sort of morphed into a proof of concept. Before we do a solve-a-thon and break it open to any and all competition, they don’t know enough, and we don’t know enough about it to be able to meaningfully create that framework. This turned into first generation little prototype proof of concept, let’s find a community and basically say can we use HHS data, can we use social media data, can we use it to solve a problem, and where are the pitfalls and the learning, and then use that to build a toolkit that we could actually, if we so chose, launch a solve-a-thon, which is a really nice turn.
Instead of coding something up, using the data to solve something and commendations to HHS in that group, there’s a reason why this hasn’t been done so far. There’s a reason why communities aren’t doing it. It’s good stuff.
DR. CARR: So let’s pretend that we were going to take Google inquiries about symptoms of flu, and we were going to take CDC data on flu– that we can all get our head around, I think– and we’d be doing it for the first time. What are the lessons that we could learn? If we could use that very concrete example, I see–
DR. COHEN: I will try to address– for me, I need to start with the process, what would the solve-a-thon look like? One possible model would be we select a community and an issue, either a narrow issue or a broad issue, and then we spend some period of time identifying who the experts of the community and government and other levels are who know those potential data sources that might be related to that particular issue, have them think about how their data can address the issue over a period of a month or so, bring them all together either virtually or in person, and have a conversation about how to access the data that they’re most familiar with, how it can be used, either independently or in conjunction with other data, whether it’s linkable at the individual level, what is the level of geography, how are data are protected, and have a conversation among all the data holders or those familiar with the data on how all this data fits together in a one or two day virtual meeting.
We need a pro-dromal phase where we’re gathering information around the data, and then we all need to be in the same space to talk about how the data fits together, and then out of that there will emerge a report about how to put the data from the different sources together, how that fits, where the gaps are, and where we should be taking this.
MR. SOONTHORNSIMA: I’ve been tentatively listening to everybody’s comment, and I think I’ve finally honed in on the purpose– I thought maybe instead of starting with the data of community, maybe we turn that upside down a little bit. I ask myself what would motivate me to act as a consumer, and in health care it’s typically a crisis of sort, perhaps starting with a consumer experience.
It doesn’t matter in what community, and let’s start with the indigent community and say, I have a crisis, what is my current experience. I can’t access care. First of all, I don’t know where to go, what my symptoms are– you don’t think that way, I know I’m sick. Let’s follow that experience and say, what would the desired experience be like for that individual. That individual is a part of a community. Does that make sense? Start with what we believe the desired experience would be for a health care episode, health care encounter, not in a clinical sense, and then start framing up for that individual what sort of information that person would want. She wouldn’t know, but what would be the right information, just enough information for her to act on and change that experience. It could be community-based information. It could be HHS. It could be social media, but it all starts with a person. Does that make sense?
DR. CARR: There are some different approaches.
MR. SOONTHORNSIMA: Do you start with a problem? I think that’s what you guys were talking about a little bit, what problem were we trying to solve. Let’s offer than individual and then you frame it up–
DR. CARR: I think one of the things we have to come to closure on is are we trying to solve a community problem or are we currently trying to solve an HHS problem? Who’s the customer, community or HHS? If you have to choose one, I think we were choosing HHS, but let’s continue around the room.
DR. MAYS: I put it down and I put it back up, because I got confused around that very question. Then I am back onto HHS. The question is we dont’ want to try to have a real solve-a-thon at this point of actually saying here’s the problem for HHS around flu and asking people to share or illustrate what they’re doing to solve it. As an example, the guy who does my tech stuff has a contract to actually– he’s actually working on flu, and it is in terms of an act. They have a very specific population that he has to work on to get it out. Depending on what the problem is, we may actually want to see if we can have a little contest of type, or you show yours and we’ll have the community come in and bless them in some kind of way, to get us up to speed on whatever the problem is. I think we should have a real one, a real solve-a-thon.
DR. CARR: I’ll just put in my two cents on this. So in some ways, another way to think about that would be that we convene a group of people who are already working on things and spend time saying how did you work on this, what was the challenge of that, and then get the– if the outcome that we’re looking for is to advise HHS, that here are the pitfalls, here is the vulnerability, here are the strengths, here are the weaknesses, would we be best to bring a group together and asking them to show us a mini datapalooza but focused on show me where you used HHS plus social media data, and let’s talk about that. We could give a prize in the end.
MR. KAUSHAL: I think we’re actually solving the community’s problem, and I think HHS is facilitating that along with other data sources, and we’ll learn a lot from the process as we’re looking and then feed that back into HHS to disseminate it into other communities. It’s a semantic distinction, but I’d like to the community as important here.
MS. BRADLEY: I tried to rename it “Novel Data Partners” as we moved away from solving a problem. I think it’s a misnomer.
MR. CROWLEY: I think in some instances, we just need to start solving. We’ve been talking about what is the solve-a-thon, some of the different elements, and I think we all have general agreement that there is the expertise and the resources available to put an answer to some of these problems, just to stand up this first solve-a-thon topic, and within that infrastructure, we can add additional problems to that problem list, whether it’s the list of the patient-centered care, and what are those data sets, but once we put in place this whole infrastructure, then you could have the virtual teams come together start vetting the problem, providing suggestions on the solution, and then through that process refine how the solve-a-thon works, in traditional innovation, start-up mentality, just start doing it.
DR. CARR: I just want to give you a chance to jump in if you want.
MR. CROWLEY: I was going to add one other thing on a separate topic of what we could do on the social media front in terms of the overall problem. In the overall problem I think what we’re trying to solve, too is what is reusable across the nation that HHS could provide that would be useful in many communities, but we come up with the solution in one community. One potential idea around that is within social media you have the ability to identify areas of trust and influence within these networks. If you’re trying to provide messages that will diffuse and be used within communities, finding those methodologies to define trust and influence, whether it’s in King County or these others, could be used to improve the effectiveness of health communication messages and social outreach campaigns.
DR. ROSENTHAL: If we want to– this is one of the conversations we had on one of the calls way back when, searing around communities and some of the work that related. If we want to open it up and have a real solve-a-thon, I’d strongly suggest we’d partner with HDC on that. On the social media side, there’s kind of peer play social media stuff, identify areas and influences, all the stuff you find in DTC. The particular nexus of connection, which I find very fascinating around tying that to HHS data and seeing can you find those as leading indicators of clinical outcomes?
If I find a semantic topic or a nexus, does it actually predict readmissions, does it predict HGAP scores, things like that is what I have in mind in terms of hard connections. There is definitely HHS getting savvy with social media, for sure, but what’s the nexus of the connection. Do those things serve as leading indicators? You see unwarranted variation, you see strong correlation, maybe even causation. This stuff has been coming out in academic journals lately, and there’s a bunch of sample bias and limitations, et cetera. In my mind, the question is, can you actually do it in the real world, and what are the pitfalls? One of the outputs for HHS as a customer is you’re going to define pockets of need for your data, and maybe it’s provider cost data instead of just hospital cost data. You’ll see stuff like this come up out of the woodwork.
DR. CARR: So I guess now we get a little more structure around this. Actually, I think what we’ve heard is several different models of how we could do this. I want to get a sense of what would be the time frame within which we would harvest the information that we would then begin to further analyze. What I’ve heard is a couple of different ideas, all very exciting, some with a short time frame and some with a long time frame. I think it’s up to the workgroup, do we want to look at this over a period of months or are we looking at this as a kind of short time frame?
MR. CROWLEY: 90 days.
DR. CARR: That is fine. It’s not a year, and it’s not a week.
MR. CROWLEY: I think one of the first decisions we’ll need to make as a group is what is going to be the infrastructure that’s going to support the solving, the interaction, whether that’s a community wiki or one of these other private market solutions from IDEA or one of these other places? IDEA is kind of expensive.
DR. ROSENTHAL: If it is an event, the first fork in the road is if you’re having teams and in communities, it’s kind of partnership and advertising and facility. That’s a different beast. I’m agnostic either way you want to go versus a proof of concept, which is sketch out a little prototype with the output being build a tool kit and get a sense of what it would look like to put on a solve-a-thon for other communities. Is it something where you work internally with the community and you sketch out some stuff, or is it bringing people in and actually doing something? Those are two very different resource dedications.
DR. CARR: I am not sure what our available resources are. I’m tending toward the small proof of concept. I’m wondering whether we would align with the work that Populations is doing, speaking with communities, et cetera, so that if we had something proof of concept that was appealing, it could align with some of the work that populations is doing.
MR. CROWLEY: Another is this HHS Innovates program, which they use to get ideas from HHS employees and refine. I don’t know if that’s an existing licensed platform that we might be able to repurpose for this, meaning the toolset that’s used to support that. I’m not too familiar with it, but it sounds like it’s an open innovation platform, which is what we’re doing here.
MS. BRADLEY: I can show you some of the stuff we’re doing. There are certainly things we’ve been using. I’ll need to pause because there are three or four different kinds of programs, I think Innovates might have been consumed by one of the other programs, and see what you’re thinking about. Some there is the judging of the projects. Are you looking for judges, or–
MR. CROWLEY: I am not trying to make anything overly complex, but basically a light platform where we could pose the problem, have people who write opinions, maybe have some community voting or vetting within the structure.
MS. BRADLEY: Sure.
DR. CARR: Let me see where we are. 90 day timeframe, small project, and proof of concept. What is the infrastructure that we would need? We don’t need a hall or a prize necessarily, but we need expertise to address this. We mentioned that Ted Smith would potentially be interested. Are there folks that we could bring into it as part of civic responsibility to help the government? Is that feasible? We would be looking to invite a volunteer or several into this project, this proof of concept project, and to partner with us in an ongoing fashion as we studied the things that were done and what was learned. Is there anybody who disagrees with the model as just stated? Hearing none–
MS. BRADLEY: Wait, can I– I don’t know exactly how it would relate, but is there potential to have case studies? If there’s so much of this already going on, could we in evaluating not leave those case studies behind? Maybe they go in the toolkit of, look, this is what King County did five years ago?
DR. CARR: I agree that we are trying to learn on our own, some of us more knowledgeable than others. Do we do both things? Do we convene a hearing or a virtual hearing or a presentation, a webinar, just a webinar really of a couple of different groups presenting what they did, what were the issues that arose, what were the challenges, and the opportunities? Would that be good pre-work before we embark on our next step?
DR. COHEN: Are there communities who have actually used social media data to link to existing traditional source data to address a problem? Are there?
DR. ROSENTHAL: I think it depends on what you mean by that. If you think of profile intervention and communication around intervention and then outcomes, there are definitely a whole bunch of communities doing the intervention piece of it. This is Chicago throwing up some stuff and manually having a team of typists search through and say, hey, by the way, we have this resource here, and sending you a web link. That’s a different thing than actually tying that to an outcome.
There’s not so much of that. You’ll find social media being used on the profiling side, which docs, which hospitals, which areas of influence. You’ll find it used on the dissemination, PR, and intervention of awareness and literacy, but in terms of actually tying it to HHS data in a meaningful way around outcomes, I’ve found very little of that. There are teasers in some of the academic works, and I’m sure there may be things out there, but I think on the whole it’s very nascent. It’s e-commerce 15, 20 years ago.
DR. CARR: Let me ask if there are things that CDC is doing, Vickie?
DR. MAYS: I agree with Josh that the way in which it’s being used is a lot in terms of the intervention side, that’s really big use. I think the flu campaign– there are lots of ways in which it’s being used. In terms of the actual analysis of data in which there’s some kind of linkage back to HHS, I’m less familiar with it. I’m more familiar with people that are doing it relative to NIH funded data sets where they’re doing it, but not necessarily that it’s something owned by HHS.
DR. CARR: So we have two Vickies. We have the Vickie on the phone from CDC, so let me ask Vickie from CDC, anything on the horizon there?
MS. BOOTHE: Not that I can think of right off the top of my head.
DR. ROSENTHAL: You will find exploratory, big data projects, NIH sets, but in terms of public, simple, easy-peasy HHS access to data that anyone can go get at a community level, there’s very little of that. I would write that into a separate taxonomy.
DR. CARR: Let me go back to the question that was posed by Lily, are there people that we could put on a webinar and hear what they’ve done? It sounds like the concept is good, but the content, we’re still running up against the fact that there isn’t anybody doing this.
MR. KAUSHAL: To Josh’s point, it’s nascent. There are some people doing it. Some of the folks I know doing it wouldn’t be willing to share it. A lot of this is going on in the private sector as well, so they’re experimenting with what the data could be, new business models, doing something with a small start-up, but I think it’s too early for them to start sharing that. For us, I think we’re really well-timed to do something like this and bringing it out into the public domain.
MS. GOSS: If you are looking to garner the information to support case studies and really see who’s out there and who has the principles of experience to offer, why not put an RFI together, request for information, with a series of questions and issue it widely as opposed to doing a webinar? You can get information back. You can support your data acquisition, and you can inform your refinement–
DR. CARR: Is that Michelle? Welcome, we’ve been looking forward to your arrival. We’re exploring some ideas. We’re still sort of evolving in our thinking, but this is an opportune time for the workgroup to hear some of the things that we talked about the other day, some of the things you’ve been doing and what you’re envisioning and potential areas of resonance with the workgroup. Could I open it up to you and ask you to talk about some of the things we talked about on the phone the other day, what you’ve done, what you’re looking to do, and how, kind of recap from the other day?
MS. PLORDE: Absolutely, so I am the deputy director for King County Emergency Medical Services. We serve about 1.9 million people, and our office is actually the regional oversight for delivery of service out in our community. Over the past seven or eight years we’ve been looking at strategies for how to address some of our specific call volume areas, and in our upcoming six year lute plan, we came up with a program called Vulnerable Populations, so looking at specific target groups to try to address either their access to our system and/or while they’re accessing our system, are there any things we can improve?
Some of our ideas, and this is where we’re ramping up because the lute period starts next year, and we’ll be hiring a program manager to implement some of our ideas. Where we’re looking at are some areas addressing the interface between callers and dispatchers. Also another area would be the interface on scene between our service providers and the people who call. Then the third area is really looking at the follow-up, what happens in trying to interface them into the community services that may be required.
In the conversation with both Josh and Justine a couple of days ago, we were really strategizing about what are some of these areas that we might look at in terms of data collection and in terms of engaging with some of these vulnerable populations. We have done some studies around limited English, and those folks calling into our systems and some of the challenges around that. We are partnering with the University of Washington around some of these studies and trying to move forward, looking at the Chinese community and their access to the 911 system.
We’re also looking at high school students. We just did a study last year that indicated that limited English speakers and low income high school students had a lower rate of training and CPR as well as a lower competency in CPR when we’re testing them. We’re trying to target CPR training, access to information. I was just strategizing with my other cohort at VU-Dub(?) and looking at possibly stroke, possibly diabetes, and other target areas.
That’s the breadth of what we’re looking at over the next six years. What was intriguing to me was we sort of had a centered approach to how we implement our programs. I think Josh and Justine were really looking at different ways we can engage with our community with different approaches to data collection and public education. That’s where our conversation really was organized around. Did I do a good job on summarizing that?
DR. CARR: That is great. I will open it up to the workgroup in a second. What we have been discussing leading up to this is basically generating a small model, proof of concept, of the integration of HHS data and social media data. I guess we’re still kind of filling in the details of what that would look like. As you think about your projects, do you see any aspects of it that are early, quick kind of proof of concept that would help you then decide on a path for a larger initiative?
MS. PLORDE: That’s a really good question. One of the things that sparked my interest was a little bit around something Josh said about Facebook and social media. This is a challenge for me because I’m not deep into or super knowledgeable about what the capabilities are, and there are a couple of things that I’m interested in asking you folks, which is a little bit around is it the public education component, an intervention that you’re really looking at, or is it data mining and getting something out of it that you’re utilizing the different tools for. I think I asked that once before.
The idea I had, we’re very, very interested in for sure targeting our high school CPR training, but I’m not really sure where the interfaces with social media would be. Is it in asking questions or providing the intervention, meaning education around how to learn CPR in a limited English population or in a low socioeconomic status population.
We haven’t picked our target yet because we’re in that earlier phase, but that was kind of the idea I had. I wanted to bounce it off you folks to say is your interest really on the intervention side and using social media to get at people, or are you really– I also heard about the mining of maybe Facebook or trending around data gathering and information gathering around social media. I kick that back to you.
DR. CARR: So I think what we have been talking about is the latter, mining the discourse that is taking place in Twitter or other social media. Let me open it up to some of the folks here.
MS. GOSS: I have a sort of tangent question. It’s a really quick one. OneHealthPort, are you involved with them at all?
MS. PLORDE: Not that I am aware off.
MS. GOSS: Thank you, it is Washington’s HIE backbone and connector for the state strategy for connecting health plans, hospitals, providers. It sounds like an integral data point.
MS. PLORDE: Sorry, you kind of cut out a little bit. What was the comment?
MS. GOSS: I was trying to link why I asked the question and explaining what OneHealthPort is, providing the Washington state backbone for health information exchange in a collaborative with healthcare providers, health plans, et cetera. It seemed like an applicable data integration point.
DR. ROSENTHAL: In my mind at least we could use the social media data in all three phases, one in the profiling of the target– I think just as important when we talk about toolkit, techniques, part of it is a mental framework. We put it out there as output, right? Ever so quickly, in my mind at least, there is opportunity to use social media in all three phases, profiling the target population, defining the problem, then the intervention outreach as a channel for awareness, literacy, et cetera.
Then on the assessment phase, data mining but also doing very tactical things, finding leading indicators, et cetera. I think there’s an opportunity to put out an entire framework to think about this, and all three phases of that problem have applicability for the new data. There should be three separate pieces of the output corresponding to how do you go about profiling a community, what are the downsides or target group, what are the tools, what’s the expertise, bit by bit, profile intervention and an outcomes assessment. By and large, most people like you’ve described are the furthest along on the intervention piece of it. There’s definitely an opportunity once he’s set up that data framework to think about it from beginning to end of lifecycle.
MS. PLORDE: I like those options. I think from our perspective, it would be really very specific to which target population– I think it’s harder to use some of the social media for targeting the seniors in a falling prevention program, which is one of our vulnerable populations, but I think we’ve had to target it based on what the plan is.
DR. CARR: We are just kind of having group development here about the concept. I think your ideas and challenge and also your approach has really sparked some interest here. I think what we need to do though is create some framework around the work that we’re planning to do. It’s going to be the work of this afternoon to do more of that.
MS. PLORDE: One of the questions I had was that we talked about what we’re doing and a little bit about what you’re doing, but as we move forward, what are you looking for from the communities?
DR. CARR: I think there are a couple of things. At a high level, just a community that has a coordinated voice and focus and issues and resources and interest in addressing the problem. Beyond that, and this is something we were just exploring when you called, who else do you need to be able to solve this in terms of gaining access to the data, assessing the data, et cetera, that kind of technical expertise, and whether that resides in the community or whether we try to leverage folks that we know from around the country who might have an interest in participating is what we were just discussing.
DR. FRANCIS: I am Leslie Francis. I’m from the University of Utah and the privacy of NCVHS. I really just have an informational question for you. You mentioned some areas of interest, diabetes, some other areas of interest. I wonder if you could say a little bit more about what your interested in with respect to that area of interest?
Are you interested in things like finding out whether changes in transportation systems effect rates? Are you interested in finding out how to get people into have their hemoglobin A1C tested? What kind of questions are you interested in? Then, I’m interested also in just hearing from you what kind of data you’re currently using and/or potentially interested in using to deal with one or another of those questions?
MS. PLORDE: Those are really good questions, and hopefully I can explain this. From our perspective there are two approaches to what we’re trying to do. Obviously improving the health– EMS, which is unusual, is in the public health department out here. Most emergency medical services out here operate either in the private industry through the ambulance services or under governmental, fire departments and things like that. Within King County, it’s under the health department, and we contract out to the fire department to provide the direct service. We’re responsible for the oversight, the medical direction, development of the dispatch criteria, all the protocols for the personnel, all of that, and the training.
We tried to think of our service as a public health model. As I was mentioning before, our model before was really simply response and demand. About ten years ago, we decided that wasn’t sustainable. We started looking at strategies to try to reduce demand, and particularly focusing in on non-emergency calls. We implemented a telephone referral that got the lowest of the low levels of calls diverted over to telephone referral lines.
We started looking at strategies around targeting folks that either had chronic diseases or were accessing 911 because of other reasons, i.e. they didn’t have transportation, they didn’t have a regular source of normal care or primary care and were calling 911 for those reasons. We started looking a little bit more on the preventative side. That’s where our fall programs started where we actually go out and intervene with a faller and try to reduce subsequent falls, so we actually implement handle bars in showers and things like that.
That’s one aspect of the prevention side. The other one is really looking at trying to reduce repeat callers. We go to people– that’s where the diabetes comes in. We had one gentleman who called 34 times in one year because he just couldn’t manage his care. We try to get them connected into the right kind of care. That’s sort of more after the fact as opposed to preventative measures. That’s what our mission is, really looking at managing EMS calls on two sides, for trying to do the prevention and trying to do the follow up.
The other question that you asked is on the data management side. It’s actually quite simple because we require on every call, should there be a patient, that they send their medical that they send their medical incident form to us. We have a couple hundred thousand every year. We have an incredible ability to mine our data sets and look at trends. The one challenge we have is that we don’t have indices like socioeconomic status, race/ethnicity. We don’t have some of those indices, so we rely a lot on our public health department to provide that for us, and then we kind of compare. We’ve done that on a number of occasions, looking at substance abuse and some of the other things we’ve tried to address. Does that answer your question?
DR. FRANCIS: Just one sort of slightly further– have you used any social media data or do you have any thoughts about how you might?
MS. PLORDE: That’s why we’re on the phone with you. The public health department does a little bit around emergency preparedness, and I’ve been looped into that loosely. We have a very robust– we call it Vulnerable Populations and Emergency Preparedness. We’ve had a number of crises where there were a couple of dozen folks that died because they used the little hibachis during cold weather stretches. That’s kind of where it kicked off. We were really looking at how to get messaging quickly during an event to sub-populations. That’s a very intricate, robust system that’s set up for notification. We’re hoping to coattail on some of that to get into some of the populations that we’ve identified. But as far as social media, we’re neophytes in that area.
DR. FRANCIS: If I could just follow up one last bit, there are two really major different directions. One is figuring out how to use social media to get a message out. The other is how to get information off of social media that either helps you figure out how to get a message out or helps you figure out what’s going on. That could be interactive or– the two are not necessarily separate, but they do go–
MS. PLORDE: Exactly, which is why going back to what Josh was saying looking at it for profiling and getting information to target and strategize for your actual intervention and follow-up is a really nice way to categorize your utilization. I am just very intrigued, and so is my cohort over at the U. about looking at different ways to communicate with subpopulations within our EMS population.
Again, it really is going to be very dependent– we’ll have to go through what sort of falls through, meaning we’re going to come up with a list of things that we’re interested in that we know that our sub-populations– that we might have a feasible impact and then how to make sure we have in-roads in partnering with populations. We already have an in-road to the Chinese population and have found that they almost never call 911 even when they should, and trying to build the intervention that would help educate them about when to call– that’s going to be one because we already have an in-road there. Looking at strokes, another of our top priorities in general, over the entirety of our population, but we’re going to have to do a little bit of work figuring out how to get at that– which vulnerable population? You could look at a whole bunch of them.
DR. COHEN: Michele, I didn’t understand what you said your primary database was that you had records for. Could you please repeat that?
MS. PLORDE: Those are for each of our responses. We require everyone who goes out on a response to fill out a–
DR. COHEN: We are talking about an EMS log reporting system?
MS. PLORDE: I don’t know if I call it a log because they’re officially the medical records. Most of our departments now have electronic systems. They’re like EPCR, electronic patient care records. They’re protected. They’re officially medical records. They’re like a hospital record.
DR. COHEN: Are these in-patient records, emergency department records, trip transport records? I’m not sure exactly.
MS. PLORDE: They are designated as pre-hospital.
DR. CARR: As you can see, we’re all in a stage of discovery together. I think I don’t see any other hands up in the room. What we’re going to do at this point now is to just immerse ourselves back into the logistics of this, what we might do. Then what I’d like to do is give a call back to you later in the week to give you a sense of where we are and whether there is alignment in the kinds of things you want to do and the kinds of things we think we can be helpful with. It’s great to meet you through the phone and hear about the innovative work that you’re doing. It would be fun to partner together, but we want to make sure that we’re prepared to be meaningful partners.
MS. PLORDE: There are still some questions that we have, which is why I ask are there things that– I’m not clear what partner means on either side. I’m not sure what we would be looking for, and I’m not sure what you would be expecting from us. Just to be clear on our side, we’re a completely independent, functioning, financed project, but we are always looking for new and innovative ways, and the more minds are in the room, the more you can come up with interesting ideas. For me, that’s how I see it. If you’ve got a plan that’s at another level that could help with us, help us come up with new ways of doing things, I think that’s great, but also knowing what would you like from us and being clear about expectations around that. Those would be our questions.
DR. CARR: I think basically we are trying to assess what does it take to do this, what’s the value of this kind of data, what are the pitfalls, what are the challenges, what are the privacy and security issues raised by it. We’re kind of looking at a proof of concept opportunity, and then this workgroup, which is part of the larger NCVHS committee would like to take that use case and dissect into what are all these issues and how do we begin to think about them.
MS. PLORDE: Well nice to meet you all, and we’ll talk soon.
DR. CARR: Thanks very much. This was not as easy as it seemed last June. Again, where I think we are is that we’re looking for a finite proof of concept. I like Lily’s idea about inviting folks into it, but I think we have a challenge that either somebody has something innovative and proprietary, or they’re not quite mature with this yet. I’m just trying to say what’s on the table and what’s off the table. I wish we could do that. Maybe there are some folks who could inform us, but we’re back to– are we in agreement? Is anybody disagreeing that we’re looking for something for a proof of concept? I’m not hearing any disagreement. Would it make sense– I’m looking for direction, like where do we go with that? Do we go to Ted Smith in Louisville and try to come up with something there?
DR. FRANCIS: I have a question for you, which is you say proof of concept, but what’s the concept we want proof of?
DR. CARR: Is there value, are there– is there a product of value– let me step back a little bit because today’s work is the work of discovering, and I think that what we’re trying to get to is can HHS data be enhanced by other types of data? We have a heat map of obesity in America. It’s great. We have a map of many different things in America, and it’s great. It’s taking that data and making that map. Is there some other thing that we can do that is a different type of data– we have obesity and stores, we have all kinds of– we have that concept of matching of two different data sets from HHS, but now we’re looking into another domain that is unregulated and unexplored, but at least for flu seemed to suggest that there’s a different type of information that can be gained and add value.
DR. FRANCIS: Basically what I try to push was is our question can we find usable data and put them together, or is our question will this particular kind of combination of data solve one or another of their problems?
DR. CARR: Good clarification. It is the former. Can you take this kind of unstructured non-standardized data and put it next to some standardized data and have a broader concept that has validity than not having it?
DR. MAYS: We have given ourselves 90 days, so if we want to stick with that, I think there are a couple of things to think about. We need to pick a topic that’s well developed. We need to pick a topic that tends to be in the social media, and we need to, for example, make sure that we can get the people. I’m going to make a suggestion, which is that some of the ideas that there’s a lot of chatter about, which would be things like in the summer when we have heat outbreaks, there’s flu, we have obesity– I think we have to pick things that there’s a whole lot of conversation about if we want to see if this is going to work through social media.
The other idea of picking that is that is there something about what happens in the social media that’s different than the stagnant data that we’re getting on a yearly basis or however often from the federal data set? The social media part should be helping inform something that we don’t know that people either are or are not doing, or myths that are out there that we would shoot off a public health campaign about. I think we need topics like that.
The first three that came to my mind were that in summers we have all these heat strokes and stuff, and that’s getting worse. WE have the flu, which is time limited and we need to work quickly. Obesity has a lot of social controversy around it. We sometimes enter communities thinking that we’re going to get them to move a certain number of indicators, and they’re saying I’m find with looking with that way, I’m fine with weighing this.
Hearing what those dialogues are would help us to be able to make changes I think in terms of not having a stagnant thing. For me, it would be the social media part means we pick an idea that has that need for a time frame to it that’s different than the stagnant– I shouldn’t call the data– the time-limited frame of when we get federal data. I don’t want to talk about our data. I would throw that out as a way to think about it.
DR. CARR: Thank you. I think that’s tremendously helpful
MR. SOONTHORNSIMA: Maybe another way of looking at is what’s going on in social media. Maybe you guys have already talked about this, so forgive me if I’m repeating what you’ve already talked about. Look what’s already taking place in social media. What do the various communities discuss relevant to healthcare? That is, it doesn’t focus on healthcare naturally, but when they touch on topics of getting sick or looking for doctors, looking for those different ways to engage– that’s not really the starting point. All these venues typically focus on something else, they just happen to be the venue for people to share ideas.
Here’s another thought. If you were to look at all these social media venues outside of healthcare, don’t start with healthcare. I wonder if you couldn’t come up with an angle where you would truly marry– an opportunity where you could truly look for opportunities to engage with HHS data that we have and try to marry the two. I think that’s a goal, is it not? Trying to make good use out of HHS data. Typically if you look inside the industry, we tend to look at things more traditionally, even I do, like engaging the patient. In the example I came up with earlier, it’s within the domain of healthcare, just a thought.
DR. CARR: So here is one that won’t surprise some of you, and I’m being facetious, but something like football and head injuries, something like that. It’s October. I think that’s right. The challenge that we’re at is we want to figure out why Mandarin speakers don’t use 911, but there’s not a lot of chatter about that, so that’s the problem.
DR. MAYS: I think it depends on how you think about– you all probably can answer this. If you’re only looking at it on Twitter, it’s a problem, but if you look at it in community blogs or Chinese newspapers, if they can do that kind of scraping– that’s the kind of stuff we look at. That’s where the dialogue is going on. They’re reporting that the local hospital is bad because of this and experiences they have.
This stuff, like in the black community, you go on The Root, things happen, you really get to hear– and then it will tell you to go to another blog. You really within a matter of a couple of hours can see what the trending is about what people are thinking about. That’s what we have to do, be a little more creative based on the populations that we’re trying to reach out to. It’s not going to be Twitter, Facebook, and Google. It’s going to be blogs. It’s going to be the offshoot of the blogs and that kind of stuff.
MR. KAUSHAL: I completely agree, and I think we’ve struggled with this before. We have to go in without a hypothesis on some of this data and let the data speak to us. I think that’s the beauty of fusing this type of data.
DR. GREEN: I am Larry Green. I’m not a member of the Working Group. Justine, I want to clarify something. Is the idea of a solve-a-thon now gone?
DR. CARR: Yes, the solve-a-thon– no one knows what a solve-a-thon is. Everybody is saying what they think it is– let’s work from novel data and then how we’re going to do that. It’s not off the table, but I feel that everybody came to the table with an image of some event with balloons and prizes that represented a solve-a-thon. I want to dispel that notion and say we’re going to try to learn.
DR. GREEN: I want to endorse that. I’m absolutely convinced as of 2:41 that no one’s notion of a solve-a-thon is within reach this afternoon. I want to express support for that. I think it’s a great idea. I think a solve-a-thon is in our future, but I don’t think it’s in our present. I’m going to work with that assumption just for a minute. I want to underscore what you said Justine about the goal.
The goal for the work right now is to understand, like you just said– it’s not to answer a question or this hypothesis– to understand the combining of HHS data with data from social media. That’s our goal. That’s pretty much what the charge to the working group is that formed the group. I want to support the fact that if that’s what you’re going to do, that’s right in the center of your work and that’s what we’re supposed to be doing.
I want to argue that the crucial step in order to understand that combining is finding a place where it has happened because we don’t have the funding to produce it happening. We’re going to have to rely on naturally occurring phenomena somewhere in the country where HHS data and data from social media have been meshed up together for health purposes in some way. I did not hear us having found that place on the phone call. It still may be around there somewhere, but we didn’t hear it on the phone call.
The way I would think about this is not necessarily the way this group wants to think about it. We’re really talking about evaluating an implementation of the combining. Implementation evaluation is a pretty well-honed scientific enterprise. People know how to evaluate implementations. It would be a good method to use. Use standard scientific methods for doing that.
Then I want to channel and echo Josh. About half an hour ago or so he said what we need to know is how they profile their population, how they define their problem, how they actually implemented their data exercise, and then we want to evaluate what happened and what the effects were. I think Josh nailed it.
DR. ROSENTHAL: I think this takes us to the heart of it. If the proof of concept is how you study or how you evaluate and walk through those stages, and also can you produce a position or any collateral for helping communities solve actionable problems using HHS data and non-traditional data, social media, in a scalable way, then it’s very definable. All the conversation swirling about takes us to the heart of it. Twitter is a shorthand for a bunch of other stuff. There are issues– you can do a Google consumer survey. There are issues around this though. If you want to get a certain data density of overlapping data, Twitter is pretty well defined because there’s one percent geocoding, let’s say.
We know how to do that in the world. If you’re screening blogs, you have to get an IP address. There are different things going on. This takes us into the conversation we need to be having around that. That’s very good. Point number one, we’re going along where we should be. The standardization, there are tools for looking at social media data, standardized, that a community can hit.
I definitely think we need to keep this idea of data density first and foremost. HHS data density– in the back of our heads, we know CMS, especially if you’re thinking about any sort of market relation moving to this performance rather than fee for service, there are very well defined models both metrics and benchmarks around certain populations, Medicare, Medicaid.
When we’re talking about falling, there are specific STAR Metrics for that. There are specific readmission indices. Are you over actual expected– there’s a certain data density for that. If we go into concussion, you don’t have the same sort of density at a large scale. Point number one, we need to find a topic. You need to make sure there’s enough stuff out on the social media thing like you’re talking about, but there’s a specific kind of layer you need to filter on that.
I think Ob said two important things. One was there are topics around the health topic. Do people use the word “diabetes” or “flu” or whatever? What you find in this sort of exploring, having done some of it, is a lot of non-linear stuff. This is all the classic the people that buy minivans and they’re single men happen to be more obese than in other population-wide. There’s inner logic to it because it’s easier to get in and out of. With that non-linear stuff, you find a lot of that stuff in the social world as well.
Finally, density, do you have enough of it, broader, is it scalable for a community? That’s why we’re looking at the HHS data sets. We’re not looking at something that’s difficult for someone to get into, but something that has population density. On the communities, we could do one of two things. We could either pick somebody– we have a commitment from Louisville Metro who are doing this right now as we speak. We don’t have to do it. I like bourbon, but just so you know there are people who are doing it. If you want to open it up, you could do a standard press release, or you could partner with HDI.
MR. KAUSHAL: Maybe I missed this before, but what exactly is our budget to this? We have numerous options all with different time impacts and costs. If I’m hearing we have no budget, I think that’s an easy trade-up to make–
DR. CARR: I think what we talked about was to approach the usual suspects, RWJ or someone along those lines. Is that right, Larry?
DR. GREEN: When we know what we want to do.
DR. CARR: Right, hence today’s discussion.
DR. ROSENTHAL: That may be one part of the selection criteria for the communities that are interested. If they’re already doing things, if they’re willing to dedicate not only expertise and sophistication but dollars, that’s something we might want to keep in mind as well.
DR. GREEN: In my non-profit quilt everything together world, that’s very attractive.
DR. COHEN: I actually totally agree with Josh.
(Laughter)
DR. COHEN: We need to move forward. We need to find a partner whom we already know, who’s willing to participate with us on this rather than try to go out and find somebody new and explain what it is we’re trying to do. We need to pick a topic that’s relevant to this partner and to us where there are both data from the usual suspects and either they’re using social media data or there’s potential to use social media data to address that. We need to discuss and chose that topic together. Then we can begin.
DR. CARR: I agree.
MS. BERNSTEIN: I said this before to some of the people here, I think we still need to be careful about the optics of which community we pick and what kind of people we’re studying and who we’re actually helping if we’re going to get money from anywhere, what that story looks like when it’s publicized.
DR. CARR: Lilly has gone to the– were you going to show us something Lilly?
MS. BRADLEY: I am just pulling up a map with the HDC affiliates. These folks are pretty eager to work with us. In Lafayette, you remember Romesh Kumar, who did the Cajun Code Fest. Up here in Seattle we have the– oh, you know these folks, Peter Speyer at the Institute for Health Metrics and Evaluation. I imagine he’d be well-poised. Wyoming, Institute of Population Health– these are both people and they’ve submitted populations to HDI to the group that Dwayne Spradlin runs, and what they’ve committed to is helping us convene other folks to use HHS data to be exploring this space, and I think like us they’re looking for the appropriate role to play.
DR. CARR: This is good. We need structure. We need focus. We’ve got to come out of here with a plan. We probably need a break at this point. Why don’t we take a break, and then we’re going to come back and hammer through what are our next steps. Then Lilly’s going to give us some updates from the department. Why don’t we say we’ll come back at 3:00?
(Break)
Agenda Item: Continue Discussion
DR. CARR: Let me summarize where we are. I think it’s slow a little bit, the process of discovery, but it’s necessary. Where we left off is that we are looking for a 90 day proof of concept– we want to find an area where there’s high likelihood that there might be social media related. We want to find a group that has some infrastructure and some expertise and some HHS data. We want to partner to help us do this proof of concept and come away with it both with is there a case that social media can enhance the traditional learning from HHS. Secondly, what does it take to get that social media data, and what are the lessons learned in terms of gaps, obstacles, challenges, whatever? I would like to now turn to my colleague, Josh, to make a proposal as to how we might achieve all this.
DR. ROSENTHAL: Let me just throw out one example of something we could do potentially quickly. This is back in– I’m not set on this example at all, but I think if we’re going to do something in 90 days that achieves these goals, it has to be something similar to this. Can everyone see this? Let me tell you a quick story and make a proposal. We partner with Louisville metro and Ted Smith in particular around a particular intervention that’s going on in Louisville. It’s called New Roots, Inc. You can look at it on their Facebook page. Basically they’re taking produce out into impoverished areas with an idea that they might be able to see some health outcomes.
The data they’re currently collecting is just your standard survey-based data, et cetera, although because there’s enough density of population and co-terminus geography, county, city, et cetera, they’re looking at youth and elderly. There are pre-defined metrics at least on the Medicare and Medicaid side, as well as the provider and hospital side. We can see data not only in terms of internal collection, but even population health data.
Another interesting thing about this is this is a non-profit. A woman at the CDC, Wendy Kersinger, basically a biostatistician, helped them frame up an application for Robert Wood Johnson. They won it in 2013, so they’re funded by them around this. Ted at Louisville has significant social media expertise and a budget they’d be willing to put towards this. We could potentially look at intervention for fresh fruits and veggies, is there any outcome using their internal data, using pre-defined HHS data, using their expertise and analytics department to exist, as well as potentially some expertise here at the CDC.
That would be my quick, 90-day turn around proposal, something current, something backed by Robert Wood Johnson already. Then we’d essentially go back through and say can we use social media, whatever it may be, to profile that population, to look at the nature of the intervention and message delivery, and then outcomes as well, and also with an eye towards outcomes– tying that to specific HHS data around those populations that’s easily accessible so any community could replicate it.
DR. COHEN: I think it is a great idea. A couple questions, is the target population in the city of Louisville or the surrounding counties?
DR. ROSENTHAL: I have to dig into this a little bit, but the city of Louisville, and they’re focusing on some of the areas on the west–
DR. COHEN: So neighborhoods in the city. That would be an interesting challenge. I think it’s a great challenge for applying the usual suspects in terms of– some of the data are there from Medicare and Medicaid, but a lot of the basic surveillance and survey data systems really are targeting much higher levels. Thinking creatively about can we do small area estimates given characteristics of neighborhoods and those kinds of approaches would be a very creative synthetic way to capitalize on federal data streams as well, as well as Department of Agriculture data systems and a variety of others. Go with it.
DR. CARR: I would like to hear back. Let’s run the table.
DR. MAYS: I don’t know. If I think about California, we have so many of these already going that if you ask would we then change and do something different– I don’t know. For example, we have our First 5 program. I think of all these different places where they’re doing this already. It’s kind of like, if we do it, can we get other people to change? They’re trying to collect data to see what difference does this behavior make.
DR. CARR: Are they using social media?
DR. MAYS: A few, yes, and others, no.
DR. COHEN: That would help us build in the piece we talked about trying to identify other communities and areas with similar projects who’ve had experience in this similar space. I think that would help us understand and make our efforts more productive.
DR. MAYS: Maybe the social media would be the contribution. I am trying to make sure that what we do is an add-on. Maybe it is if we’re really good about the social media part, then that’s the add-on that some but not a lot have. Then I think we’re value-added, but it’s just worries me a little bit.
DR. CARR: I appreciate your expertise and big picture experience. You’ve been at a lot of these big solve-a-thon, hack-a-thon things like that. I think we need to do one, because Larry once said we have to do one thing before we do everything. We’re putting our toe in the water. I think there actually is a clear opportunity here for us to look at what social media, where, how, and as Bruce says, to compare it to programs that didn’t use social media. Are we better off because we did that?
DR. MAYS: Could I ask a question? Can you tell me what in the social media they’re doing?
DR. CARR: They’re not. We are going to approach them. We’ve had a conversation with Ted Smith to say what if we piggy-backed onto what RWJ has already funded you to do and we’ll add the dimension of social media. They have resources that actually could do that. They hadn’t had that as part of the original plan.
DR. ROSENTHAL: There are kind of three parties here. This is just an example. They’re just doing your standard push, what everyone’s doing. They’re not using it to profile their population. They’re not really using social media in any sort of sophisticated way for the distribution of it beyond influence and PR and communication. They’re not using it to identify any outcomes. If you do this, are there key words that are health topic related or non-linearly, not health topic related for certain groups of people that predict what you’re going to see for readmission or outcomes or prevalence or utilization or risk?
They’re not doing any of that sort of stuff. I’d also say they’re not doing what a lot of people– I haven’t seen anyone really do this– they’re not connecting it to HHS data, behavioral risk surveillance factor, or even Medicare or Medicaid. They’re not looking at it from a population in that way.
When they do this, here’s a question. They do this for the elderly. If you sum total traditional fee for service or you want to look at Medicare advantages, for instance, if you look at those eight related metrics or conditions related to food access, when they started this intervention, they’re not looking at the change year over year. There’s this HHS framework up there that they’re not using at all. They’re just doing their own internal data collection.
DR. MAYS: I get all that. What I can’t get is from the community side what they’re doing that they’re going to capture social media.
DR. ROSENTHAL: It is just a pre-existent non-profit that isn’t related to the community whatsoever. They had a CDC researcher help them frame up the Robert Wood grant, and they got it. The community is the third party here. Ted and company are saying we’ll do anything you want us to. We can do that social media profiling for the population, outcomes assessment, and even some of the analytics as well. There are three parties, and then there’s the community saying, you know what, this is an interesting intervention that’s going on. You’re interested in tying it to HHS data and employing social media data in a meaningful way, and that particular non-profit has been responsive to having CDC help them frame things up around that.
This is just one example. Anything else we want to do is just an example of having resources and dollars that can help you on the social media side, which would come from the community, having a pre-existing intervention which is pretty good. It’s not bad, and then also having applicability in terms of population to some of this HHS data, not the risk surveillance stuff, but at least other kinds of outcomes pieces. It’s HR freestanding, basically. Does that help? I’m not explaining it well, I know.
DR. MAYS: No, it helps. I agree with this. We’ve got to put our toe in the water, and since it is 90 days and not 90 months and $90,000, I’m okay.
MS. BRADLEY: I was also wondering if in Louisville with the Asthmapolis, was there a game component to that with the kids, that gets shared? There are asthma metrics, so that could be a different one we could do right there.
DR. ROSENTHAL: That is also one other thing to think about as we frame up what we’re doing. There are communities that are using vendors to turnkey this stuff. Asthmapolis is a little bit of that. Louisville did the right thing and presented with Asthmapolis at HDI around that. That’s kind of a vendor doing the work, not the community doing the work or a non-profit doing the work. There’s that, and there are a couple of other things that we didn’t like. There’s a couple of other things we could do.
MS. BRADLEY: I thought Ted Smith did it.
DR. ROSENTHAL: It is Ted Smith in Louisville, but Asthmapolis– there are vendor components. It costs money. It’s not like you go in and do it on your own.
DR. CARR: It’s geocoding for your inhaler.
DR. ROSENTHAL: There is a bunch of different things we could do in that location.
DR. FRANCIS: I guess I am kind of puzzled about what it is we’re going to be measuring or trying to figure out what we’re using the data to measure. Here’s what’s– it’s going to take a long time to figure out whether the availability of groceries in the community has an appreciable impact on–
DR. CARR: That is the intent of the grant that was submitted to RWJ. We are piggybacking onto the infrastructure there to say if one were to enhance this information in any way, is there something in social media, and if so, where do you look, how do you get it, what does it tell you, what were the pitfalls?
DR. FRANCIS: Could we find a better– now, I understand the question. Could we find a better answer in the RWJ funded study using social media?
DR. CARR: I think we are trying to say we need someone who has the proper infrastructure. It looks like this RWJ, Louisville well-populated group– then we’re trying to have a vehicle where we can look at what kind of social media might want to add to that. What does it look like? How easy is it to get? How easy is it to integrate, to geocode, or whatever? I thin, it’s very procedural, tactical, but at the end of it, it could be a tool that a community could use. I think while this is based in the community, a community initiative, it’s really coming close to what we set out to do.
DR. ROSENTHAL: I think one way to think about it is here is a well-defined intervention with a defined problem, and they’re going to measure for an outcome. Our output is going to be something different, a toolkit or an assessment for how does social media impact, can it impact either profiling, intervention, or outcome?
DR. FRANCIS: That gets at what I’m trying to figure out. Are we going to be looking at whether we get a better picture of outcome change by incorporating social media data into the other data that they’re using, or are we going to test the use of social media as a way of augmenting their outcome? The latter might involve things like using social media to show people how to cook the vegetables rather than having them rot, for example. I don’t see which way we’re looking.
DR. CARR: As soon as we finish rounding the table, we’re going to get tactical. Who’s the group that’s actually going to get into the details on this. We thank Josh and Lilly for suggesting it.
MS. KANAAN: I think this is basically an informational question. I somehow missed the discussion about why of the non-traditional data you focused specifically on social media data. Communities are trying to– part of the challenge and opportunity for communities is to use a variety of different kinds of data sources. One of the ones that was profiled so nicely or highlighted in the San Francisco summit that some of us went to is local government data. It seems to me there might also be opportunities to also leverage local government data or other data sources in addition to social media data or HHS data. I just wondered– I guess the question in that alongside the suggestion is why social media in particular or to the exclusion of other “non-traditional”? Is that your explicit charge?
DR. CARR: We are going back to our charge. Our charge is to innovative uses of HHS data. That’s what we were trying to say. As we look at our charge, it says HHS data and innovation. That was the innovation that we thought of. We didn’t think of local data– local data seems to be more traditional that one would use. We were trying to take it to–
MS. KANAAN: Some of the new uses of local data are not traditional, like Yelp data and restaurants–
DR. CARR: I mean I would have a broad definition of that. It would not exclude Yelp data. It just would be county indicators is what we’re saying. It’s not just those ones, excellent point.
DR. ROSENTHAL: First of all, the charter specifically names social media data. Second of all, Yelp, most people would consider that social media data. Third, in addition– I think having additional data is worthwhile. In this little instance, it’s also socioeconomic data, specifically wage stridations.
DR. GREEN: So I want to ask a question, but first I want to react to Susan’s question. It seems to me that we stay on charge in scope if we are certain that the enterprise includes social data. I don’t see why it cannot– once social media are included in consideration of being mashed up with HHS data, state data or other data that were involved also, so be it. It seems to me it should be a permissive filter, not a restrictive filter.
DR. CARR: It is not in lieu of. We need social media, we need HHS, and anything else can go into it as well.
DR. GREEN: I want to get to Vickie’s points and ask a question about when I look at Lilly’s map, and I have heard at least six people on NCVHS say this is already happening. Should we be able to identify more than one site to work with? Would that be okay?
DR. CARR: We actually did exchange emails with Dwayne Spradlin to tell him what we were doing and that we would have our conversation today and then go back to him. I was just trying to get structure around what we were doing. I think that if we identify that we can find an interested party that is willing to add this onto what they’re already doing. Our role is observer, analyzer, commentator. I think we have bandwidth for that. I think if we were trying to stand up a whole solve-a-thon, we would be consumed with that.
DR. GREEN: I personally agree with that. I just want to make sure there’s consensus in the working group with the notion that this does not have to be a solo performance. I think given our budget we could have as many people– it’s not like we’re going to exhaust our budget by adding another person. We might exhaust ourselves in our human capital.
I would like to see a scope– because of the nature of the work, what we’re really talking about doing here is a qualitative observational study of naturally occurring phenomena so we can understand it enough to figure out what needs to happen about it. What strengthens that type of research method is a comparison. Right over the horizon to say that in Louisville, here’s what they were doing. They ran into three big problems. By the way in Seattle, or God forbid, Wyoming, they ran into the same three problems. Now we’re getting somewhere.
DR. CARR: I like that. As we think about how this plays out, if we were able to identify three programs that have the kind of infrastructure or we could share among those places how to proceed, then– when’s our next meeting– we could at a certain point in time bring groups together and have a hearing with very specific questions about what did you do, how did it work, touching on the privacy issues, and the logistic issues, the transferability.
DR. GREENBERG: My reaction, and I think that’s what you were saying, to people saying there are people already doing this is, okay, let’s bring them in, hear from them in a structured way. They don’t all have to be addressing the same problem, but they’re involved in similar processes. Short of the datapalooza, it’s almost impossible these days to have a conference, but you can. We can as a national committee hold hearings or the equivalent. I think that’s where you’re going.
Before you actually try to launch some kind of event, let’s bring them in, work on getting some structured information from them, learn from them, and maybe out of that would come a) some common practices, and b) some common problems and a) ll of that, but also maybe a proposal for something that none of them really are doing but they think would be really interesting and would like to be involved with. That would seem to– and that we should we get on with that in early 2014.
DR. CARR: Actually, it is not too early to start planning for next year’s datapalooza. I think if we did this, it would be very interesting information to present there as well.
DR. MAYS: I just want to caution us not to get too big too quick. I think the notion of the comparison groups, maybe that does come later, but we set ourselves for 90 days. We’re starting anew so we don’t know exactly all that we’re going to find out. If it was a proven entity it would be fun, but then let’s use the meeting that we’re going to have to bring in people that we think could be comparison groups, but to launch can we do– 90 days is going to go like that.
DR. CARR: Very judicious advice. I think I will follow up with Dwayne to now be more specific about where we’re headed and ask him if he knows of groups that are doing something similar to this already.
DR. MAYS: And RWJ. They funded them, and they might be happy to just do whatever we need extra to take them as the controls.
MS. BOOTHE: This is Vickie. I actually wanted to ask a question. Did you guys say you know of a community that’s actually using social media to assess the impact of more fresh fruits and vegetables in that community?
DR. CARR: No, we’re saying we know a community that is assessing the impact without benefit of social media. We plan to approach them, and we have some preliminary reconnaissance that tell us they would be interested to add that to their study so that we add a dimension of social media.
MS. BOOTHE: So I would just comment that thinking of it from a community level data perspective, this is going to be probably way more challenging than using social media data to assess an outbreak that has specific symptoms because you’ve got that long chain of causality from everything to access to affordability to just because they’re there, are people consuming them, preparing them in the right way, all the way through to– I mean, are you trying to get to diabetes or obesity? There’s also the issue of how big is the geographic area for which they’re expanding access and what’s the scale or the geographic identifiers in your social media data.
I would challenge you guys– I think it’s interesting, and I think if social media is really going to add value to existing data you really have to see if it can assist with some of these chronic diseases where all of the attention is. This sounds really challenging to me.
DR. CARR: Thank you for that. I think we are looking at it really as two outcomes. Our outcome is really going to be the logistics. When one has a topic, where does one look for social media data? How does one capture it? How does one leave out Whoopi Goldberg from whooping cough, all those kinds of things? A longer term RWJ applicant or candidates will have to look further to say did we see value from having social media data. I would say we have a short term goal, which is simply how does one do this, what are the obstacles, pitfalls, opportunities? A longer term will be, as this project matures– it would look to that group to assess what value it added. I think–
MS. BOOTHE: I agree, I just want to say that if you get the wrong search terms or it’s a geographic mismatch, that could be the source of it not providing any utility. I would suggest that some careful thought be given to that.
DR. CARR: That may be one of our outcomes, that it all seems great until you try to get down to the detail. I think that unto itself would be a contribution.
DR. ROSENTHAL: Hi this is Josh, this is a microcosm of the problem we’re talking about. You have some experience with this, and you’re talking about a relatively small percentage of geocoding being coterminous with an intervention like location. Most people don’t understand what you’re talking about. In fact, it’s far more complicated than even that piece of it. If you’re talking about involving HHS data on there, then you need a common framework whether you’re talking internal community collection, top-down prevention– there’s a whole bunch of other stuff. Rather than having it exist as just in some people’s heads, the idea behind it– consider it a meta-analysis. This isn’t redoing an impact and finding causality in a classic study. This is basically being able to carve out a position paper, a toolbox, tools, tips, tricks, and techniques, methods–
MS. BOOTHE: I see, it elucidates all that complexity that isn’t in a case study.
DR. ROSENTHAL: How do we take your conversation and document it and expand it and actually do some real world tactical analysis around it?
DR. CARR: I think we’re on exactly the same page.
DR. GREEN: Let’s remember her phrase there: you’re going to elucidate the complexity. That’s really good.
DR. CARR: That is what we do.
DR. FRANCIS: I have another question about what we’re thinking about here. Do we think that by using social media data we’ll be able to answer different evaluative questions than the original ones? Let’s say the original evaluative question is frequency of emergency room visits before and after the veggies show up. Do we need social media data to figure out frequency of emergency room visits, and if we do, how can we use it? We haven’t changes the question.
Now suppose what we thought the social media data would let us do is figure out in addition to frequency of emergency room visits who ate the veggies, who– well, anything identifiable, who do we need to target for further work-up? Are we using the social media data to answer different questions, and if so is the huge advantage of it that it gives us some identifiable individual information?
It seems to me that if we’re going down that path, it raises– there’s a whole new set of questions that are not raised by population based data such as correlating numbers of vegetables with numbers of emergency department visits. That’s a sort of silly version of the example, but you get the point.
DR. COHEN: I don’t know whether social media data will be better hypothesis generating or hypothesis testing. I also don’t know whether we’re talking– I do know we’re not talking about causality, but we’re talking about association. I think communities need to understand that as they look for any kind of data to help them identify areas that they need to explore further. At this point in time, my concern isn’t about individually identifying persons who need interventions but really trying to create the context and landscape to help communities address problems.
DR. CARR: I think that is one of the things we take away. We’re so accustomed to very granular, specific peer-reviewed data, and this is direcitonal. Have you thought about this? This looks like it might be important, or not. That’s how one uses this kind of data as opposed to the more traditional peer-review, 17 year uptake data.
DR. ROSENTHAL: The difference in this conversation in a year or two is crazy. I don’t even know what to say. I’m sort of beside myself. This is just fantastic. I was going to give an example. What if it turns out that all the people who like Miley Cyrus performance, they’re all the ones who have horrible outcomes? They’re the fresh veggie cohort-Miley Cyrus likers. There might be some inner psychological gyrations. You can do some pathology or whatever, but the point is it might be non-linear.
DR. CARR: How many people Googled what is “twerking”?
DR. ROSENTHAL: So you do a heat map, and you see a pop-up, Miley Cyrus, and all of the sudden it pops up on the west side of Louisville in the neighborhoods co-terminous with the vegetable thing. Are the carrots making people like Miley Cyrus? They might not be, but the point is it opens up– you said it so well, hypothesis generating outcomes or evaluation. The point is to sketch it out and get a handle on this.
MR. CROWLEY: Just put a Miley Cyrus hashtag on where the vegetables are going to be.
DR. CARR: This is what I love about this group. Honestly, we really are a very different group.
MS. BERNSTEIN: I am sure it’s the first time the word “twerking” has appeared in the record of the NCVHS.
DR. CARR: I think that we have a plan. We’re probably going to have a couple of calls going forward to develop this. Thank Lilly for directing us that way and Josh for getting into the details. I look forward to this. We’ll follow up with calls. Everybody can take a deep breath now. Lilly’s going to update us on what’s going on with health data.
MS. BRADLEY: So I did want to give you update about some of the cool stuff that’s going on at HHS and across the federal government and actually all over the country. I won’t get into the world, but it’s going on all over the globe as well. Before I give you that high level view, I’m giving you a homework assignment that you can do right now during the meeting instead of checking your Blackberry. I would like to start creating this innovation map. It has a few components to it. I started by tracking innovators. This is actually called “Doers”. So when we were thinking about building out our teams, if you remember the 4 Ts, the technical, the tools, the team, sort of along those lines, doers can actually do the things we need them to do.
I actually don’t know Nicholas Stricklin, but he was a NASA innovation guy. Zachary Jiwa is out in Louisville. He’s one of our external innovators in residence. Richard Bookman, he’s an innovator down in Miami. These are people who actually will invest time to explore things with us. I’m looking for more people to add to this list. Who do you know that’s out in a community? Vickie Mays just helped to introduce me to AJ Chen. He’s a doer, a doer out in San Francisco working on health disparities. We’ve got HDC affiliates. There’s this lovely developer named Zach Goldstein up in Oregon. He actually works for Health Leads, which is a Boston-based non-profit. He’s looking at how to understand community resources.
You guys know these a lot better than I do. We’ve highlighted advisors here. Advisors right now include all of the NCVHS full committee members and workgroup members. I started to highlight some tools. As we begin to map this out, I hope we can bring them together to do some of the projects that we’re interested in. This one is actually the conveners. There are beacon communities throughout the country that we could be going to, to talk to, to try and figure out what they’re trying to do. The brigade captains are folks from Code for America. They tend to be both conveners and doers.
We’re trying to link up different kinds of skill sets. The developers know a lot about how to code, and they’re looking for a problem to work on. We have health experts here in the room. We have people who have list serves, so associations I even highlighted as how are you disseminating and getting some traction. The National Governors Associations–
MS. BERNSTEIN: Can I interrupt and ask can you back up for me a little bit, because this is the first time I’ve seen this? You’re trying to find people in this different categories who are in the same geographic location that you might make relationships with those people because they’re together in the same place? That’s why we’re on a map– all the people who are in Portland, for example, with your guy up there, or if there are Code for America people?
MS. BRADLEY: I actually did post something to a LinkedIn, it’s called the Health Data Group, that you are welcome to join.
MS. BERNSTEIN: Is there a specific higher goal for these people that you’re joining them all together?
MS. BRADLEY: This conversation about the Solve-a-thon, as we went around to try to recruit communities, the obvious thing that jumped out at me was that on the one hand it seems like we don’t have a community to work with. On the other hand, there are so many communities out there doing this. Somewhere in the middle is the real truth, and maybe one way I can help people see that is by putting it on a map. It didn’t really have to be a geographic map, but I didn’t have that school software that does the network-y thing with the big bubbles. That would have worked okay.
The funny thing to me was I had automatically defaulted to a geographic community, but the first suggestion on LinkedIn was actually on cystic fibrosis, and the community had defined itself as a community of cystic fibrosis patients and how they were using social media data to address a problem within their community and to further research. It’s just a way of 3D displaying information. I do think that proximity can be helpful. I would love to get these folks in Louisiana, for instance, to join up.
The thought here is if you know people who are doing cool things who we should be engaging with who you think maybe don’t know about each other, if you could just jot that down on a piece of paper, I would love to connect them. That sort of goes into how I’ve been organizing my work, which is where should be government be playing, where should HHS be playing? It seems clear that there’s a coordination role to be played at this level. From where we sit at the very top, there are a lot of people that come and talk to us about what we’re doing, so we have an ability to see and share that story.
The coordination then would be for us to help each other know about the places where things are happening. In terms of if someone else is doing it, I’d like you to be able to learn from their experience. If there’s another skill set, one, you should be working with each other so that you can share information, but you should also be connecting with people who don’t have your area of expertise.
There is a whole other layer to this that I would add, which is around HIEs. Each state has an HIE. In fact, Linda Kloss had brought up one of those. We shouldn’t forget about public health departments, public health schools, CHNAs. All the 3,300 non-profit hospitals in America have to deliver community assessments about needs. Sometimes when I hear that communities don’t know how to use our data, I don’t really think there’s an excuse for it anymore. I think there are people in the same community who aren’t talking to each other. Maybe we can take a step back and look at it from a different perspective.
What is the department doing? At the federal level we’ve been working with OSTP, and there was this May 9th executive order and OMB guidance about making our data open by default and treating it as an asset. This is Project Open Data. It is a website built on GitHub. This is actually one of the more complex GitHub sites. The point of this is actually that you can continue to help improve the content over time. NCVHS committee members can be doing this. Anybody in society can actually go in and make a suggestion.
It provides both the background to why this website exists, some of the key definitions. I provides implementation guidance. This is for the federal government. It provides all sorts of tools so that we can build a database to API and convert CSV to API. It’s got a JSON validator. The idea is that you can continue, and if you have another tool that you would want to add here, you could.
I was bringing it into the schema. A note down here as well is that we provide case studies, so when the Department of Labor went to build its API program, I believe it’s gone with the strategy of having one API across all of its data sources versus HHS, which has APIs for different kinds of data warehouses. The health indicators warehouse has its own.
Under resources, the metadata resources, this has become one of the critical pieces for moving things forward. It’s really boring in a way. I think a lot of us know that– this is like standards. This is the standards committee. We’re building standards around claims data and EHR data, HL7. We’re defining our core data for the metadata that describes our data. This is the way we’ve defined it. The criteria for it was to be extensible over time so we can add to it, so all of our data sets in the federal government will have these descriptors. This was decided out of OSTP and OMB.
These are some of the required if applicable fields, like time, space, license. You’ll notice up here there was one on access, public access level. There’s going to be a clearing house for the metadata itself, so if you want to understand what type of data the federal government is collecting, you’ll be able to. The expanded fields are defined here.
There is a piece of this that the NCVHS working groups should be particularly interested in. There are other folks in the room that could speak about this more clearly than I am, but I’m learning and I’m sharing with you. Josh is nodding his head because I think he knows more about this than I do. The W-3 is this consortium that is a standard setting body. They’ll publish, for example, this. This is a proposed schema. Schema is the term they use for metadata, one set of metadata to describe a certain type of thing.
This was proposed by Google.org recently. It is the services schema for civic services or public services. Basically what they came back and said is schema.org is relegated to W-3. There are four major search engines, Bing, Google, Yahoo, and the Russian one. They crawl the web, and they try to return relevant results to users. They do that based on what the webmaster describes the website as. Are you describing your website in a useful way? There’s metadata that describe our websites. Right now, the federal government, local government, state government, we do an atrocious job of telling the web search engines how to find us.
We started a dialogue with them and one area where they thought we could make some headway is around finding civic services. If you’re a citizen in the United States or you’re someone in need of a service, like I need to find a doctor, one thing you might have it pull up is that HRSA has a “find healthcare provider” or healthcare finder website. The deal is if we describe this on our website using the schema, this is a layer behind the website that we see, it will pick it up. Google will find us. We no longer need to figure out why we don’t make any of the rankings pages.
It’s difficult to read. I probably spent a couple of hours just trying to understand. They’re using reference models. If you’re the name of a building, they have a schema for buildings. You’re going to say in yours “I’m referring to the building schema over here”. There’s a thing and an object and then there are hours, so you’re always displaying hours in the same way for whatever service. One thing that I think we’re learning as we realize how important it is for us to be communicating in the same languages so everything’s interoperable is that we really should be talking to the librarians.
While I think librarians are looking maybe for a new purpose as well, we could be engaging them as well and building the ontologies. The web guys and the CIO’s office needs to be talking to the National Library of Medicine. The PubMed folks have been thinking in these terms for a really long time. They understand how the different kinds of concepts seem to relate back to each other. Hopefully there will be more discussion in that area.
I learned about this just yesterday. This is part of how being at the level where we sit there’s an opportunity to do coordination. The National Human Services Interoperability Architecture, does anyone here know about this? Neither do I. It is run out of ACF, the Agency for Children and Families. It’s one of ours. This is a subgroup of– the NIEM is the National Information Exchange Model. This is going to be some component of it, and basically they are agreeing– well, this is for human services information. It’s targeting right now predominantly at the state levels. They’re going to try to use the same kind of schema I think is the point, that they’re going to start describing things in the same way so you can compare and you can pull things into the same type of data warehouses.
That’s through ACF. Hopefully they’ll launch a data.gov type of site. Yesterday, CDC launched it’s data.cdc.gov, and that joins the family of cms.data.gov and up and coming will be data.fda.gov. The way the government’s doing this right now, we’re consolidating but we’re still a federated model for the data clearinghouses and exchanges.
If everybody uses the same kind of format, that would be nice. NIH is considering its own data catalogue. They’ve launched at least $100 million in it to do a big data to knowledge initiative. It includes translation. It is this other piece of the pie– they’re working a lot with biomedical informatics. They’re looking at how to leverage RCTs.
Something else that we’re looking at, usability.gov was launched out of our department through ASPA, our public affairs group. It sets up some standards for how we should be engaging the user experience. One thing it uses is response of design, which we were really excited about. It means that you no longer have to have a separate platform or different system for mobile devices, that if you just program it through responsive design, it will show up correctly on whatever device you’re on, your iPad, your computer. If you want to ask more about that, please ask Josh.
A couple of activities within the department, Brian Sivak’s group, the CTS group, in some ways is modeling out in the department what kind of activity we would like to encourage outside of government, so we’re trying to be a model. We’ve launched a second round for our HHS entrepreneurs program. There are two. I don’t think we’re doing all of these. HRSA’s going to be doing a cloud-based GIS map to display aggregate data on medical malpractice. Then we’re going to be modernizing the national plan and provider enumeration system. That’s probably very exciting to some of you. This is how we track doctors, I believe, through Medicare, and it’s in need of modernization.
Something else that we’ve done that I found particularly fascinating, and this probably goes back to what Kenyon and brought up earlier, instead of HHS Innovates, which might have had a different time horizon, this is HHS Ignites. We looked for proposals across the department. There was an open period. You could get some help desk support if you wanted to learn how to apply. We offered up to $10,0000 in funding. The Secretary was the ultimate picker.
This was judged by four judges outside of HHS, GSA, DARPA, VA, NASA. Thirteen proposals were selected ultimately. I think it was only going to be six, but the secretary went all in. The CDC micro-tasking project, this is something we’ve borrowed from USAID where you could actually engage with the outside world to get help to do some of your micro-tasks.
MS. BERNSTEIN: Going back to the ignite thing, what is that? I heard all how we picked them, but what are they? What is that program for?
MS. BRADLEY: Basically it is to encourage innovation within HHS. If you’re an HHS employee, you could be anybody, you can have a cool idea and propose it. You have to get team support, but you’re getting cover to run a project. There are 13 examples I’d love to walk you through. Mike Grotowski(?) was one. Right now USAID is doing this, and we’re trying to see how far we could leverage it. Say you have a task like you are staff to the committee and you would like to write up 50 case studies of what entrepreneurs are doing out in the community doing to improve healthcare using social media and HHS data. I can micro-task that out and ask people, could you write this one up? Could you, one, nominate it, and two, write it up? Those are the kinds of things we’re looking to the outside world to do.
There’s a CDC health game jam. There’s coordinated press response strategies. They’re trying to create an internal tool and application for organizing cleared statements. This was the issue, that they had these cleared statements, and they don’t have a great way of tagging it or managing it. Each time they got a press inquiry, they were reinventing the wheel.
This doesn’t fully relate fully to what NCVHS data access workgroup does, except to the extent that the department is learning to be innovative and take new approaches to solving problems, these are things we can support outside.
DR. CARR: Lilly thank you. From the things you presented at the beginning, who is the expected audience and are we reaching them? Harkening back to a year and a half ago, as you ran through those things, I could see that Josh was on top of all of them. I was not. Is it expected that the audience is really developers or people who use that, or what should it mean to the rest of the group? If we’re not developers, it’s important information, but how should we think about that?
MS. BRADLEY: So Josh is welcome to say– can I make one stab at it, which is that I think for us it’s that we should be working more with the developer community, that we need to break down some of these barriers that are artificial. If we ignore things that are like the schema, which is how the search engines are picking up our information, we’re just not doing our job as public servants because we’re not getting our message out there. It’s just not appropriate for us to ignore.
DR. CARR: I think there are internal HHS things to do, and certainly we want to provide suggestions, but in terms of the scope of things that we should be really focusing on and giving feedback on, there’s a spectrum, some too detailed or too specialized knowledge for everyone in this group. Then there are other things that are bigger picture.
DR. MAYS: Some of these I think the committee would probably benefit a lot from getting updates. As she was talking, it would be great if she did an update for this group the way that Jim does for the other group, but she has to make it more user friendly in the sense of what does this mean? For example, the National Human Services Interoperability Architecture, there’s a ton of stuff I had to learn just to be able to pick a person to do the work.
There’s a lot of stuff that we don’t know. I think it would help both sides to be able to have a better conversation. Knowing where these things are and how to find them and all that I think would be useful for everyone. Just park it, and if you don’t get what it is, just put it in the save file, and then when you need it you’ll at least know how to find it. It’s very hard to find this stuff. I think if she did an update the way Jim did.
DR. CARR: I would just parse it, and we can talk it through or whatever, but this is important to who in the group, for the different kind of work, or Bruce– you kind of did that. I would just co-locate that. There are things that are very technical that a couple of folks in the group might know about. It’s a lot to take in, and I’m nervous that I didn’t understand everything.
MS. BRADLEY: So part of what we discussed, Debbie and Susan and I were talking about the update to the NCVHS website, and if you go back through and look at what is going on in this area even though it feels very far ahead of me– this is nothing, I spoke seven months ago. We’re putting our reports on our website. First of all, there’s no metadata on our website. Google has a very difficult time finding it.
Why aren’t we just posting it to the bookshelf at the National Library of Medicine on PubMed where everything is fully indexed? Why aren’t we better posted to the FACA? GSA runs a general FACA page. We’re publishing a report. Did we publish proper metadata behind it? If we didn’t, why not? I would like people to find the great recommendations we’ve made. There’s some space for us to learn from it. If we were updating it, shouldn’t we be using the new mobile device–
DR. CARR: For us as a reactor group, if you tee things up and then we could say oh my goodness, we should really have that metadata out there and we could speak with one voice back to leadership– I think this is a very important piece of what we’re supposed to be doing. It’s discovery. That is the name of this workgroup. It’s really shared learning. What’s your doing is hugely valuable.
As we think about it for our next meeting, you know how on the healthcare things for providers, for consumers, for payers or whatever it is, if you could tee it up like that for whatever those categories are, for developers, there’s a new taxonomy for standards. This agency is working on something that touches on the work we do, or for the full committee or for privacy– we can think about what those categories would be.
MS. BRADLEY: In fact there is this big thing for standards. That W-3, do you work with that Walter? They have a couple of different medical entity schemas, and we’re not even using them yet– I don’t know to what extent we are.
DR. CARR: What you are doing is fantastic, and you’re educating us. Let’s frame it up in a way so that our attention is focused on– for developers, we have this and that, and Josh and Kenyon can comment on wow, this is great, we need more of it. Maybe we give that feedback either through our minutes or whatever. For the full committee on privacy, do we know this or that? I think what you’ve done is great. If we can just put it in categories and provide some commentary, that would be great.
DR. ROSENTHAL: Would you mind just writing it out, just a couple of links? The geocoded piece, and if you’re asking for a list, what the ask is? The schema stuff, is that something– remember when we looked at how Google has three million and HHS has three, and one of the reasons for it was those metadata steps. Is that something HHS is doing, or we’re just saying here’s how to do it? What the ask is, do you need a list, do you need a comment on what the link is? There’s really good stuff. I’d love to look at it.
MS. BRADLEY: That is a great question.
DR. CARR: Maybe it is for Greg, maybe we have a conversation with Greg–
MS. BRADLEY: Greg isn’t paying attention to it.
DR. CARR: I’m just saying that our role is to be the reactors. Why don’t we talk off line and we’ll tee it up? If you can put together and get it out to folks today so we can have it.
MS. BRADLEY: So the one question I’d have then is it seems like the tabs of provider, patient– it’s leaving you too siloed.
DR. CARR: I am not saying that. I won’t be able to contribute much on the metadata, but Josh will. Walter probably could contribute to that but also would want to know about the standards thing. Leslie would want to know about that. If we just say a headline, here’s something new in privacy, or here’s something that we’re doing here but we’re not doing elsewhere, it will at least frame an opportunity for the response to say what if we do it everywhere or that aligns with the work of the full committee and maybe we’ll bring it back.
This is a perfect start. You’ve done exactly what we want you to do. The theme of today, we’re learning how do we take that information in and responsibly respond to the department with our advisory role. Thank you.
We’ve been so efficient today. We have a plan, and we’ve heard the report out. Are there any other topics that we’d like to bring up at this time? Hearing none, I would say that we can adjourn early today. I thank you for all the excellent ideas and input. We will get the work plan out shortly.
(Whereupon, the Working Group adjourned at 4:07 p.m.)