Transcript of the November 13, 2013 NCVHS Subcommittee on Population Health Breakout Session
[This Transcript is Unedited]
Department of Health and Human Services
National Committee on Vital and Health Statistics
Subcommittee on Population Health
November 13, 2013
National Center for Health Statistics
3311 Toledo Road
Hyattsville, MD 20782
CASET Associates, Ltd.
Fairfax, Virginia 22030
P R O C E E D I N G S (3:24 p.m.)
Agenda Item: Welcome
DR. COHEN: So what we hope to do – ultimately, we will get filling out our matrix, which we all had about the details for our work plan. Since I am senior member or senior co-Chair of the Population Subcommittee, I get to ask Bill, since you’re here, partner, to begin this discussion. I think what we want to do – Bill will frame exactly what we hope to accomplish in the next 45 minutes to an hour.
DR. STEAD: What we are thinking or would like to put on the table as a straw person approach to maybe get traction on the long-range framework – we know there are other things we need to do – my sense is all of us are sensing a need for frameworks because the things we are doing as we try to converge, it is very hard to understand how to scale them without the framework. I think that we are all converged around.
My sense is no two of us are hearing the same things in the words that we are saying. If two of us do, certainly, no three of us are hearing it the same way. I think we have actually got to stop and spend the time to develop a little bit of a common vocabulary and to build some non-abstract examples.
What I would suggest is that we identify a relatively small workgroup that is drawn from each of the key scientific frames – that is the way I think about it – that make up this group and start something approximating a weekly call, in which we do something like work the obesity or some other use case. We can start with one and throw it away because we want another.
This will all be rapid cycle, iterative discussion work. I would think of us as having two white boards as we did this. One that had a straw person of what the data continuum framework might look like and one that has some hooks about what methods might look like. As we tried to work an example, we could pop things on one or another. At the end of each call, those frameworks would look a little bit different. Hopefully, we would iterate and oscillate and begin to get to something that looked like an example that would be concrete enough that we could then bring back to this group, hopefully, before the February meeting and it would make sense.
That would be one idea. A second thought, because I think it is the least risky thing we could do, is if we think we know enough to charter the literature review of the case statement, I would go ahead and try to have that work done by staff with whatever guidance they need from this regular call so that – because that will give us a basis. It is a relatively safe thing to do. It is not as invasive as starting hearings, et cetera.
If those two things made sense and we elected to go after them, we would – presumably, by the February meeting, we would have some examples of the kinds of things we are talking about and the literature review. Then we could decide how fast we wanted to try to go forward beyond that. I think, by then, we might at least have an interdisciplinary group that had built, together through those calls, a shared vocabulary, enough to let us try to frame this better.
That is an idea of how we might move forward. Did I get that right, Bruce?
DR. COHEN: Yes. I can try to record on here or I could record on paper and then transfer.
MS. GREENBERG: Can you define a little bit better what the literature would be?
DR. COHEN: That is a great question, Marjorie. That is where I think we should start, defining, first of all, what do we need to find out, if we need a literature review, and what that literature review would be.
MS. GREENBERG: I, personally, think your proposal can be in this small group, particularly if you are willing to kind of lead it because you understand what you have in mind better than probably anyone else. I think you are right. Concurrently, if we can continue to define the literature review – I don’t see why you can’t do that oncurrently or start it a little later, the literature review. It is not a huge resource investment, either one.
DR. STEAD: I think that the literature review would need to identify – I mean, I would start by identifying what has been written or reported around convergence of the data/measurement infrastructure to support these various populations scales. The two things I first read were the primary care to public health thing, which has a few pieces in it. A couple of sentences would pull out of that. The analytics and measurement has a little bit more, but still really doesn’t focus on that idea. I don’t know what else is out there.
MS. GREENBERG: Both of those are IOM reports.
DR. STEAD: Both of those were IOM reports. We, obviously, would want to make sure we back up the things like the 2000 report of NCVHS you mentioned. The question is just really to get us the foundation as we begin to work the examples.
MS. KLOSS: It is probably an international literature review because there really may be some thinking on this by other countries.
DR. STEAD: Maybe the way we do it, if we get the right person to do with that – I didn’t know if this was – if Susan is here – if this was something she might deal with. That would be appropriate. I didn’t know if that would be something you would be interested in. If so – if we had somebody that could do it, if they were to then participate in the calls, they would sort of watch the way this thing is going on. We could bring – we could do it a little bit in parallel.
My thought is we want the little working group to really be – I agree I need to be part of it. I could also bring the systems perspective to it. We clearly need the community perspective, which might be Bruce, to keep us continuing to – we need a privacy perspective. We need a standards perspective. Justine might want to be there for the workgroup. Given some of the conversations that Alex and I had over lunch, she might bring sort of a systems engineering perspective to it.
That is the kinds of group, I think, that actually might – if you had some group like that, you would end up with a common vocabulary at the end of probably several calls. I think that would put us in a position to sort of – I know it is not exactly meeting what Larry asked us to do, but it would put us in a position to really understand – if we could decide we wanted to make the investment and do that between now and February, I think by February, we would want to know how far we wanted to push something beyond February.
MS. GREENBERG: Thinking of when we first decided to have the – it might have been the first Community as a Learning System hearing. We just kind of asked key informants about communities that they thought might be good exemplars or ones that you should talk to. I am wondering if there is some way to do that – to start this literature review that way. I think, yes, there are several IOM reports, probably, that are in this space sort of. I have a feeling if there was something really definitive out there, you would know about it, Bill. Someone would have heard about it.
This is such a convergence. These are broad ideas. I can’t – it is not like starting a literature review on X. Where do you start? I think that there needs to be some reaching out to people, sort of a networking approach. Those of you who have people you know to even get started. Then someone like Susan, if she is available and she is not doing other things or some other person – we don’t, obviously – as Jim likes to say, we don’t make policy in committees. We don’t make assignments so much in person – like this, in a meeting – but could at least then find those documents and see what they
can find in them. It seems like – it is sort of daunting to just say go do this. I am just thinking of a way to direct the research.
DR. COHEN: I am trying to capture some of the points of the discussion. The two tasks that we are talking about now are a literature review for the case for convergence and a workgroup for developing the framework for a common vocabulary across the different subcommittees and data workgroup here.
DR. STEAD: The latter would build – take one or more use cases and begin to assemble an example of the data continuum framework.
MS. GREENBERG: You are moving that up because you think that kind of concrete work is necessary to get everyone on the same page?
DR. STEAD: Yes. In my written thing, it was a parallel task all along. I think until we stop and take some use case, which we may discover we can do in two or three calls. It may not take – I am not trying to get it right. I am trying to get an example. If we can – if we start with that continuum framework that I put in the written thing and if we try to begin to get an example of a dataset at each of those levels, by the end of that call, we will have a few examples and we probably will have morphed the framework, itself, to
If we do that a couple of times – if as we do that, we discover key things, such as biases that would be in it or differences in privacy for a standard component, we can note that in the methods framework. We just would work back and forth between them and in a rapid, iterative, collaborative design process. That is what it would, in essence, be.
DR. COHEN: So the two parallel tasks that we are thinking about first are one and two, here? Is that – are you comfortable with that?
DR. STEAD: That would be – we don’t need to micromanage. That is it.
DR. COHEN: Why don’t we talk about each of them separately and go through a more clear definition of what those would be?
DR. FRANCIS: I am very concerned about framing one as the case for. That, I think, assumes that we are going to be positive on it. I would much rather see it as possibilities for. It is a neutral – what we are looking at are what are some of the ways that there might be convergence of these data streams? Then what we are going to be looking at are what are the good things we would get out of it? What are some of the things we would need to think about?
I would also say that when you think about the literature, there are going to be all kinds of – one sort of question that is out here is the use of clinical data for population health. A very different sort of question is the whole question of how do you even define mental health data? Is that something – is that a subset of clinic data? Is it something else? That is just, obviously, a huge open question that will require exploration here. I am not saying one way or the other, but mental health has just been a kind of flashpoint in certain respects. I want to make sure we pay special attention to that issue.
DR. COHEN: Any other comments? Larry?
DR. GREEN: What has worked well for us in the past is not literature reviews and meta-analyses. I think we are verifying one of Bill’s first comments about we keep having conversations today using words that there are about 18 different interpretations of what that word means. A way out of that for us, in the past, has been to formulate a parsimonious set of questions that we want to know the answers to about a piece of work that we are going to take on.
MS. GREENBERG: Speak up, Larry. You are not sitting next to me so it is harder to hear you.
DR. GREEN: For your rejection, Bruce and Bill, I think you might just re-label this literature review and case for and all that sort of stuff to just maybe get this small group together and their first task is to articulate a small, parsimonious set of questions that you say, given how much effort we are going to put into this, we have to check some stuff out here before we launch into it. We need some background stuff.
It doesn’t have to be the perfect answer to everything. It doesn’t have to be a publishable thing that is the last word about this. These are things that we should be able to start off our future with word set ups. When we decide to have a hearing and we are going to invite some people to come to us, we want background work done that has a believable evidence base statement of some sort that sets that up for us.
I think that is what I heard this morning. Again, I may be just the one person in the room that heard you setting it up that way. That is not exactly a literature review as much as it is a key preparatory step. We have defined those steps before by saying we want the answers to these four questions. If you need to go interview people, if you need to call somebody, if you need to check with WHO, chase that around, but come and give us an answer to these questions and we would like to have it by the next meeting. How does that hit you guys?
MS. KLOSS: I think we have just teased out three steps. I think it makes sense that that would be the first thing this workgroup would do is clarify what the work it was doing is. I think I also heard Bill say this morning that we need to make sure it hasn’t been done. We need to do enough of a review to see if there are any models that can be built on. It probably hasn’t been done completely or someone around the table would certainly know that, but pieces of it certainly will have been looked at.
MS. GREENBERG: Can I say something? That has to be the first task of this group. I don’t know if you want to call it a literature review or you want to call it an environmental scan, whatever. Even convergence, everyone is not using this the same way. I really, not surprisingly, I suppose, I am kind of seeing convergence as how you connect the dots, as opposed to merging or morphing. I think that is the way you see it.
Right away, if people start saying clinical data will replace survey data that is not what we are talking about, though, in some cases, it might. This data will be – we won’t need this data because we will have these other data. Sometimes that is the case, too, but it is more this idea of how you leverage the different data streams and how they relate to each other. I think even that kind of common understanding needs to be arrived at before you can even start doing some kind of environmental scan or asking someone to do it. Do you agree?
DR. STEAD: When Bruce and I started saying – I said the first question is do we actually know enough to charter the literature review? I view it as the lowest risk thing. I was going to err on the side of moving ahead. My sense is we have answer the question that we need to do a little work before we can charter.
I agree with Larry’s statement about what I mean by literature review or the
thing I want to solve with it. It is clear from this we also, just as we go
along, in addition to having one sheet that would be a data framework and
another a methods thing, another one just needs to be a list of terms, which
have very simple definitions like convergence, we are meaning how you link
DR. MAYS: I guess I want to kind of get back to what Leslie was raising. It
is a little bit of what Larry is saying as well, which really has to do with
kind of how much we are deciding down, as opposed to a little broader. My
concern is I am not sure we are lined up as well with the Secretary. One of the
things that always is very good for us is when we are really hitting something
that the Secretary really wants and needs at this time.
Again, I am going to kind of go back to – Marjorie, this is one of
these that we need Jim and Susan here or something. It really is when we think
about this notion of what is the top issues that the Secretary is struggling
with? It is almost like we are pushing them down, as opposed to pushing them
When we are talking about measurement and systems, it is almost like what
they are trying to do is break some of this down and be further into the
community and to ascertain whether or not what they have been doing is actually
the way to go. What I am more concerned about is the reifying that, you know,
the structures we have and the way that we are going and how to get that to
kind of come together.
The Secretary’s Office is actually trying to do I think somewhat a little
different task. I think one of the first things is aligning this with the new
directions and new things that they are trying to do within the Secretary’s
DR. GREEN: Let me react to that because that is in my wheelhouse here. From
what we know from talking with Susan Queen and Jim Scanlon and Marjorie
Greenberg and others, where we are with this is just fine. The feedback that we
have gotten so far is this looks like a productive enterprise from our
perspective. What I want to build off of what Vicki said, though, is one more
meeting from now when we get down in the weeds and start doing this, we better
check back in again because we might have taken a sharp turn in one direction
or another that starts getting out of alignment. For today’s discussion, I
think we should assume that you have run a very important flag up from where we
are for today. We should just keep going assuming they would be happy with it.
DR. COHEN: My response would be Jim mentioned some of the work that the Data
Council is doing. We need to really check in soon with them if we are pursuing
this avenue. It sounds like they are in a very similar space. We should really
collaborate and coordinate with their effort, rather than duplicate or going in
a different direction.
DR. GREEN: So make that one of the questions that we want an answer to. Is
someone already doing this? Are we duplicating someone’s other space, yes or
no? If the answer is yes, then god bless them and give us peace.
DR. STEAD: This is huge. It is bigger than a bread box. It will go on
forever. That said, if we do it – when it is systems engineering it is a
rapid cycle, spiral development process where you actually can use 0.001. You
can use version 0.002 to provide some organizing context for doing the kind of
things that were being talked about right after lunch and so forth so that it
is – we don’t want to put this work in the way of hitting important
What we want to do is start this work and use it along the way so that
increasingly, over time, we turn things we are doing against immediate
priorities into something that provides broader benefit than they otherwise
would have by themselves. That is really what it is about. That, in and of
itself, is a hard concept.
MS. GREENBERG: Regarding the data council, I haven’t gone – I used to
go religiously to their meetings. You can’t park down there. It is hard to get
into the building. I stopped going. Sometimes I listen on the phone. Of course,
I know what their agendas are and everything. You go pretty regularly. I don’t
really think this is what the Data Council is doing.
MS. BERNSTEIN: They are talking about mosaic effects today.
MS. GREENBERG: What?
MS. BERNSTEIN: Today, one of the agenda items is the mosaic – they are
having a presentation on the mosaic stuff that we have been talking about. They
do other stuff.
MS. GREENBERG: They take bites of this. That isn’t really what the Data
Council does, more of this kind of building conceptual frameworks and that kind
of thing. We obviously missed our opportunity – we missed a number of
opportunities during the furlough. The next Data Council meeting is December
11th, I assume. It is the second Wednesday of the month. I don’t
know whether you are available then, but I was thinking we could go for a
twofer of presenting that report, as we had intended to, unless they have
– often they are looking – I mean they have certain things that come
up that they have to do. They are often looking for topics.
If it were possible, also engage them on this thinking or this direction,
having held a few of the workgroup calls, hopefully, before that, in a very
broad way, not with anything definitive at all, and just see – in
conjunction with presenting the report, see what kind of reactions you get, if
there is interest, if there are things they are doing that are compatible or
maybe complementary. These are people representing – the data people
representing all the agencies. If they have ideas of things to put into this
environmental scan or the literature review or that type of this.
That, if we could do it at the December meeting, it would be very timely. We
could have one person on the phone and one person in person. I always think it
is better to at least have one person in person, one member in person.
DR. COHEN: I could be available to do it then if you want me to, Linda, it
is up to you.
MS. BERNSTEIN: I am not available.
DR. COHEN: My only thought about December is it is only three weeks away.
MS. GREENBERG: I know.
DR. COHEN: I would like us to have developed this a little more. Is it a
monthly meeting? I think January would make more sense. It would still be close
enough, but we will be able to develop our ideas a little better. That is just
my initial thought.
MS. BERNSTEIN: We can send a note to the staff who is developing the agenda
that you want to be on that agenda.
DR. COHEN: What date would that be in January?
MS. BERNSTEIN: Well, first Wednesday is the 1st. I am guessing it
is going to be the 15th. It says, actually, the 8th on my
calendar. It says the 8th on my calendar, even though the first is a
MS. GREENBERG: It is almost always the second Wednesday.
DR. COHEN: Is it morning? Afternoon?
MS. BERNSTEIN: It is one o’clock to three.
DR. COHEN: One to three? Yes. I could come and do that.
MS. GREENBERG: We have the presentation pretty much put together. Now, we
are talking about January. Can you do January the 8th?
MS. BERNSTEIN: Yes. That works.
MS. GREENBERG: We can work out who can do it. If it is not December
11th, it sounds like both of you. I am wondering if Bill might be
available on the phone for this second part of the discussion. It depends. You
all have to decide how you want to do it. I would certainly go for both. I
wouldn’t come back. If you want to do this scan – you were talking about
having something by the February meeting.
DR. STEAD: If we work through some number of calls, we should have an
example – we ought to be able to have something that is less abstract. We
can engage the committee in conversation and then decide how far we want to try
to get during the rest of the –
MS. GREENBERG: January is more realistic for that. Even then, that is early
January. I know people fall off after mid-December.
MS. BERNSTEIN: The point I was going to make is you were talking about how
to understand the priorities of the Secretary even when Jim and Susan are not
here. If you look on the front page of the HHS website, there are 14 priorities
that the Secretary has set, which are attached to the strategic plan for the
Department. I think they are a very good indicator of the direction that the
Department is trying to go or to focus at the moment.
If you can figure out how to work your agenda to dovetail with those thing
or some – not all of them, but some or one of those things, then you are
very likely to be going in the right direction. They are pretty broad
categories. It is insurance market place, Affordable Care Act, mental health,
stop bullying, be tobacco free, food safety, getting vaccinated, flu
prevention, raising healthier kids, which is Let’s Move, Michelle Obama’s
program, open government at HHS, stopping Medicare fraud, HHS digital strategy,
supporting military families, and fighting HIV/Aids.
Those are the things that the Department is focused on. If you are far
afield of those, then you should sort of check yourself, maybe, and find a way
to get closer to them. These are very broad areas. You can do a lot of work
that falls into those areas without being exactly in them.
DR. CORNELIUS: Wouldn’t it be apropos to focus on the Strategic Plan? The
stuff on the website is good. It is very boots on the ground.
MS. BERNSTEIN: They move together.
DR. CORNELIUS: The Strategic Plan is a five year vision for the Agency.
MS. BERSTEIN: Yes, and they go together with the priorities. Those are the
things that are driving the Strategic Plan usually. They are usually organized
DR. COHEN: I think it is really good advice. I don’t think we will have any
problem making this consistent with the strategic priorities of the Department.
I see this fitting in in a variety of ways.
DR. GREEN: The one set Jim passed out this morning is a bull’s eye for go
for Objective B.
DR. COHEN: So any more thoughts about this first activity, in terms of
understanding what is being done, what has been done, what we need to say or
the questions we need to ask around the convergence or I am calling it the
continuum of population health, mental health, and clinical data?
My comment is, Susan, if you are available, it would be great if you could
work on this activity. We should also try to get NCHS staff and I think ASPE
staff to focus on some of the stuff that already exists and help with this.
MS. GREENBERG: We need to scope out what the work is, the work of the whole
committee, and then figure out who can do what. There is someone on Jim’s staff
who might be able to help with this from what Susan and he said.
DR. COHEN: Let’s move on to this workgroup. Essentially, Bill is suggesting
the way for the workgroup to actually develop a framework is taking use case
and thinking about – I say data availability and data needs across the
continuum and figure out what that looks like and develop a framework from
there with all of the attributes of developing standards for standard data
definitions and protection, privacy, and stewardship for those data,
repurposing and using those data, and a variety of the kinds of issues that we
have been discussing already.
Is that a fair statement to what we are trying to do for this small group?
Okay. Volunteers to be part of this? Okay. I see Justine, Leslie, Bill, Jack.
Let’s see. We have Privacy. Standards should be represented. Anyone from
Standards? We will recruit somebody there.
DR. STEAD: Alex had said she would do it.
DR. COHEN: Vicki, I don’t want to put you on the spot, but I would love for
you to help us work through some of the issues around community data with
respect to this. Sorry. I withdraw my comment. I will email you later.
If there is anybody else who wants to be involved in this, we will set up a
fairly robust work schedule to interact pretty frequently to move this forward.
If we need support from staff, we will do some outreach for that.
MS. GREENBERG: We certainly need it to be staffed.
DR. COHEN: Yes. Who do you suggest?
MS. GREENBERG: We have to talk about that.
DR. COHEN: We have to talk about staffing.
MS. GREENBERG: There may even be a member of the public who wants to
participate. I don’t know if that person would be allowed to.
DR. STEAD: I would advocate for not getting beyond the people who have lived
through this discussion.
MS. GREENBERG: I was referring to myself.
MS. BERNSTEIN: What are we calling this?
DR. COHEN: Framework workgroup. I don’t know. Anybody have any suggestions?
Framework Group. There are other things we want to do, but if we can take a big
chunk of these two apples between now and February, we will be in great shape.
Any other thoughts or suggestions?
DR. CARR: I just want to go back to the plans that the workgroup is
developing, in terms of having a hearing around the social media examples, et
cetera. We were thinking about February as a timeframe. I feel like, in some
ways, they want to move along. I think there will be missed opportunities in a
way, in terms of framing.
DR. STEAD: I do not think there is any problem with them moving along. If we
start the calls, we will have some preliminary, some early drafts of frameworks. You are going to be part of those calls. You can then both use them
to tag and feedback.
DR. CARR: I agree. I just wanted to make sure.
DR. COHEN: Justine, if there is anybody from the Data Workgroup who wants to
be involved in this framing, that would be great, too. Maybe we can find out
DR. STEAD: We can figure out how to – the calls can say how to – they can work as another form of use case.
DR. COHEN: Great. Do I hear a motion of adjournment for the Population
MS. JACKSON: Clarification of the timing and scheduling of things and
clarification of previous working group clusters that Population had in 2013. I
am assuming that the work of these clusters, the technical and one of the other
groups, has kind of worked through its course. We had a dissemination group and
the technical group. Both of those seemed to have worked through what they
needed to to get us to this place. I am concerned about the membership kind of
dissipating and focusing.
If we are looking at a couple of calls, Nicole and I will set up – get
you a time period for something before Thanksgiving and then after the early
part of December. Those are the two live times that people are available. After
that, they are dead zoned. Just so you know if we don’t get something on board
at those time periods, then you are looking at the next year. I would like
clarification about the expectation for these other groups and products. I can
let you know about dissemination before we adjourn.
MS. GREENBERG: Yes. I agree. There is dissemination, but the Technical
Assistance group, I can see it picking up again. I don’t know whether it is
going to have a separate stream or will it kind of merge with this effort.
DR. COHEN: Integrate into this. Yes. That is a great question, Debbie. We
haven’t really thought it through. Maybe, Bill, you and I need to talk about it
offline and figure out how these activities will either continue in parallel or
integrate into this new framework. Thanks.
MS. JACKSON: For Dissemination, the subcommittee, for those that were
involved in the conference calls, were informed and also in the notes that we
did disseminate the report, the Summary Roundtable Report, to the major players
who were involved. It is on the web. Susan prepared a great transmittal for us
to push it out now to the full committee, which I had not had a chance to do.
That is on my task to get done by the end of this month, definitely before the
end of the year.
We can find out later on as to what kind of communication you want to this
body of people. Like the list, we have the NCVHS listserv. We can let them
know, generally, what you are doing and where you are going. It is not just a
push out of the report, but a push out of expectations for next steps. There is
some follow up I need to do with the co-Chairs just to see what their
communication would be.
DR. COHEN: Thanks. Any other last comments for the Populations Subcommittee?
I will turn it over to Privacy. Thanks.
(Whereupon, the meeting adjourned at 4:09 P.M.)