Transcript-Full Committee Meeting-May 16, 2018

Department of Health and Human Services

National Committee on

Vital and Health Statistics

Full Committee

May 16, 2018

Hubert H. Humphrey Building
200 Independence Ave., SW
Washington, D.C.

P R O C E E D I N G S 8:30 a.m.

Agenda Item: Welcome

STEAD: Good morning. Let us do a roll call of the members. I am Bill Stead, Vanderbilt University Medical Center, no conflicts. Chair of the Full Committee.
KLOSS: Linda Kloss, member of the Full Committee, co-chair of the Privacy, Confidentiality and Security Subcommittee, member of the Standards Subcommittee and no conflicts.
PHILLIPS: Bob Phillips, member of the Full Committee, co-chair of the Population Health Subcommittee. My only conflict is that I run a national registry.
COHEN: Bruce Cohen, member of the Full Committee, co-chair of the Population Health Subcommittee, no conflicts.
STRICKLAND: Deb Strickland, member of the Full Committee, member of the Standards Subcommittee, no conflicts.
LANDEN: Rich Landen, member of the Full Committee, Standards Subcommittee, no conflicts.
CORNELIUS: Lee Cornelius, member of the Full Committee and the Population Health Subcommittee. No conflicts.
MONSON: Jacki Monson, Sutter Health, member of the Full Committee, member of the Subcommittee Privacy, Confidentiality and Security. No conflicts.
THORPE: Roland Thorpe, Johns Hopkins Bloomberg School of Public Health, member of the Full Committee, member of the Population Health Subcommittee. No conflicts.
LOVE: Denise Love, National Association of Health Data Organizations and co-lead of the All-Payer Claims Database Council, Full Committee member, subcommittee Standards, no conflicts.
ROSS: Dave Ross, The Task Force for Global Health, Emory University, member of the Full Committee, member of the Population Health Subcommittee, no conflicts.
COUSSOULE: Nick Coussoule, BlueCross BlueShield of Tennessee, member of the Full Committee, co-chair of the Standards Subcommittee and member of the Privacy Confidentiality Security Subcommittee and I have no conflicts.
GOSS: Alix Goss, member of the Full Committee, co-chair of the Standards Subcommittee, I have no conflicts.
STEAD: We have a quorum. Could we now have introductions of staff, led by Rebecca.
HINES: Good morning, Rebecca Hines, Executive Secretary and Designated Federal Officer, and I am glad you all came back for day two.
JACKSON: Debbie Jackson, National Center for Health Statistics, CDC, Committee staff.
KANAAN: Susan Kanaan, writer for the Committee.

(Introductions around the room.)

MAYS: Vickie Mays, University of California Los Angeles, member of the Full Committee, Pop, Privacy and the Review Committee and I have no conflicts.

(Continued introductions around the room)

Agenda Item: Committee Updates

STEAD: Thank you. We now have a short time for updates that people have on things that they have done that intersect the committee. While you’re thinking about whether you have those things, just sort of to highlight the shift in how we’ve been working, which I think has been as being effective where we’re using these full committee meetings as places where we can actually do blocks of substantive work on our projects in addition to getting updates in the various coordination in general work plan kind of work.

I think this is really increasing our ability both to think as a full committee and to get tractions in ways that our dedicated separate hearings or workshops would be much more effective. It takes a lot of work on everybody’s part getting ready for these meetings but I think it’s really paying off and I simply need to thank you all for making that effort.

Bruce – well, we’re going to have Soma here later but do you want to say something about the Friday meeting or –

COHEN: Sure. I continue and several of us continue to be involved in measurement framework activities and I’m really excited that Soma is going to be here this afternoon to update us. Friday – she’ll describe in more detail what process we’ve been going through but the focus has been on developing measures to flesh out the domains and subdomains and it’s been a remarkable effort. She and her colleagues have pulled together an amazing group of folks involved in community health to come up with some really wonderful ideas of how to use the framework than move this forward.

The measurement framework project – we launched it and I think it’s going to be something that ends up being one of our finest sustainable products, mainly because it’s not just us. We’ve given it over to folks who are really very much connected to it and have adopted it and now are adapting it.

STEAD: As an example – if you have not seen the US News and World Reports’ Healthiest Communities, that has really all been built around the framework. And it’s an example of using a balanced set of domains for the framework for benchmarking where we’re also developing a much more robust set of measures that can be used in ways that call out the communities which is what the Hundred Million Healthier Lives effort is doing, to the degree I understand it.
COHEN: In fact, in the middle of June, I’ll be participating in the next – I think Bob and I will both be participating in the next round with US News and World Reports to further develop and dive more deeply using these data. So this is the beginning, I think, for them of an ongoing project that focuses – they started at the county level. They want to drill down now at the subcounty level which we always wanted to do and it will be interesting to recognize how to deal with the challenges of developing measures at the subcounty level. So their work is continuing as well.
HINES: Can I just add one thing? That that process that Soma is going to come this afternoon and describe for us actually is culminating in a meeting this Friday where the people involved all around the country have been invited to come to DC or call in remotely to continue reviewing the metrics that are in play or proposed metrics that don’t exist but people think would be really critical to have and move forward from there but it’s amazing that it’s been just over a year and so much progress has been made.

It’s a testament to how on-track this committee is that you identify this need and our charter says stimulate public-private partnership and that’s exactly what you did. It’s really the best-case scenario that you really could have imagined and it’s happening.

STEAD: Thank you. Other updates before I review the agenda?
GOSS: I wanted to give an update, part of the update, Denise is going to do the other half. Thanks to some connections with the Utah Health Information Network and their ongoing monitoring of the national landscape they reached out to us for some updates. So they reached out to Denise and we tag-teamed to cover their board of directors and their standards subcommittee discussions. Very good opportunity to provide education on our work plan where we’re focusing within the standards and engaged to further our collaborative work with them as an interested stakeholder.
LOVE: I was surprised how closely they are following the work. The board members of the Utah Health Information Network were probably more up to speed than I was on a few of the things and they really were engaged. So the work here is getting noticed and I was impressed.
ROSS: I just wanted to follow back about the community initiative and the metrics work. As I think I told you a good while back, I use that to engage leaders in Atlanta. There is now the Atlanta Regional Health Collaborative or maybe it’s called the Atlanta Regional Collaborative for Health Improvement. But it’s led to the point now where one of the county CEOs has actually come up with local money to go to the subcounty level with projects. We have collaborative teams now struggling with what area among the social determinants to select. But the engagement has happened and I think the framework really helped trigger people’s understanding of how to think about health broadly. So if nothing else, just having that framework put in motion a lot of local activity that I’m living with right. So thank you all.
STEAD: One other thing I forgot to mention. Aetna announced a challenge at Datapalooza that is in coordination with US News and World Reports and that is giving communities the opportunity to respond to the challenge with projects and get support.
KLOSS: At the invitation of Walter Suarez, our champion for life, I had an opportunity to present an overview of the beyond HIPAA environmental scan to the Confidentiality Coalition which is part of the Health Care Leadership Council and made up of all kinds of companies and major health systems and associations. So that was a good chance for overview of that work and I do think, as we talk about often, a lot of these work products just need to continue to get wider dissemination.

I think one of the thoughts I had actually is that that is such a wonderful cross-sectional group of people who are in this privacy-confidentiality world. I don’t know, are you involved in that at all Jacki?

MONSON: No.
KLOSS: That I think we could go back to them to float ideas or models or get additional input because it’s kind of a natural cross-section of the larger industry with very knowledgeable people.
STEAD: Then that might be one of the groups we might vet on the early versions that come out of yesterday’s conversation and the next.

Then let’s briefly just sort of look at the agenda. We’re next going to have a look into the new NCVHS website and the design work related to it and then we’re going to do a little bit of a deep dive back into health terminologies and vocabularies with our team from the NLM and Susan is going to join us, Susan Queen, to give us an update on the activities of the National Center and then we’re going to do a block on improving access to data.

Bob mentioned that he and Vickie might like a little more time if it frees up as we work our way through the agenda. So we’ll see if that turns out to be possible. And then lunch and then a status check-in on the new Medicare project. That may not take its full time so that might be a place that we get more time for data access. And then Soma is going to join us for the update on the measurement framework and then we will close with public comment. So that’s our plan for the day if that still works for people. Okay, very good.

Rebecca, would you like to introduce the website we’ve designed for discussion.

Agenda Item: NCVHS Website Redesign

HINES: Sure. I am sure you have all noticed as of this moment that I’ve been spending the last number of months with the contractor redesigning the website for two main reasons, one just for the overall look and feel but also to add functionality to do search – so if you want to search on various terms.

The other benefit of what the contractor’s been doing is doing some analytics. Who is using our website? What pages are getting looked at? Are there problems? So I welcome this morning your thoughts and feedback which we will take and then over the next couple of months make any other refinements on. So without further ado, Sam Thomas is the project manager for this effort that we’ve been working closely with over the spring, and Sam, take it away.

THOMAS: Thank you so much, Rebecca. I appreciate the introduction. I see we’re running about five minutes early and I understand there’s a desire to free up more time so I will try to help you do that. What I’d like to do is give a quick overview of the first quarter website traffic data that we have which kind of gives some context of what we’ve tried to do with the website redesign. Then I will, of course, answer any questions on that and then I will walk you through the main components of the website, how you can reach different types of content and use the various features. We will do that briefly and then open it up for discussion and questions about the website.

I did want to mention that each of you has two copies of the Google Analytics from the first quarter. I’ve seen some of you looking at both of them. They are the same. So they are just in a different format if one is easier to read.

So I wanted to start just by mentioning that this is a fairly highly trafficked website. There were about 6000 sessions in the first quarter of 2018 from more than 4000 different unique users. For now, I would ask that the new users that we can ignore that because we only started tracking the traffic in about December. So that measure will change over time and we’d be happy to provide further updates in the coming months.

We also have users primarily from United States, not surprisingly. And we’ve listed the top 10 metropolitan areas that have visited the website in this quarter as well. I did want to mention that not set is going to be a common location or feature throughout these analytics. That’s just simply the advent of things like that private browsing or incognito browsing depending on the web browser you use. If you’re interested in privacy because pretty much almost every website tracks information, that’s where that comes from.

I did also want to talk through the – or just take a quick look at the top pages that have been viewed because this really has guided the work for the redesign that we’ve done. So you can see the home page, not surprisingly, is the most commonly visited page.

Followed next by the full committee meeting agenda in January. Other than meeting pages, we see that a lot of visitors are using the reports, recommendations, reports and presentations looking for various types of content from the committee and its subcommittees. And there have been some visits as well to historical materials from the 50^th and 60^th anniversaries as well as the transcripts and minutes from committee meetings and information on the membership of this committee.

I did also want to point out that the last item that made it into the top 10-page views unfortunately was a 404 page not found error. The reason that this came here – it’s actually a link from a third-party website, icd10monitor.com, so this is not an NCVHS page. But what happened is there was an article on that page that used an incorrect URL to refer to the slides from the January meeting.

But what we’ve done now that we’re tracking these analytics is when we notice something like this even though it’s not a mistake on the NCVHS website, it’s a referring website, we’re able to grab this URL and redirect sot that users can get to where they are going. So if you go to this article now on this third party website and you click on the link, it will take you to the slides.

Similarly, we have a list of the top 10 most downloaded files. The printable Acrobat agenda was the agenda for the January full committee meeting. The very clearly labeled National Committee on Vital and Health Statistics, this was actually the strategic plan. And one thing that we’re working on, currently, now that we have launched the new website is we are designing a set of file naming conventions for how files will appear on the website which should make looking at information like much more transparent. I think it will also make the user experience easier as you look for various content.

I don’t think that we need to go through the rest of this in great detail but I should mention that we do have information about what types of devices users use to find content and where traffic is being directed from. Most of our users are coming from a desktop computer. The new website is a responsive design and it should look good no matter its screen size that you used to access it.

So before we jump over to the website, are there any questions about the usage statistics, which again, are from the previous website from the first quarter of this year.

MAYS: Thanks for the report. One of the questions is can you distinguish when we come in from the rest of this because we come in kind of frequently and it would be great to be able to know it’s not us.

And then the other thing is do you have any ways in which you can have – there’s ways in which you can have – there’s often a little popup and you can ask people if they were satisfied or if they can give you feedback so that we might be able to get a sense of the things that people were looking for and couldn’t find, or something that they want so that we can do continual improvement.

THOMAS: Great questions. For the first question, figuring out who the users are. This is a little bit tricky to do and I would say that with precision, you cannot identify the individuals. But, what we can do is, first of all, we can look metro area and we can dive down to a closer geographic layer. That will not give you perfect resolution but, for example, when we see that Hyattsville as a town pop up, we can infer that that is probably NCH staff.

Similarly, I think a lot of it hits from Atlanta or from CDC officers. If users join the site from a major corporation where they have a dedicated IP address then yes, we can say, this is an employee of Aetna or whichever company it if. The caveat is that we access this site regularly but we do so from many different locations.

So we do pick up the IP address from the computer that we’re hitting but if you do this from home or on-the-go or you’re using private browsing or a feature like that to kind of mask your identity, the short answer is not with precision, no. But we do have some aggregate ways that we can at least roughly say what proportion might be internal users – it’s not going to be that precise though.

To your second question about whether we can put up some kind of survey or satisfaction questionnaire, that was not within the scope of the website redesign for our contract but what we do have – and you’ll see this is in multiple places on the site, one at the footer of every page and also at the top there’s a link to a contact form where users could voluntarily elect to submit information. It’s not a popup that’s going to interrupt the user experience so of course you’re only going to get users who proactively decide to go ahead and do that. But another thing that I would say is if the committee wanted we certainly could send out emails to various end users announcing that the website has been redesigned and asking for them to fill out those surveys. So I would leave it to Rebecca to determine what the best way to go about doing that would be.

So with that we will go ahead and look at the website itself. So what we can see here right away is that this actually is not a standard screen size that we’ve got here. Typically there would be menu headers across the screen. Those still are here under this menu icon. So again the website, depending on what size screen you use to access the website, it determines how best to display the content.

I did want to just come through the home page. So you scroll down. There is a paragraph about the committee with a link to more information about the committee. And there is also this circular icon which gives very brief snippets about each of the subcommittees and a link to the page about the subcommittee.

Directly under that we have presented the content that we see as most frequently used. So that would be first and foremost the committee meetings – and we’ll look at some of these pages but each of them will present the dial-in information for the meetings if it’s the day or shortly before the meeting. We’ll also have the agenda posted and links to any other items that are related to the meeting such as written testimonies or presentations.

We also have links to past meetings which we have highlighted a little bit less prominently than the upcoming meetings which have these calendar icons next to them. And finally we have the most recent reports and letters of recommendation to the Secretary, for example. Coming down to the bottom of the site we have some basic contact information and, as I mentioned, the form that you can use to submit feedback about the website or questions of any kind.

HINES: And if you are wondering where those go, they come right into our staff mailbox so we will get them.
THOMAS: What I would like to do is just briefly walk you through each of these main page types and then we will be happy to answer any questions. So the first main page after the home page is the committee organization where again we list brief information about each of the committees. You can go to the full page for each of these committees. For the full committee what we have is a link to the committee charter and then each committee page has got these unfolding sections for key content related to the committee.

So if you click on recommendations, reports and presentations, it shows you all of the recommendations, reports and presentations that are associated with the full committee from the most recent reports to the oldest. Similarly, you can view the upcoming and most recent past meetings for the full committee.

Planning documents, currently we have just the strategic plan. This is the only document that we have received for posting as of yet. However, I understand that there may also be work plans and project scoping documents which, as they are developed, will be added to the website. We’ve also set this up so that the currently used planning documents are separated from any potential past documents which we would want to keep for historic purposes but make clear to our users are not active planning documents.

We’ve also got the transcripts and minutes –

ROSS: So what is the cutoff of the archive? Is it within two years or –
HINES: Right now, we have no cutoff. So for instance, if we have a scoping document for the predictability roadmap and let’s say eight months from now you update it, we would put the old one under inactive or maybe we should change that to archive and then have the new one under active. And so we can just keep them. We don’t have old scoping documents so there’s no old ones really plugged in here. If you all think about some old documents you would like to have archived by each subcommittee, we can certainly add them.

So this is a good time to think about what would you like to see here that’s not here that we didn’t pull from the old site. So it’s a good time to think about that because we can set that up. It’s not that hard.

THOMAS: Thank you for that. As I mentioned, we also have transcripts and minutes. I’ll show you that for content that we do have we go back quite a ways in time and you can scroll down here – it’s really quite a long list? We’ve got documents dating back to the 1990s. I will show you in a minute there is a search feature that can help users find information from the era that they are looking for.

We’ve also got a list of the members and staff for the committee. And there’s a link at the bottom for fuller information and we’ll walk through that as well in a moment.

The subcommittee pages look much the same as the full committee page. They begin with the charge for the subcommittee. And they have the same unfolding components here with the key difference being that any report or any other content that appears under these is just the content that’s associated with that particular committee. So you can see, for example, for this subcommittee there are no currently scheduled upcoming meetings but we do have a list of the past meetings.

HINES: So that is not right. The July Roundtable, for instance, should be associated with that and that’s my error in classifying the agenda. So you all know, we’ve been working – Debbie and I have been working tagging or classifying everything in to huge sort of classification arrangement, with Bill’s help, and so every document has to be associated with the correct subcommittee so that it appears when you click on upcoming meeting, unless we tell it, obviously the database doesn’t know that the July Roundtable is –
KLOSS: And it wouldn’t be part of privacy –
HINES: Sorry, that’s my error. Standards. So, at any rate, that’s a new business process that we’re, as a team, going to have to undergo when we upload things to the website.
KLOSS: But I don’t have to know what subcommittee to find the document. I can just go to reports, correct?
THOMAS: Correct. So really the major innovation that we have with this new site compared to the old site is that the old site, each piece of content was posted manually in each different location and it was a bit tricky to sometimes find items going through one avenue versus another. So we anticipate that end users may look at the home page just for very quick information. They may look by committee if that’s their interest. They may wish to look at all of the committees together on the meetings page and the reports page.

And as you’ll see, all of the content is just updated in a single place and is categorized by these various components and so the same information will come up in multiple places and our goal is just to make life easier for the end user to find what they are looking for. And so yes, this is now the standard subcommittee page and we do see that we’ve got tomorrow’s CIO form listed here as well as the Roundtable coming up in July.

So moving through to the membership pages, the main membership page is for the full committee and here we list full details for each of the members as we have had on the past site. And this is another area where I think that many of you will appreciate. I know that many of you – I think most, if not all of you are also on one of the subcommittees and when we’ve had information such as your current job change or your contact information, there have sometimes been issues where that information was updated maybe on the full committee information but not on one of the subcommittees so we’ve now got that tied together here. So, for example, we can hop over to one of the subcommittee pages and we’ll see the chair of that subcommittee, the members of the subcommittee, again with their full details and the staff who serve that committee.

I would actually like to save the reports page for the end. The about page and its subpages contain general overview information about the committee and links down again to the membership pages and the meetings pages. We also have preserved content from the old website where we’ve got, of course, the committee’s charter, most recent years of accomplishments and the 50^th and 60^th anniversary pages which remain intact from how they were previously.

The meetings page is really intended for users who are interested in all of the upcoming and past meetings, regardless of the committee or subcommittee. So again, just like on the home page, we have highlighted, we think very clearly, the upcoming meetings. And if you click through, for example, to today’s meeting, what you’ll receive is the dial-in information for those who are connecting remotely, followed by the agenda for the meeting.

COHEN: Sam, can I find the schedule for next year there?
THOMAS: You can find the schedule once we have that codified. But yes, this page will have all of the currently scheduled upcoming meetings.
HINES: We need Lorraine’s slide with the guy sort of divining –
THOMAS: What we do have is also all of the past meetings here. Again, of course this is a very long list so we’ve actually separated out to a second page if you really want to go through all of them in chronological order. But I did want to mention that if we click on, for example, this March 26^th Hearing on NCPDP Standards – so what we’ll have is the final agenda from that day. And on the right side, we’ll have any related items. So this particular hearing had many submitted written testimonies and so we’ve worked to be able to link those directly to the meeting page so that users can find these easily.

In the past on the old website, these were embedded as hyperlinks in the final agenda. So we no longer intend to do it that way. We’ll be linking them as related items on the side which will allow them to be discoverable in the search engine in other parts of the website as well.

MAYS: When I put NCVHS into Google up came several NCVHS things but each one I clicked on it said, oops, page not found. Because I was trying to do a shortcut to get to – I was trying to catch up with you. Usually if I just put in NCVHS then that will get me to the site and then I know the site well enough. But I’ve just done it and –
STRICKLAND: That was the first page. If you pick any of the topics on the top it will go to those.
MAYS: No, I just put in NCVHS and all of the things that I had –
STRICKLAND: But if you click on any of the ones on the top, it’ll bring you to those. So it’s just the first page –
MAYS: But you don’t know to do that.
THOMAS: Great comments.
MAYS: Can I finish because there were some other things – so then when I follow – because I was trying to catch up with you for members. And so I put NCVHS and members in, I come to a page that says the full committee – and remember, I’m looking for members – and then when I go down and I click members, it’s not the place where I have the information so I have to click again. What I get coming up says members and staff but not the information about the members and staff. So if I were looking for members, I’d want to know the information but then I have to go down to the bottom and then it tells me, please visit the full committee membership page to view the full details. And the page I’m on I can’t get anything about members other than the name.
THOMAS: So the committee pages have just the names as opposed to the full details. And that decision was made because the thought was that folks who are looking at a committee page probably are looking at the general content on that page, including the members but not just looking at –
MAYS: My search was for NCVHS and members – that’s what I’m saying – I wanted the members.
THOMAS: Let me address the larger comment about the broken links and Google search in particular. Two things on that. For the broken links we have extensively catalogued a list of current broken links. We are aware that there are still a few. Having just launched this site last week, we are working to make sure that those line up to the correct places. A lot of the URLs from the old site are not mapped directly to the new site. We have solved the great majority of those but we are aware that there are still a few remaining.

The second part will be about Google or any other search engines. The way that those work is that they must index webpages first and there is some lead time to when Google will have fully indexed the new website. And so that may take some time, a week or a few weeks, to catalogue and make sure that the current versions of those links are appearing first. There are a few steps that we can take to try to accelerate Google’s indexing process but of course we are not Google. So we’ll do our best to resolve that as soon as we can.

KLOSS: I think this looks fabulous and this has been long awaited. This is terrific. I went on last week right after I got the new link and I tried to download the agenda for this meeting. And I didn’t get a clean PDF. It was garbled. And perhaps you’ve fixed that. I brought the copy just in case you want to see it. But I wasn’t sure the PDF was working as well as –
THOMAS: Good question. We are aware of that issue. What we have done is we’ve replaced that PDF with a direct URL that takes you to the formatted version.
HINES: Any other questions before we go to the – Rich.
LANDEN: A couple of questions. What’s our standard practice for posting materials to the website prior to the meeting? We’ve got a pretty robust set of what we gave out, the presentations and whatnot afterwards but before the fact, specifically is the eAgenda book available to the public and what about presentations –
HINES: So I actually go to each co-chair set and say, here’s what I have for the eAgenda book. What of these would you like for the public? And I let them decide.
LANDEN: So we do have a standard?
HINES: Absolutely. And for this meeting, the decision was either – basically, nobody wanted anything pre-posted. I got no instruction from any co-chair saying please pre-post this other than the agenda. They wanted to wait for the letters to be final, which I agreed with. We could have put the draft report up but we would have had to put a – for the vitals but we would have had to – we weren’t going to spend days making something 5OA compliant that wasn’t final. So we decided not to put that up until it was final.

So there is some resource issue where we don’t have endless resources to make things 50A compliant and that’s one of their roles. And so for letters and things we could – but again, the process is we ask the co-chairs, do you want your materials put out in advance of the meeting and for this one we decided having them for the live broadcast was sufficient and then afterwards, when things are final, obviously they’ll go up.

LANDEN: Okay, thanks. Second question is under the subcommittees, the section on recommendations, reports and presentations, if you could open that up please. All the reports are just simply listed in date order. Is there some way to give some context to that, like all the dates there – what was it on December 13^th, was it a – this is not a good example. Maybe if you go to the standards subcommittee. Everything is just listed March 26^th. And then written testimony by an individual. Is there some way to give context to that? What was it on March 26^th that all these testimonies related to?
GOSS: I have to agree with Rich on this one. It’s also – the presentations are great but the recommendations and reports are the things we really want to bubble up and then have the ability to get to the testimony that was the fodder we used to get to those recommendations available. Also people I think are going to struggle a little bit with the fact that we used to be able to go to a hearing, link to the testimony for each one of those as a discreet item within that agenda. And I get the Google dynamic or the search dynamic you mentioned but in addition to clarity about what topic or why this was presented for which topic coupled with the ability to kind of get an aggregated view would be helpful – of all the testimony for a given topic.
HINES: That is really helpful. And one of the things I think I’ll be consulting with you all on is the naming convention for files because part of this is we came up with this and its obviously not quite right.

The other thing is the filter for this list probably shouldn’t have testimony in it because that’s not what you want to see, you want to see the agenda.

GOSS: People do want to look at the testimony, at some point be able – that’s why my second part of the question is how do you link to it? We have a couple end users in the audience who might have some comments so you might want to open it up for public comment before we – if that is possible.
HINES: So this is really good. So Sam, we’re going to have to look at how we filter the subcommittee recommendations, reports and presentation page to see what we can do around that and then also the naming conventions for testimony.
STEAD: But I think recommendations and reports are mature and summative and approved and so they are very different from presentations or testimony or transcripts. And so I think the idea of having those meeting materials, that set, all of those meeting materials in one category that’s different from recommendations and reports might be helpful.
HINES: Okay.
COUSSOULLE: I was going to suggest the same thing because oftentimes – I don’t want to speak for everybody else here but we go in and look for a testimony on a regular basis to refresh and go through what we’ve heard and seen. But that’s very different than looking for final work products that have been delivered. So being able to separate those two out would be very helpful.
THORPE: I think what would be helpful for me is to see the evolution of the workflow, coming from the hearing, the presentation, and what came out of that. So if it’s a testimony that came out of the presentation or the hearing, I think for me if I was to go look for it, I’d want to see the evolution of the work and how it got to be.
THOMAS: These are excellent comments. I echo Rebecca’s thought that there are a couple of things that we can do in the near term. One would be the naming conventions for the files. The other is the filtering.

So in a moment I’ll jump over to the reports search feature and you’ll be able to tell from that very easily – first of all, I think this will answer some of these needs. And second of all, you’ll be able to tell that we do in the back end of the database already have these different content types categorized separately and it would be very easy for us to output them in different ways across the site.

So, for example, we’ve got recommendations, reports and presentations all together. The reason for this is that we were beginning with the old site and that’s how they were over the. What I’m hearing is that presentations and testimonies really don’t belong in that same category. So as you can see, we’ve got transcripts and minutes pulled out separately. Similarly, it would be straightforward for us to pull out the testimonies and the presentations because those are already labeled as such in the database. So it would be very straightforward again for us to have this just be the recommendations and reports, those final work products here and we could have another unfolding section, for example, for testimonies.

HINES: Bill reminded me that I think what we had talked about a few months ago would make sense, is to move presentations down to transcripts, minutes and presentations because those are all work product related to a live meeting. So that would be an easy way, just basically saying meeting stuff – however we want to classify that.
STEAD: Work in progress.
HINES: Whereas like they are saying recommendations and reports are final products so that will be a good improvement.
THOMAS: I would like to walk through in our remaining minutes here the reports page which really is a search engine, not just for reports but it covers all these other content types that we’ve been discussing. So when you land on this page, the default shows you – actually, just like we’ve been discussing, just those really final work products. So these are the recommendation letters to the Secretary and mandated and topical reports from the committee and its subcommittees.

On the left side of the page, and I realize that this projector, the resolution is not great with the gray on the background but what we have here is a free text search box and then filters to restrict your search by the various committees and we have included all of the historical workgroups and subcommittees that are no longer active. Again, all of the reports, letters and presentations – so you can see that we’ve got all of these categorized, for example, historical materials, letter, general type of letter, mandated reports, presentations, recommendation letter to the Secretary, topical reports, testimony, et cetera.

And we’ve also got transcripts and minutes and those other various meeting-related content types and the planning documents as well. At the bottom there’s also an option to restrict the search to a particular date range with a start and a stop date.

So, just for example, if I were interested in looking at any material on the website that contains the word HIPAA, you just simply type that into the box. You come back with 500 results. I should mention also that the way we’ve got the search engine set up, this is searching the full text of all of the PDF documents as well. So you can really use this as a search tool to get to the content inside the reports.

COUSSOULE: Can I ask one question? Actually this is really helpful but how are the results presented? So right now I know it doesn’t come out in a date sequence so I’m not sure. There’s some relevance measure I’m sure that the search algorithm figures out. But it probably would be helpful to have an ability to sort that in a certain way, if nothing else than by date, just to see the most relevant or the most recent documents first.
THOMAS: Great point. I would need to check on exactly what the algorithm is in terms of what comes up on the top but certainly we can look into adding the filters. I think that’s a great comment.

I did want to – before we open up for questions – just to demonstrate that if, for example, we really only wanted to see HIPAA that was in a topical report –

HINES: Or a letter to the Secretary.

MR THOMAS: Or a letter to the Secretary, you can go ahead and rerun that search with those filters and you see there’s a much smaller number of items here. So I think that for those users who say I’m really interested in testimonies that mentioned whatever the term is, you can come to this report search page – and perhaps reports is not the best title for this page – but you can come to this search page and just select testimonies, put in your search terms and all of that content will come out. So I think that this page probably for internal users and advanced users who are intimately familiar with the committee and know what they are looking for, this perhaps is going to be the most highly used part of the site.

So that’s it for me but I would like to answer any other questions or encourage discussion that we can use to improve the website.

KLOSS: You just touched on a question that I had on the home page where we – it’s title reports. And we’ve always thought of reports differently than letters to the Secretary. And I thought we might want to discuss what that should be labeled as because I think reports is narrow.
HINES: Because in fact, it includes everything. Committee work product.
STEAD: I don’t know where this is in this. Rebecca and I worked through a sort of high level metadata tagging system that in essence tried to identify the key types of our products. And this structure and that tagging relate but are not identical. And I don’t know if that tagging is somewhere where we could show the committee so they can see what we were thinking about as the granular level because if we get the granular level right then we can figure out how we aggregate it. And it also ought to be searchable at that level.
HINES: Basically everything that was on the old website, whether it was a final report, a letter to the Secretary, an other letter, meeting agenda, transcript, minutes, presentations are all in this page that’s right now, as Linda points out, mislabeled reports. So the question is there’s not a lot of room across the top here if you have the screen open on sized properly but I do think this is an excellent point, that reports doesn’t really do justice to what’s available on this page. So maybe reports and work product, I don’t know. I’m not great at this so if someone can come up with a label for that –
STEAD: What I would like you to do is share the granular taxonomy we came up with and then see if people can help – one, say is that right or not, because it’s relatively short and it’s not built into a structure so people can process it. And then the question is which things we want to put in the buckets of the structure of the webpage. Those are really two separate questions. I think we will make more progress if we work up from the granular because it’s not enormous. It’s one page so it’s not a –
THOMAS: I will say that we did use that document for this and so I believe that this does capture those granular buckets. We did have to add a few because as we went through our exercise to actually categorize all of the documents, we did come up with some new categories that were not represented there but certainly we can continue to add new categories or make a different decision about how to aggregate them up into the top level categories.
KLOSS: What everything on that list has in common is that they are approved by the committee.
HINES: That was by design because that’s how the current website is.
KLOSS: But I’m saying maybe that needs to be reflected in how we title it.
HINES: Just so you all know, the current configuration, the filter, if you will, the default when you go to the reports page is the same default that was on the old site. So basically letters and reports are what come up because it seemed like there was so much, for lack of a better word, guff, stuff, testimony that was getting mixed in and I just thought that’s not what you want to see in a default page. You want to see the reports and the letters that are all approved and then if you want more you can then go start searching. But we can certainly rethink that.
COHEN: I think that is fine. The confusion is calling those reports and calling the ubiquitous search engine reports. So it’s a labeling issue as much as anything else.
HINES: It is, and anyone who has the answer send it and we’ll change it.
MAYS: I think that once we are finished and probably even now in the process, it’s going to make a big difference so I appreciate the work that went in. Because I think in terms of being able to push our things out, that’s the new thing we really want to do is to not just talk to us and our friends who are the ones that showed up for the hearings but really to talk to a broader audience and I think that this will just be really good to be able to get it out. So I appreciate that because we work hard.
HINES: Lorraine, you are on the phone and I see you sent a message. Is there anything you wanted to ask or add? Did anyone else have any comments? Feel free to just weigh in while we’re in this working session trying to let us know what would be helpful.
MONSON: I wanted to go back to the page where you would have like a previous meeting and how the testimony that was given in the previous meeting is all listed on the right-hand side, I’m looking at the March 26^th. Personally, I’m used to the fact that the links to the testimony were right in the agenda. And I think it gets confusing to see the testimony separated because they are not ordered in the way – by the testifier, so you’re bouncing around trying to find like Jean Marcisi testified and then you’ve got to search through the list to look for specifically her statement or was there a statement.

So if it would be possible to go back to having the link right in the agenda, that would be helpful. And then maybe perhaps I know on some of the bigger hearings you might not have someone who testified but they submit written testimony then that would go on the side as related item. So just a preference because I preferred the way it had been where it was right there in the agenda and you could see.

COHEN: Can you do both? Just put the hyperlinks right in the final agenda document so if people wanted to search via the agenda they could find out where it is and then they could move over and just download the PDF.
GOSS: I am starting to wonder if there’s a maintenance dynamic with support on this which is I really resonate with Nancy’s comments – just because I’ve been looking at these pages for years but it does – so you’d really have to be very crafty in trying to figure out that okay, you look for the who was in which session. Then you’d have to go find each of their agendas on the right-hand side. You can do it. It’s just a multistep process. But I also appreciate that just embedding links you get it either way, it’s not a just.
THOMAS: These are all great comments and Alix, thank you, that’s basically the two-part answer here to why we have done it this way is one, because these documents that are on the side are now also discoverable in the search tool labeled as a testimony. And two, as a maintenance and resource concern about maintaining all of the individual links as well.

To the question from over here, technically yes, certainly we could also embed the links.

GOSS: I also think that your earlier comment about your naming convention can help address this especially if you link it by panel, that’s a happy compromise. So that way you could filter either alphabetically or – so that you could then get like panel one, panel two, panel three. It sort of is the stop-gap to make it easier for the end user to look at if you’re particularly interested in one topic of a hearing then they would be grouped by the naming convention.
LANDEN: Nancy’s comment resonates with me also. I also notice on the final agenda the way we have the session listed is not the same naming convention as in the list at the right. So it’s work to figure out which of the documents on the right actually belong to the, like in this case, pick on Margaret, Margaret Weiker, NCPDP, how do you find that on the right side. It’s not always – it may be Margaret, it might be in NCPDP, it might be something else.
THOMAS: Great point. So I think that we’ll extend the naming conventions not just to the file names but also to how agendas are written.
DE CARLO: Michael DeCarlo with the Blue Cross Blue Shield Association. I just wanted to say I’ve been attending NCVHS meetings since 1992. At the time I was representing nursing homes and went on to represent DME. I was in private practice for a while and I’ve been with the Blue Cross Association for the last 15 years and part of my job description is to come to these meetings and monitor what you guys are doing and report that back.

I remember the days before there was a website. I used to come with a thumb drive and coerce the staff into giving me a download of your presentations so I’d be able to take those back and putting them in my memos for my members. As a matter of fact, Linda called me once because I forgot to pick up a thumb drive from her staff that was left behind and she read me the riot act for not delivering it.

The website is a very useful tool. As a matter of act, Blue Cross Blue Shield Association and our member plans come and present testimony to your committees and hearings. I go to your website to find copies of those because your system is better than mine in terms of archival.

So please keep as much information as you can on this website. It is a research tool. I go on there almost every week, if not almost every day. So thank you. And this looks like a good update, it’s a lot cleaner but I echo the comment that you just heard that there has to be some linkage between the agendas and the materials. So thank you.

THOMAS: Thank you all for the feedback.
HINES: Thank you Sam. I just want to thank Sam and his team and Debbie for all of the hours of deliberation and work to get us to this point. So thank you. And also a mention to Rich and Vickie and Lorraine and Bill for doing an ad hoc look about six weeks ago, that really, as you probably could tell, got us to this point. We really heard all of your comments.
STEAD: I want to thank Rebecca because she’s taken this with passion. Okay, that then gets us ready for our next dive into health terminologies and vocabularies.
THORPE: I noticed this morning when we went over the agenda, maybe I missed it – at any point are we going to return to the letter that we were supposed to amend yesterday. At what point are we going to do that?
STEAD: We returned to that yesterday in the middle, while we were waiting for Genevieve.

Agenda Item: Health Terminologies and Vocabularies

KLOSS: I want to start by thanking Suzy and Vivian for the yeoman’s work that they’ve done in getting our environmental scan to the point where we can have a preliminary discussion and get your feedback on it today.

Just to again get us into context into the health terminology and vocabulary gestalt, let’s just review what it is we’re doing in this space. Just recall once again that two of our charges are to study issues related to the adoption of uniform data standards for patient medical record information and the electronic exchange of that information. And also to advise the department on health data collection needs and strategies, review and monitor the department’s data and information systems, to identify needs, opportunities and problems. And this is an important area of our charge that I think as we noted earlier we’ve executed by monitoring the status of ICD-10 and the implementation process of ICD-10 but we haven’t taken a deep dive into terminologies and vocabularies for some years.

So our initiative is again to study the changing environment, implications specifically for timing and approach to health terminology and vocabulary adoption standards. Needs, opportunities and problems with develop, dissemination, maintenance and adoption. I’m underscoring those words because we’ll come back to it when we look at the sections and how we’ve organized the outline of the environmental scan and how we’ve put the Roundtable agenda together.

In other words, we’re not looking at the vocabulary rubric, internal rubric of what naming conventions are used in the vocabulary itself but to look at the processes relating to develop, dissemination, maintenance, adoption, governance, coordination, other related issues with the goal of discerning actions, short-term and perhaps longer term we don’t know yet what form those will take but that might improve those same development, dissemination, maintenance, coordination, governance and adoption and use of standards in this area.

So what we’ve accomplished to date – this is kind of midway through year two. About a year ago we launched this. We had a briefing on SNOMED-CT, ICD, RxNorm, LOINC, ICPC. That’s when Vivian and Suzy introduced us to both SNOMED-CT and its current status as well as UMLS and we engaged more closely with NLM to help us with this project in a more direct way.

In September we looked in an overview of named standards and we framed some issues for the initiative. In January we took a deeper dive into the design of ICD-11 and what’s missing from the slide is NUMLs. We filled in the gaps in our understanding.

And where we’re at today is to look at a draft of the environmental scan report and the design of the July Roundtable. So that’s really where we’re at and where we think we’ll be going next in the shaded area. We will host the Roundtable and then we believe it will be time, based on Roundtable, what we learn there, to go back and update the project scoping document. We decided not to do that before the Roundtable.

We intend to bring the environmental scan to the September meeting for approval. So we’ll go from draft state and we’ll talk a little bit more about that process in a moment, but look forward to approving that document which will be even longer by then. So I think we’ll have another long eAgenda book for September. And then we’ll begin drafting, we hope, a report and a letter to the Secretary, again looking out to 2019, subject to change. We would presume to have a preliminary discussion at the first meeting of 2019 about that letter and report and with luck have final approval a year from now. So we began a year and this will be a two-year project. We’re at a really important midpoint.

STEAD: Not to be my normal challenging self – I would hope that we would have the themes, if you will, that might go into the letter by the September meeting so we can begin to discuss the themes at the September meeting and then work the letter over the fall and hopefully finalize it in January if practical just because I’m trying to align with the preparation of the report to Congress.

I think that there will – my guess is that when we update the scoping document after the Roundtable, we will begin to be getting our heads around the 2019 calendar work in this space. And that probably will take all of calendar 2019 and might be a letter actually in September of 2019 or possibly even later. So this is notionally correct but to the degree we could pull out a piece that would be helpful I think given the activity a discussion around things like the data classes for US CDI and given the long lead time I think around ICD I think to begin to get something directionally would be helpful, if it proves practical.

KLOSS: Thank you. Any other questions on sort of where we’ve come and where we’re at today? Our goal for today then is to look next at the environmental scan and get any comments you might have and then just discuss several themes. So perhaps Suzy or Vivian could describe how we settled on this outline and what this process has been like to pull together what we’ve pulled together.
ROY: We have been working with the subcommittee very closely so thank you. And we primarily used a lot of Linda’s original report from last year and we used that as our guideline. And what we’re really trying to do is look at the current standards, as Linda was saying. We’re looking at the current, primarily the adopted standards so those that are already named in Meaningful Use as well as other regulations. We’re staying a bit away from more of the transactional standards so that’s why there is a heavy focus on SNOMED-CT, LOINC, RxNorm and others.

What we’ve primarily done in this first draft was go through and taken a deep dive on these various standards. So as you probably noticed that it is quite a long list of standards and what we were hoping to find out today are there any additional terminologies, vocabularies, ontology standards that you would like us to include in the draft. Are there any that you think that we’ve gone beyond the original scope and should not be named?

And that way we can start to get those information into the environmental scan and then after, what we are planning on doing over the next month is looking at the issues surrounding these named standards as well as the unnamed standards. What are some of the gaps in the current standards and in the regulations and taking the deep dive into that so that it can start to frame what they will discuss during the working group at the Roundtables. Any comments or any questions?

COHEN: I think this is a great draft and I focused on actually page 141, the known gaps. And my interest was in thinking about how clearly you identify social determinants of health in population health. There’s a category of things that appear in clinical records in the courts that I don’t consider exactly social determinants but I think are aching for standard development.

When somebody comes to a hospital and they ask what’s your name, a standard for names. People can say Joe Johnson, Jack Johnson, is there any particular standard for including basic personal information or family history? It’s not exactly social determinants in the classic sense but it’s personal information, personal behavior and personal history that describes individuals.

And I don’t know whether there are standards. I’m not familiar with any requirements for standards for inclusion of that kind of information and I think that would be a real interest to review to see if those kinds of standards exist and where they might be and how they could be coordinated and developed.

KLOSS: I think I will take that and then Alix and Nick can chime in. These are ADT kind of standards, admission, discharge, demographic data, are not part of the clinical vocabulary and terminologies. They are part of the transaction.
STEAD: So you think these are out of scope for this effort.
KLOSS: I do believe that they are certainly part of transaction standards but naming conventions, some of the demographics are more determined by the design of EHRs and individual – I just think they are out of scope. Don’t you?
STEAD: I do. So if we do not have words to record the family in like cancer than that would be a terminology gap. If we don’t have a standard for how you assemble a set of codes or words to include in something that’s a standard packet and that’s more of a standard. A different kind of messaging standard or a structural standard. So we’re really – and the terminologies have become knowledge bases or databases where they are not just lists, they tend to show the relationship amongst terms. And the words and the relationships and the code sets that are designed to capture medical concepts in a consistent way. So you might peak at what’s the page – we have a definitional page relatively early in the environmental scan that you might peak at because that attempts to set the scope. If that’s helpful.
KLOSS: Page 103, terminology, code, classification, nomenclature or vocabulary.
GOSS: Can I build on a little bit of that? So Bruce, I heard you talking about we have new kinds of data that we need to factor into our care delivery and potentially the financing aspect. So I kind of heard your comment about how do we get these new types of social determinants of health data into the overall process, whether it’s a clinical or an administrative financial function in health care. Did I not hear that correctly?
KLOSS: And the issues of classifications as it relates to certain social determinants are embedded in some of the terminology, certainly SNOMED, ICD have some elements but with regard to social determinants and those kind of environmental issues, some are part of code sets but it’s not one system. And I think that would be an area where we would include that as a gap. So I think naming a person’s name is different but some of that is in some ways spread throughout terminologies, it’s not brought together. And whether when it’s brought together it’s adequate to future or current and future needs is one of the gap questions that we want to explore.
ROY: If I can just add a little bit. Exactly as Linda is saying, that is definitely one of the gaps that we wanted to make sure was included in the list of gaps but also I think that that starts to play into some of the future projects that we were looking at. Again with this group you’re very on top of everything and how everything is kind of intermingled and that is one aspect, the population health, social determinants and everything along those environmental conditions related to health for potential kind of next steps. Just because a lot of them haven’t been identified or – we need to explore that as a next but for this current environmental scan purposes, we’ll definitely include that as a gap.
GOSS: I recognize that it’s a gap and I support where you’re headed with this, Bruce, and sort of trying to tease out the specific issue and I think it isn’t – to tie it back to some discussions from yesterday, it’s an intersection point with ONC, especially related to value-based care and so I think that there are – I’m looking forward to us modernizing the project scoping document after the vocabulary Roundtable because I do think that there’s tendons of all this that’s going to weave back into sort of the 13^th report to Congress objectives as well as what we do in standards.
KLOSS: So where we are at is that we’ve done a pretty thorough deep dive in section one, two and three and in four, five and six are areas on this scan outline that are yet to be fully flushed out. So this discussion about gaps, what your understandings of coordination and maintenance, dissemination issues and any other questions that emerge from your read are really what we sort of want to focus our precious time on this morning.
LOVE: I’m sorry. I just had a question on gap and maybe it’s just in my scoping education that needs to happen but pharmacy data is a big deal and the states are forcing – are being forced by Medi-Span, a very expensive, private proprietary technology so that they can get dosage and NDC full information and they maintain a private terminology. So how does that bridge into what we’re doing here because that’s not sustainable as states look at the pharmacy and opioid problem. And the data available through the claims transaction is not sufficient for what they are doing.
KLOSS: In some ways what we have covered here are all the native named standards and yet we know they are being adapted and used in various applications and proprietary products and we made the decision that we couldn’t go down all of those paths, even if we wanted to. But what we have described – what Suzy and Vivian have described are all the named drug-related standards.
LOVE: And then another – I notice in the report they encourage groups to expand and fill those gaps in name terminologies so as I read the report I just need help understanding like the Public Health Data Standards Consortium payer typology. Is that just not even part of the scope? It’s homeless right now.
KLOSS: That is another issue.
LOVE: It’s a five-year project on hierarchical coding and then no one supports it. We have a private sector group that’s interested in it but of course they want to charge for it. So I don’t know, I’m kind of out of scope here. I understand that, I’m trying –
KLOSS: These are all good areas to probe because they need a home.
STEAD: I am just trying to understand how to parse the conversation. So one question is one way we can handle this journey is to take – is it Medi-Span –
LOVE: Yes, that apparently comes out as a very –
STEAD: We can take that as an example and look at it. If you look at the – we have done all of the main standards. We’ve done a large number of additional standards that are, if you will, in common use but we quickly decided there was no way that could be exhausted. We do need to know if there are ones that are key pinch points that might be the subject of either things that could be advanced into main standards or could be examples of gaps.

The key ones there we should try to pick up and we sort of got two places we can try to do that. One is the additional standards and the other are standards used in other domains. So those sort of two buckets where we’ve tried to bring in the most obvious ones. So it would be very helpful if there are specific candidates you would like to see added, we can easily add a specific candidate. I think Linda and Suzy and Vivian and I felt we had already, if anything, erred on the side of being too exhaustive – or exhausted as the case may be – and so we would really like the additions to be point specific, if you will, which Medi-Span could be a test case for, it that’s helpful.

KLOSS: And the other criteria for – we wanted to kind of span the waterfront so that while some of the nursing terminologies aren’t name standards, they are important and they use – and part of UMLS. So yes, we did err beyond those things that are – and there are a lot of proprietary applications based on standards that take name standards and adapt them but we could illustrate some of those but not describe all of them most certainly.

So that was our rationale for going through those descriptions and for the description, for each of them we’ve done purpose, usage, how it fits into the naming schema, who approves it or where it fits, ownership development principles requesting what’s processed for requesting new content, the update schedule and sort of access. So all those points squarely related to development, adoption, dissemination, maintenance, coordination, and we think this – particularly this section three will be an enormously valuable standalone and put a stake in the ground as what this environment looks like today.

With regard to gaps, a couple of questions in addition to where Bruce started us. We’ve discussed known gaps, perhaps in the area of genetic vocabulary terminologies. There are a couple of them that are now in UMLS but are not named standards so that’s an area. Another area that’s emerged is functionality. There are international classifications of functionality that are not in widespread use in the US and are not named standards. So I think those are the three, the social determinates, the genetics, functionality that kind of mind for our little subgroup but obviously we’ll throw that out to the Roundtable participants to add to and refine and to this group. And I think your public health be added as a fourth that we want to explore.

STEAD: I think we have not actually clearly distinguished in our head public health and social and behavioral determinates – we probably should think of those as two complimentary lenses.
MAYS: I guess there are two. One you started to mention and that’s the functional dimension because I know that the Social Security is taking a whole look at that and trying to determine how to talk about it. That’s number one.

Number two, I want to go to when Bruce started, and again, I don’t know if this fits or doesn’t fit, and that is right now in the UC we’ve had to re-engineer a lot of our records because we made a change. And the change was that of allowing people to do their names as they wanted to but we have to make it roll back up because we have a lot of international students and we were having problems with them coming in and they call themselves Jane and their name is really much more complicated.

But the other thing has been around gender identity. So that’s the other thing is that this issue of how do we classify in terms of – let’s see if I can get this straight – at origin, and kind of currently and kind of what the standards are and how that’s to be utilized. So the UC is currently doing this and it’s a lot of work. But I think that’s one of the social determinates that you want to think about this standard of how that’s to be used within the health care setting.

ROY: So in terms of gender, sex, biological sex, and even sexuality, a lot of those terms are actually currently in SNOMED-CT. How those are utilized in some sort of framework that I think goes beyond the terminology realm, that’s more of like a transactional standard. But as far as the actual gender or sex fluidity, and as well as orientation, sexual orientation, all of those are currently being remodeled in SNOMED-CT. Some of its already in there and I think in the next couple of releases those will all be included. So those terms are actually in the terminology already.
KLOSS: And there are ways to capture variation in ICD and I don’t suppose that there’s necessarily harmonization across how those are being handled which is –
AULD: And the fact that it’s being remodeled in SNOMED reflects the request of the members, the needs of the international members whereas we need to focus first on what we need in the US to address Vickie’s needs.
CORNELIUS: In terms of gender identity, we are seeing the use of pronouns for self-identification. Now, that’s the easy part. The challenge is how do you come around that way and figure out where you want to enter that as a field.
KLOSS: Could we advance the slide please. So we had three discussion themes for today and we’ve launched right into them which is great. And these are not unlike what we’re going to pose to the group coming to the Roundtable but what are the near-term opportunities to improve the development, maintenance, adoption, dissemination of terminology and vocabulary standards to redress known operational problems and improve the value, cost and quality in their lifecycle from development through use. And what I think is revealed powerfully in part three that is in this environmental scan and largely complete, is that each vocabulary and terminology sponsor has developed their own best process for doing this but there is not much consistency across processes. And so if you look at this from the user perspective, we’ve got updates coming at organizations in a lot of different forms, on a lot of different timetables and formats, training. There’s a lot of variation that’s revealed when you read the detail under these and we’re going to try to raise some of those issues.

And I guess our question to you is top of mind what was revealed to you when you looked at the draft in this regard? Near-term opportunities, did anything just kind of strike you that will be helpful to us in framing the discussion in July? Bob.

PHILLIPS: Thanks, Linda. This is like a curriculum. I hope you’ll use this for other educational purposes because the history alone is worth having organized like this. I’m increasingly interested in hearing from the data use community about how the current terminologies help them or don’t help them. Especially with big data and epigenetics and personalized medicine, is how can we make these more useful so the data that we’re painstakingly extracting from health care actually serves a bigger purpose. And I just would encourage us to ask that group to come in.
MAYS: That reminded me. One of those groups is actually behavioral science. Because what happens is that the code sets don’t seem to work real well in terms of intersecting mental health into some of the ICD stuff and so there has been this misfit there. So I think that if we can get some of those individuals to come that would be good.
KLOSS: Any other questions around that that may have emerged in your reading? Another area we want to pursue is whether the current kind of governance of code sets and terminologies as they are used in the United States is adequate. If not governance then level of coordination and communication. And I think that will be an interesting discussion and not unlike some of the discussion we’ve had around administrative standards.
STEAD: To what degree do we have our head around a list of known operational problems?
COUSSOULE: What do you mean by operational problems?
STEAD: I think the – I’m assuming that means the operational problems that result from update cycles, for example, but a part of trying to get a list would be to clarify that.
GOSS: And is there value to do some homework before the July meeting. Do we want to either ask the people that are coming to that Roundtable or maybe ourselves to – because I’m thinking maybe that’s a definition that we – I know you’re making a joke but maybe it does need to go in the definition section when we talk about operational problems, the things that we’re trying to address, the real-world dynamics.
KLOSS: I don’t know that we’ve organized that yet. I think what we’re coming out of this meeting with is then now planning the – detailing what the discussion will look like in July and we have an opportunity when we distribute – our plan is to distribute the draft environmental scan in draft form and certainly take corrections and additions and deal with questions that emerge from people reviewing it. And I think we have an opportunity at that time also to ask for input on – pre-input on what people think and make sure we know what they want to accomplish.
STEAD: To put a point on that, we’re hoping to distribute a then current and a smidgeon more cleaned up, particularly in the sections we mentioned, draft of the scan to a fairly broad electronic vetting group on June 5. Bring back those comments. This is for those of you that live through the measurement framework, this will sound familiar – bring back those comments, update the scan and then redistribute it to the participants in advance of the Roundtable. We will also update it after the Roundtable. So this is an iterative process.

We also have in some of the headings on the Roundtable things such as examples of best practices, some of the things we’re asking people to bring in, that can in fact – those presentation preparations will be a form of pre-work but all of my scar tissue is we will do better if we begin to populate a list of what we already know so that we have a working straw person or example that we flesh out as we go through this process. So that’s where I’m trying to nudge us, if that sounds familiar.

KLOSS: I think we can. I think we can develop –
GOSS: And I appreciate that there is advanced timing either to general feedback loop or the attendees because it is a – I suspect that the attendees will want to get some of the home office support team to look at this and sort of give them enough time to – it will serve us better in the end.
ROY: Could I suggest that operational problems could be defined as the outcome of page two of the scan that say introduction one and two which is the purpose of the scope of the report. So thinking through that, the operational problems would be from the changing environment and implications for timing and approach to health terminology and vocabulary standards adoption and the needs, opportunity and problems for development, dissemination, maintenance and adoption of health terminology and vocabulary standards.
KLOSS: Good suggestion.
COHEN: So would be in scope to review the release of the next version of ICD and what works and what doesn’t work to make suggestions about how to improve that roll out. Is that something that –
KLOSS: How to – well reflection on the last roll out. I mean that certainly was one of the triggers but – now we’ve got new systems out there and they are being maintained and what issues are there with regard to use issues, how fast are they getting resolved with guidance disseminated. The issues are different by terminologies. When you look at SNOMED-CT, one of the big issues is how to get it really mainstreamed in use, how to build and optimize use cases. So we’ll have to give some careful thought to how to do that but I think you make a really good point – all of you – that the more we can distill what we’ve learned it’ll help take the next step and make the Roundtable more effective. So thank you for helping us think through this.

Another area – and I think probing on governance, coordination will be a good process. To help us do that, you noticed in the Roundtable we have invited and have a representative from the Canada Institute for Health Information coming to describe how they coordinate, governance in communication across Canada. It’s just a way of us kind of thinking a little out of the US box, if you will.

Strategic gaps. I think I would like to just take the next few minutes with us is next year what would this committee consider to be a real measure of success of this initiative. Because if we could get that goal, it’ll help us kind of update the scope and make sure we go into the July meeting very clear about where we need to land. Could we have a little thinking on that? What would really satisfy you as a successful initiative? How would we measure the success? Could we do a little brainstorming on that? Rich, start us out, thank you.

LANDEN: As I look at earlier slides on the charges and the scoping document, it says essentially we’re supposed to identify operational issues and if that’s what we’re trying to do than the measure of success would be can we identify operational issues and to come up with solutions. If so, if we resolved 80 percent of the issues, or possibly X number of millions of dollars of value to the US system, be it health care or public health or however we want to measure it, but just what we expect would be different in the future based on our actions and some metrics around that.
KLOSS: Those might be shorter term.
LOVE: Some sort of coordination across these terminologies because people are menu picking and choosing and paying and when you’re building a state say reporting system you may pick and choose the terminology that maybe isn’t the best fit or that’s just available or that you don’t have to pay licensing fees so you do workarounds. But having some sort of guide to how you migrate through these terminologies and make the best choice for what you’re trying to do. I mean if you want to use CPT, there are certain things that are embedded into that decision.
COHEN: I think the stewardship tool that you developed is a great model as a potential guide, a guide for use of terminologies and vocabularies. So to building what Denise said, think about a product that would be easy to use and straightforward for a variety of organizational levels to help them sort through the options.
KLOSS: Suzie.
BURKE-BEEBE: With those two last discussions it makes me think about whose success we’re trying to measure, our own or our audience? Our communication success, our measures of success as a committee or those that we’re trying to help.
KLOSS: I think the latter.
BURKE-BEEBE: So how far down in that stakeholder realm are we going? Down to the granular level or somewhere above that? And so is that part of what we’ll be hearing at the Roundtable? Or when will we be able to begin to capture that?
KLOSS: Good questions. I think another – and just from your discussion, it seems like we’ve got both – not unlike what we talked about yesterday – the idea of some short-term operational improvement recommendations that may come out of this. And then some longer term strategic, and the longer term strategic would call for, I think, improved governance and coordination and use support and then linking it to the predictability roadmap so we know over the next 10 years or something where – how we are going to progress. I think we have to do a shorter and longer term view.
STEAD: I think if in this round of work we can get our hands around the handful of solutions that help the community reduce operational problems while getting a level of understanding that would let us think about a 2019 or a 2020 project that would lay out a one or two decade path to continuous terminology, to evolving from static periodically updated terminologies to a continuously incremental and evolving terminology knowledge base and maybe that could be a one decade job, not two but it’s certainly not three to five years. So my guess is it’s in the one to two decade range. But I believe it’s now doable.

We got rid of the politics and everything else, it’s now doable. I guess that would then raise the question, could you actually in a three to five year period begin to set something new alongside what we have and ultimately allow what we have to become less important and phase out. So I think those kind – I don’t think we can have those conversations this calendar year. I think that those – I would like to see this year set up the understanding that allows us to figure out how to frame and set up those conversations for the next year or possibly the year after.

Another thing in addition to operational, we might begin to see our way towards any updates that might need to be made in the criteria for name standards. That might be something that could be done this year.

BURKE-BEEBE: So along those lines, human nature is just to be satisfied on a quicker stance if we’re being measured for success. So I would think that less than a decade would need to be considered for measuring success. And also maybe this is obvious but I want to say it, I think we need to make sure that we have named ownership of what those achievements – where they would be coming from so that there could be ownership that we thread through our discussions with stakeholders like at the Roundtable. That makes our life easier, at least as I’ve seen it over the past 20 years, that when we have this collaboration that there are those that are taking ownership of those successes that we can bring to light.
KLOSS: Should we look quickly at the Roundtable agenda and then I think we’re right on time. We’re going to – so we have how many participants, Rebecca?
HINES: I will tell you in a moment. Bob just secured someone from NCQA in the last 24 hours, it looks we have including federal staff close to 40.
KLOSS: So we really have representatives from all of the major systems and stakeholder.
HINES: So NLMs, SNOMED, LOINC, RxNorm, AMA, AHA, AHIMA, the Canadian Institute for Health Information, UNC, VHA, PEW.
STEAD: World class people. We’ve asked the people we wanted and they’ve really all agreed. It’s quite remarkable.
KLOSS: Vickie is asking whether we have somebody from the American Psychiatric.
HINES: I don’t believe we do. We don’t have anyone from the American Psychiatric Association that we invited? No.
KLOSS: We should look at that. Thank you. So we’re going to start out with just the highlights of the environmental scan. We’re not going to spend a lot of time on it because it will have gone out for comment and then it will have gone out again. Hopefully everybody will come prepared with that.

And we’re going to conduct the meeting kind of like we did with the predictability roadmap and some other Roundtable meetings. We’re going to try to focus thinking on best practices around four topics. The first morning we’ll do breakout number one on development and maintenance best practices and best practice in dissemination and adoption. Next slide. And then we’ll have a discussion around opportunities for improved governance and coordination and we plan to do that as a third. And we will have had kind of a briefing from the Canadian Institute preceding that so we’ll think about governance perhaps a little differently.

And then the second morning, we’ll tackle gaps and we’ll talk about ICD-11 and then how this links to the roadmap for standards and then we’ll close and in the afternoon we’ll have a working session to try to assimilate everything we’ve heard. So it will be a day and a half and it’ll be very active, we hope. And we hope we’ll come out of it with some good actionable suggestions and some coming together of thinking.

And I don’t know when this particular group of stakeholders has come together. It certainly has been outside of my range of remembrance of NCVHS work. So anybody wanting to come now that we –

HINES: We have many members are already –
KLOSS: Excellent. Anything else for the good of the cause today? Thank you for your thinking on this and review and stay tuned.
STEAD: Sounds great. I really want to thank Linda for shouldering the guidance of this for committee and Suzy and Vivian for really unbelievable hours of work on the environmental scan. We’re gaining ground and I think this is going to prove important.

We will take a break and then we’ll regroup to sort of talk through as a committee as a whole how we might develop guidance for the NLM about how to enhance UMLS to accommodate social and behavioral determinates in public health. And that’s not something we’re trying to do right now. We will understand it in this context because it would be a candidate 2019 project that would again dovetail well with other sequences that are going on around us. So that will be what we do after the break and that will have to be on a fairly tight schedule so that we are ready to go when Susan Queen is ready to brief us. Thank you.

(Brief recess)

Agenda Item: NCVHS and NIH/NLM Collaboration

STEAD: Could we bring up the slide that goes with this particular discussion block. While we are waiting for that to happen, Suzie told me that Patrick Mclaughlin is on the phone. As the UMLS guru, would you introduce yourself, Patrick.
MCLAUGHLIN: This is Patrick McLaughlin. I am the head of Terminology QA and User Services at NLM. My group works with RxNorm, SNOMED, and the UMLS.
STEAD: Patrick, thank you for taking out time to join us for this block. What I want to do is in essence, to have a introductory conversation about how we might go about developing guidance for the NLM on an approach to expand UMLS, to include – and I think we have to think of them as parallel public health and social behavioral determinants of health.

Advance the slide please. What we are basically thinking will happen is we will come out of the July Roundtable with an understanding of gaps at large, but also a little bit more understanding of gaps on the social behavioral determinants in public health.

We have got to be a little careful in the Roundtable that we don’t these gaps to be the dominant gaps. These are two big categories, if you will, of gaps that we think where we will want to get smarter about. We have tried to add the right expertise to have the content expertise in the room without, if you will, creating an imbalance.

But then we think that this is an area that would be worth a systematic approach. To back up, if you think about UMLS, UMLS is not a terminology. It is in my words, Patrick can correct me if need be, a knowledge base that makes explicit the relationships amongst a number of standard terminologies. The collective set of concepts that are represented in those terminologies, in essence are the scope of the UMLS Metathesaurus. That is a layman’s interpretation.

Is that close enough, Patrick?

MCLAUGHLIN: Yes, that is absolutely correct.
STEAD: As we think about how we could systematically look at how we could expand the scope as needed, it would really be two approaches that I think would work together. One would be to look at the terminologies that are contained within UMLS to figure out which of those terminologies have terms that represent public health on one hand, or social behavioral determinants on the other hand, and see whether it would be practical to recommend expanding one or more of those base terminologies, if you will, to get the level of coverage -this sort of modern thinking about health requires.

Then whatever the develop and maintenance process of those terminologies with this guidance could use their processes to make the additions, by then will have come out of the Roundtable with a pretty good understanding of what those processes are. As new terms were added to those terminologies, they would then be incorporated in the next updates to the UMLS Metathesaurus.

That is one leg of work that I would believe that we would want to think through how we could help.

Another leg of work would be to look at other terminologies possibly from domains that are not at all represented today in UMLS. One of my thoughts that the Committee can think about whether it is reasonable or not, if we take the 10, I believe it is, domains in the measurement framework for community health and wellbeing and their sub-domains, do any of those domains in fact have existing terminologies? Because we picked those domains to be – so that every cabinet level agency, every relevant cabinet level agency, could see themselves in one of those domains and the sub-domains would be in terms they were used to thinking about their goals.

That could be brought together for collective impact across multi-sector health world or world that we want to help improve health.

It would seem to me that would be one way to really open up the box but in a constrained way because we spent some number of years landing in on those 10 domains, and I believe something like 30 sub-domains.

My guess is we would discover complete areas in that that are not represented in UMLS. I may be wrong but I thought those could be two very different lenses through which we might look at this.

With that overarching introduction, Suzie or Vivian, would you like to add anything or am I summarizing sort of our preliminary conversations okay?

(No response)

Bruce or Bob, from the conversation we had a few months ago, along these lines on the Pop Health Subcommittee call?

COHEN: Where to begin. I totally agree with steps around the first couple of steps around examining terminologies related to social determinants that might already exist in understanding where those gaps are.

I would also like to get a sense of having us describe really what we mean by social determinants because in my mind, and others might feel very differently, the social determinants scope is narrower than the population health or the public health scope, represented by the framework. The question is do we want to be looking at voter participation as part of issues related to NMLS work or not. A lot of these domains and sub-domains that we use to describe population health and community wellbeing, may be relevant or may not be relevant to the scope of what we want to be doing.

It really depends on how big a bite we want to take and what the breadth of the initial effort should be.

HINES: Vickie and Bob both forwarded some very interesting articles over the last couple of months. I was fascinated by an example where for instance, housing could be added to the electronic health record because the woman kept showing up to the ER with asthma attacks and it turned out it was because there was mold in her house. So until you remediate the mold she is going to keep coming back to the ER.

Then how do you put into the medical record that we contacted someone with the City Housing Department to get her house remediated so she would not have her weekly visit to the ED.

So that to me, is a perfect example of why Bill’s suggestion that the NCVHS Measurement Framework actually would be worth looking at for concepts that could end up in the UMLS because there are incredible scenarios that we don’t even think of that actually would be worth documenting. That was just my favorite example from the papers the two of them sent.

GOSS: I have a side comment, but I want to actually build on what Rebecca has said first. The National Association for Community Health Centers with support of Robert Wood Johnson Foundation and some others, set out to create a framework with process methodologies that were supported in EHR data acquisition to get that kind of data related to social determinants of health.

I think the train has clearly left, the example that you give is a really good example of taking care of folks to keep them as healthy as we can may take some creative innovative ways and we certainly have to respect the evolution of the payment methodologies related to those types of services.

But aside I want to go back to Bruce’s comment about taking a look at how we are defining SDOH. It seems to me that the measurement framework, the domains and the sub-domains, is where we should really focus. Especially considering the level of effort we put into that or a number of years and also maybe from the lessons learned, we may hear later from Soma.

I am curious if you were thinking about – that is where you were heading Bruce, or there is something that I am not getting in what you said

COHEN: Great example and I think there is a lot to be learned from the framework. The Measurement Framework is huge and I think that we would need to distill the relevant components of it to be actionable in the short run. That is all. That is just a cautionary note.
STEAD: Let me say two things that might help as we think through how to apply that caution because the caution is very important.

To the degree that we can find terminologies that exist that are relevant to the domains or sub-domains, then incorporating those terminologies into UMLS is relatively easy to do. Our guidance in that case would almost be as simple as the framework as a good starting point, we should figure out how to do a scan for what terminologies exist in those domains and sub-domains. Once that scan is done, comes back, we have got a set of existing terminologies, which unfortunately might be small, but maybe it will be big. Then we can think through how we winnow if needed.

That said, if we discover that there aren’t terminologies that is when we get into the much harder challenge of how do we want to develop guidance, because I think this would help NLM, which sub-domains would be worth starting on or finding a partner. It really is tricky. The way UMLS is put together, NLM in large part, does not develop terminologies. Although there is a NLM role in LOINC, there is a NLM role in RxNorm, NLM roles in how we work with SNOMED, but we would have to be careful.

From my perch at least, I think that the Measure Framework is as good an overarching scope we have, subject to the caution. The other comment I would make, and I don’t know how much this has come across, but the UMLS Metathesaurus is primary knowledge base that all of the medical natural language processing algorithms use to recognize concepts in non-regularized data sources.

As you look at things like our ability to mine large urban data sets in a way that could allow them to be linked in some way, either for community health/public health at-large purposes or for linkage to EHRs for connection between public health and health care, if we can figure this out we may accelerate that work.

I like the caution, but I would prefer not to narrow it until we did the first scan. But if that seems unwise –

COHEN: I think that is a great idea.
PHILLIPS: I just want to add, going back to Gib Parrish’s environmental scan for the Measurement Framework, we have a lot of this covered there already, I think in terms of capturing that. The prepared tool that you were talking about Alix, is not there but some of the tools are in the field right now. I think we could update by pulling those into and looking across Gib’s environmental scan to help really cover the landscape of what social determinant nomenclature is in use.
STEAD: That is a really good suggestion. Other thoughts?
KLOSS: This goes back to the pre-break discussion about the Roundtable. To what extent do we need to have any kind of overview of UMLS for Roundtable participants or that not relevant or does that come later? I just hadn’t really thought about that as a background.
AULD: Are you thinking of background document or presentation?
KLOSS: I don’t know. We don’t have much time for presentation but I was just wondering if somehow we need to incorporate some background information.
AULD: I think we have a few things we can pull together.
GOSS: Most people would have a sense of what it is but maybe it is how you are leveraging it within context of the conversation to position it for the attendees.
STEAD: I guess the thing I would say, and this may be tricky, we will have the environmental scan, which puts a lot of that in context to the degree people read it. We also know the environmental scan is long. Whether we want to include something specific to UMLS – I think the challenge we get into is what else do you end up including something specific about.

We have got how much time on the agenda for Suzie and Vivian to unpack the environmental scan?

KLOSS: Not much. Thirty minutes
STEAD: My gut is people are either going to come in knowing something about UMLS or it is a bridge too far probably. Lots of yesses. I haven’t seen Roland with such a clear yes.
KLOSS: We will connect those dots after the meeting if they have not been connected.
STEAD: Good question. Other questions? Okay, then let us declare this block done. I think we have a common understanding of what we are currently thinking about. If we come out of the Roundtable with our thinking still this way, this could turn into a Pop Health scoping plan for 2019 in conjunction with NLM.

Thank you, and with that, if you are willing, we will let Susan get started with a few extra minutes.

Agenda Item: NCHS Update

QUEEN: Thank you for the opportunity to be here today on Charlie’s behalf to give the update for the National Center for Health Statistics. I talk pretty fast so I am not sure I will need those extra minutes.
STEAD: We will try to ask questions.
QUEEN: Okay, please stop me at any time.

I wanted to start with the budget history just to show where we are at NCHS. It has been a pretty flat budget over the years. The good news is we have not been subject to some of the cuts that have been experienced by other programs and agencies. The downside is that flat budget means your purchasing power is going down.

MAYS: Susan, may I ask a question about the budget? Can you talk a little bit about what the implications of this flat budget have been on the surveys? I know there was a reengineering done recently and I think that resulted in some cutback of questions. Can you say what is going on?
QUEEN: Are you thinking of the National Health Interview Survey?
MAYS: Yes.
QUEEN: Which I will talk about the redesign a little later. If you have detailed questions I will certainly forward them to the program. But that redesign was originally slated for 2017. It really wasn’t specific to budget, although many people thought it was. A lot of it had to do to due to the fact that the – I will deal with this later, too, response rates really experiencing a great decline, plus, the survey, whereas the average time might be around, I think, an hour and fifteen minutes, for people who were falling into certain categories and getting all these additional questions if they had a certain condition and had follow-up questions.

Some of these interviews were taking as long as an hour and a half. With so many people working during the day, it was getting to the point where interviewers were saying, to a certain extent, we were only able to complete sometimes an interview a day. There was a lot of contacting and recontacting and scheduling. It was really wanting to take a look at the content and how we can do it better. How we might be able to reduce the burden with the hope that that will help with the completions and with participation.

The last redesign of HIS was like 1997. I think NCHS used to say that every 10 or 12 years they do a redesign. It really was time in trying to look at some of the content that either could be removed, it was no longer as relevant, or did you have to ask it every year? Could we have something that is a rotating set of questions versus a core set of questions? It was more related to that than specific to the budget itself.

For the budget, with a flat budget and you have increased costs, increased contractor costs, increased interviewing costs, it does mean trying to do the same or more with less. There have been a lot of serious looks at the surveys to see where we can save money. There has been a lot of creative thinking about certain things that have cost more in the past, how could we do some of these better? How could we be more efficient?

Some of it is making us look at how to save money through efficiencies. But it does mean, I think for a statistical agency, the area that is most likely to be hurt is the area of innovation. Coming up with either new techniques, new approaches, improved approaches. Where it is particularly difficult is when you have declining response rates and things clearly can’t continue as they have been and we can’t keep using the same approaches. That is where you would like to have money to do more innovative research. Or with the HIS, in an ideal world, you would have had two parallel surveys running as you are doing the new survey with the old. Now we are going to do a quarter split sample. That is where you get hurt.

One thing that has been, it is in the President’s Budget for 2019, is NCHS used tot be funded out of the PHS Evaluation Funds, that meant that the agency was subject to getting tapped for the Evaluations Funds, plus a couple of other taps that may not seem significant for some agencies but for us it was a hit. That has been proposed for 2019.

We don’t know if that will happen or not. So it would be a change. Instead of having the budget authority – it is hard to know. We think it would be a good thing to have that back in place, which is what we used to operate under. There was a shift in the past of going to budget authority. I am not quite sure why, that is more of a historical shift, but this was proposed.

The downside is there is a proposed cut for NCHS in the President’s Budget. It could be balanced out if we are funded under the Evaluation Funds. Some of that might be balanced out as a result. We are just going to take a wait and see approach.

A lot of what is also relevant for NCHS in the budget of NCHS, has to do with other agencies and their funding’s. So if you go to the next. This was a FY17, the Reimbursable Dollars versus the Budget Authority Dollars that came from NCHS for each of the major surveys in the Division of Vital Statistics.

You will see the blue part is where NCHS is providing the support for the survey or the data collection. The green is the reimbursable dollars that we receive from partners and co-sponsors that participate in the surveys.

NCHS as a statistical for the Department, must and should be collaborating and engaged in partnerships and working across the Department and potentially beyond, to ensure that the data that are needed for the Administration and the Department as a whole, are being collected. So we need to serve in that capacity. That is a good thing that is being done and having that participation and buy-in is very important.

The downside is if those other agencies are experiencing their own budget issues and budget cuts, and they may be saying, well, how much can we afford to continue supporting the survey. Or if there is an expectation that we are giving you money so of course you will ask these questions. Part of it is negotiating what is the best in terms of the science and in terms of the data you need to get. There is a vulnerability because in NCHS over the years, has become more reliant on these funds to support the core survey collections.

What I wanted to talk about in the next few slides is the role of NCHS as a statistical agency, one of the first responsibilities is the production and dissemination of relevant and timely information. I wanted to give some information on what NCHS has been doing to fulfill that responsibility.

This first slide here shows the recent of information within 10 days of the event of death, by NCHS over the years. For year-to-date 2018, it is something between 58 and 60 percent. The aim is to get this up even higher. But you can see that there has been a lot of work. You have heard from Delton. You have heard quite a bit on vitals, both the ongoing work that he has been doing and the special projects that he has been engaged in with the states to improve electronic reporting and collaboration. It has been very fruitful.

As a matter of fact, if you go to the next slide, this now is old. These monthly counts of dug overdose deaths are something that NCHS started doing last August, after the highlighting of information from mortality was key to identifying the opioid crisis. So there was an interest in seeing how much could NCHS provide on a monthly basis. We started doing this last August.

So as of a certain cut-off date, it is the provisional counts, not rates, of the drug overdose deaths. So we know it is an undercount. This morning they released the next month’s counts. If you go to the site, all of the numbers will automatically be updated. It provides this 12-month count by jurisdiction. It breaks it out by the specific drugs from those that are identified on ICD codes. So it is a limited set. It also has a number of maps. It is both timely and relevant.

BURKE-BEEBE: Susan, how close to real time are we?
QUEEN: Do you mean from the event of death. These are – this one here that I am showing from last month, and it was released, let us say early April, the cutoff date was as of all the records that NCHS had received as of April 3^rd. There is a lag from when the drug overdose deaths occur to when NCHS receives the final record.
BURKE-BEBEE: I know the system is working in working with Delton that we are working on that. I just wondered where we were.
QUEEN: This has to do with the fact that the drug overdose deaths require the investigations, the toxicology reporting. We know it is an undercount. NCHS is working on – DVS is working on an adjustment that they will be providing in the coming months to be able to adjust for the fact that they know that this is an undercount. So I think using previous years of data, they are working on a methodology that will allow what is a more realistic picture of the number of deaths.
MAYS: How do you accelerate the reporting of these particular deaths? Is it something that the state does for you?
QUEEN: Delton, and other parts of CDC, have been doing a lot of collaboration with the states to try to support – some of it is funding. Some of it is working o the development of standards but a lot of it has to do with helping ME coroner offices. There is very much a state function and each state has very different ways of doing this. The support that NCHS provides tends to be from technical assistance, as well as financial support.

CDC has programs that are working as well. So they have some grant programs that are working with the states as well. I think this is just going to be a long-term process to do whatever can be done to speed up – states are interested in improving as well. They see a value in it.

BURKE-BEEBE: Delton and I, secured and he spoke to this in previous presentations, a PCOR funded project that we are doing, which is in part going to speed up through revisions of the death certificate specific to drugs and overdoses.
QUEEN: I suspect it won’t be a revision to the death certificate itself.
BURKE-BEEBE: There are fields.
QUEEN: Perhaps providing more information on the literal text
BURKE-BEEBE: That is it.
QUEEN: So the NCHS released these reports, I want to say it is actually December of 2016, with a methodology looking at the literal text on the death certificates. That is how they have come out with some of the more detailed drug codes and reporting on fentanyl specifically, versus other drugs. The combinations that you get on some of the ICD codes. Part of the issue of that is there is always this caveat. For those death certificates that have information on the literal text fields, this is what they are finding. If it is not there it doesn’t mean it might not have been something that was reported by the ME coroner. What is required to be filled out on the death certificate doesn’t necessarily have this additional detail.

So there is something to keep in mind about that. Delton, in working with you, has secured this money and has a lot of special projects going on with sets of states on how to improve these things.

BURKE-BEEBE: And we have five states that were demonstrating so it will be very helpful. California is one of them. I am trying to remember. New Hampshire, I think is one. He and I just talked about it yesterday. It should feed a lot of information and the adjustments that are being made as we go. We are working with the coroners. It is very big.
ROSS: Susan, my question is just these 10 days, is that 80 percent are now reporting a completed set of data within the 10 days or the fact of death?
QUEEN: It is a fair amount of information. I am just going to be honest and say I don’t know the answer to that. My inclination is that it is not everything on the death certificate. There is pending information.
COHEN: That is the correct answer.
QUEEN: The “I don’t know” or the pending?

(Laughter)

COHEN: It varies enormously from state to state about what they can report and how quickly they can report. Some of the state laws need to be revised to improve the rapidity of reporting. But generally, it is very basic information; fact of death, cause of death, but not the complete statistical information.
QUEEN: For most of the ones that are – there is a very fast turnaround. I think 85 percent of the death certificates get this automatic coding, but those are not going to be the drug overdose deaths. So there is a very quick turnaround taking the literal text and providing ICD code, but those are on the more common causes of death that fit with the automated system better. Plus, you have the information available to provide the coding.
ROSS: I think at the Vitals hearing last September, the idea of the notion was teased – sort of came out asking about the feasibility of reporting that a death happened with a very limited set of data about it. Like personal identification, geocoded, whatever, but a very small set of data, as a first piece to indicate event of death.

Has that been discussed any further? Within this 10-day period, if that were to be happening, would it be possible that 80 percent could report event of death within a day or two?

QUEEN: I don’t know the answer to your question. That is something that I will actually have to take back to Delton because I think in some cases it is more than just the event of death, but I don’t know the extent of the information that is provided within that first 10 days. I don’t know if he provided information that was specific to that at the —
ROSS: It is sort of an open question right now. To what extent if that were to happen, would it change the actual information we have and shrink it from say, 80 percent reporting a death within 10 days to 80 reporting it within a day or two. Understanding that it doesn’t give you causality, it doesn’t have a lot of the rest of the information – probably wouldn’t have all the rest of the information that you get with a complete death report. But just having the information about the event of death, time and place, starts to tell you something about temporal trends, spikes, raises questions. It is a different kind of information set.
QUEEN: I will absolutely take your question back to the people in DVS. I suspect that there is more information coming within that 10 days and some states are doing things even greater rapidity and with more completeness, depending on the type of death.

I would love to have Delton in here answering so I will make sure to get an answer for you.

COHEN: I think also some of the state folks in NAPHSIS, who develop the EVVE Fact of Death Reporting System, there are 27 states that populate this hub on a daily basis to gather that information. So there are a couple of approaches that folks are taking to try and increase it. The real hold-up is the money at the state level, and like for ME deaths, most of them are in offices of public safety and not even in health departments.

These offices of public safety are incredibly underfunded. Even though opioids are certainly a large issue, it is tough to advance the trail quickly, but a lot is happening.

KLOSS: I just had a quick comment earlier and I was remiss earlier not to ask Donna Pickett, I think she had dialed into our last hour. I don’t know if she is still on the line but in conversations with her over the last couple of weeks, she described for me that she was working closely with Delton to make sure that the guidelines for accurate coding of the opioids were being developed and disseminated. I think that is another illustration of how speeding up the whole process of getting that guidance out far and wide, is just an important element in all of this so the data is accurate.
QUEEN: This is just a snapshot of a period of time for the NCHS data releases and publications. This is from April 2017 to April 2018. In part to show that the agency is very productive in terms of what it is releasing to the public and disseminating. This does not include a number of additional data files that are released into the research data center.

The Public Use Files that are referenced here, are the larger ones that you would expect from the major surveys. There are a number of smaller files that end up getting released, particularly some of the files from NHANES. They also have some additional public files that they make available that are not included in that Public Use release there.

The dissemination of the information has, NCHS as an agency, has been very productive. In 2015, there was a Rapid Release Program was conceived of out of Delton’s group. It started with mortality and selected causes of mortality on a quarterly basis.

It was followed up the following year with releases on natality. So you had these data visualizations for both natality and mortality that were being provided on a quarterly basis, which is what I think sort of what led to late last summer, the interest in having well, what about monthly for the drug overdose.

All of this work – this is all happening at the same time that the survey programs are both working on the release of the information itself, as well as planning and continuing the ongoing data collection activities.

It is a continuous cycle. At one point we were asked recently, to provide a flow chart of the linear activities related to the data collection. Every data division was saying you can’t think of it as a linear. It is for constantly cycling through if we are releasing data from one year, we are also in the process of planning the ones that are two years out while we are collecting the one that is current.

I was mentioning the files that had been released into the research data centers. NCHS has its own research data center, but in collaboration with AHRQ and SAMHSA, NCHS, there is a research data center that is in the Parklawn Building, to provide one location for SAMHSA, NCHS and AHRQ, all have their data there available for researchers. In the Humphrey Building there is also a NCHS research data center.

The bigger effort has been over the years, the work that the Interagency Council and Statistical Policy, the ICSP, the membership consists of all the heads of the statistical agencies, and it is coordinated by OMB. There has been a lot of work over the years to find a better way to have more one-stop locations where people can access data for research purposes. Part of it was to not only encourage more use of federal data, but just to think of better ways of using the data, better demonstrations of linking data, trying to encourage more agencies to participate in the data centers.

So there are four main federal partners; Census Bureau, Labor Statistics, NCHS and AHRQ, Agency for Healthcare Research and Quality, have been partnering for years to work on this. The ICSP coordinated a lot of work to change the Census Research Data Centers into Federal Research Data Centers.

They actually, I think it was 2015, sort of established, this was approved by OMB, they wanted to have it done. There was a governance structure, and executive committee, plans and operations for the procedures and processes for how anyone would be interested in applying to do research at these FSRDCs. It has been very successful. The number of projects have increased over the years.

One thing I think is important for NCHS is that of all the projects, at least in FY17, towards the end of the year, almost 40 percent of the projects that were in the FSRDCs – people were thinking of them as Census Research Data Centers – about 40 percent of the projects were using NCHS data. So I think that was important for the Center to appreciate.

There are more that are opening up this year. Within these FSRDCs, the data that are available incudes data now from Bureau of Transportation, Bureau of Justice Statistics, The Energy Information Administration, EPA, and others. There is a whole-host of data. Some of it is partly to encourage greater creative thinking for research uses, administrative data for statistical purposes, and I think the interest from this Evidence-Based Policy Making Commission, and what might be coming down the road from that.

So there has been a lot of work and a lot of push and a lot of collaboration, but they have been very successful.

THORPE: First of all, I think that is wonderful and I applaud the efforts. One thing, can you talk more about the proposal process and is there a way to streamline it? I find the proposal process in order to get it, it takes – I know on the NCHS website, last time myself and a student were doing it, it was taking six to eight weeks to get the proposal approved just from NCHS.

Then I partnered with some people at the University of Maryland College Park, and their process seemed to be more streamlined, but they are one of the new FSRDCs. Could you talk about whey there is a difference? If the Interagency Council is overseeing this, why there is a difference in the application processes for the same data, or tried to get the same data or tried to get NCHS data.

QUEEN: I am going to be thoughtful in how I respond because actually I was looking at this myself. Each of the main partners for the proposal process, has its own process. To find the FSRDC you type in FSRDC and you get Census. It is operated out of Census. The links have, here are the processes. It has NCHS and AHRQ and Census and Bureau of Labor Statistics. Each one has its own. I don’t have a full answer for you but I have what I suspect to be a partial answer, which is each of these agencies, because of their own concerns about confidentiality of the data and how they protect data, have their own processes that are probably very similar to each other, but it is not a one-place where you can submit your proposal and have it be a streamlined approach. That is what I am guessing.
MAYS: Could I just ask, in those Centers, would one be allowed to link any kind of data? I know in terms of the Evidence-Based Policy Making Committee, they had a lot of goals. So if you have the secure data center, is it possible now for example, to make a request of AHRQ and NCHS or Census and AHRQ and NCHS. So that we don’t need a data set to be formally linked but we can ask to link any data set. Is that the case?
QUEEN: What is the distinction?
MAYS: The distinction is there are some linkages that exist and so you can use those. Sometimes you may have to use those. I would be told that MEPS and NHIS can be linked. If I came up with an idea and I wanted to link MEPS and the Census, MEPS, ACS and NHANES – if I had an idea of what could be linked, do I have the security such that I can propose that or is it only what is currently linked that I could use?
QUEEN: No, you can propose it. Actually, one thing that NCHS does, and I would assume Census probably has a similar process, there is enough information that is available in Public Use Files that people are encouraged to before going to the trouble of putting in a proposal or trying to request the link, so see how many cases you might get. What are the number of people that would have such and such characteristic? What are the number that might be able to be linked?

I know that NCHS has published and has available online how many cases have been linked with Medicaid or Medicare? I was actually looking at the MEPs HIS potential linkage, because AHRQ makes available the code for linking the file and it tells you how many cases that were available the next year. Year X has this many people that can be linked between the two surveys.

You can certainly request that. People have also been known to bring data – there are certain restrictions about it – but bring data in and have that linked.

MAYS: One other question. In terms of the rules that you are using for confidentiality, has there been a change over time in that or are they the same? Is it once you get into RDC you can do things that you could not do before. Or even in the RDC are you going to have the same rules you would have if you were outside the RDC about suppression and things like that?
QUEEN: There are a lot of rules related to the RDC. If you are at a Census – the FSRDCs, they have confidentiality and privacy restrictions under various things. You have Title 26 for IRS data, you are going to have NCHS 308(D), you are going to have the Privacy Act, so is it Title 13 for Census? Everybody has their privacy and confidentiality restrictions. Many of them have the same kinds of components of not being able to disclose or release.

Much of what is in the Data Centers will be restrictions on what you can take out. You can run data. You are not going to be able to have an external connection. You are not going to be able to print. There will be some kind of a review prior to releasing data.

There will be a disclosure review process that is done at any of these Research Data Centers before anyone can take information out. To my knowledge, it doesn’t restrict you to whatever you want to run.

I think Jennifer Madans has mentioned recently somebody wanting to run something and they were informed that they knew in advance you would not be able to publish this because – yes, and the researchers still wanted to see it. Still wanted to do it anyway? Okay, you can do it but you can’t take it anywhere or publish it.

As to changes, in 2015, in February of 2015, NCHS had a – at least they published a new policy. I don’t know the extent to which it was a dramatic change, but on the potential changes to and restrictions to data, were previously, I believe, files were being provided to people and it was determined that that was no longer an acceptable practice. It is an increasingly difficult challenge to insure confidentiality of the data. It is harder and harder now to release Public Use Files.

The disclosure review process has become much more complicated. There is a consideration, not just of what might be identifiable from whatever file NCHS is releasing, but I know our Disclosure Review Board is considering what other files are out there. What is on those files? What do people have to do to get that information? All kinds of approaches to considering what is the risk for disclosure for free identification.

The restrictions, the only thing that I know that has dramatically changed, if it is a dramatic change at all, was that there used to be some production of, I think, special files that might have been made available that no longer are. Now you have to do it in a Research Data – Center. Vitals is different – so this is for the surveys.

PHILLIPS: On a very quick related note, Susan, thank you. I understand that there is an opportunity for more global risk assessment for all the data holdings within the FSRDCs. Is there also an opportunity for best practice application?

So if the Census has ways of either deidentifying or suppressing data, is there a movement to spread that across the other data holdings and FSRDCs with the goal of making them more accessible?

QUEEN: NCHS also has various methods for protecting the data for public use. In the Research Data Center, it isn’t necessarily suppressed. It would be your findings. So for public use there are different techniques that are used across the federal statistical agencies, nobody says exactly what they are doing – so I am not sure what you are asking about is it Research Data Centers?
PHILLIPS: No, it is more about what the consolidation under the FSRDCs. I did not know if there was a level up where Census might be consulting with NCHS on look this is how we handle files to protect them. You might want to follow the same process. It will aid with linkages, but it also aids with preventing exposure.
QUEEN: I don’t know because the federal statistical – that is something that I could ask but there has been so much collaboration over the years, all of the statistical agencies coordinate and collaborate enough through the Federal Committee on Statistical Methodology, that I don’t know if Census – I think there is a lot of partnering and discussing and the ICSP is a forum in which some of this happens. The FCSM is the other. But I don’t know to the extent to which Census is suggesting we have a way of doing this that we think is more appropriate or would be better.

Part of it is the methods – a lot of the techniques that are applied are very similar. In some cases you have data sets that are more unusual – it is hard to apply. You don’t have like a one size fits all for any of these. Which I am sure you are aware.

MAYS: Susan, can you say whether or not the Federal Statistical Agency, do they have their minutes available? They meet on a regular basis and this is some of what they do and I know CSTAT at the National Academy of Medicine, relies on them quite a bit.

Is there a way that one can start to see what discussions they are having?

QUEEN: I don’t know if they have minutes. That is something that I can certainly ask Charlie. Part of it, at least the ICSP, is not a public meeting. I can find out.
MAYS: If they have minutes, that would be great.
QUEEN: I think I have to go to some slides. It has been a challenging time. I did hear recently that HIS, they have some good news. They are looking at – at least not necessarily a further decline for the most recent year – but everyone is looking at how we can improve things. How we can do better with contacting people; outreach, reducing burden, et cetera.

Go to the next slide.

See the bad news. Then I guess the next one you can see with NHANES. There has been a lot of effort – NHANES is working to find alternative approaches from taking doing electronic screening versus having someone at the door- offering ways to do screening. They are exploring a variety of methods that they are testing. Having outreach in communities. Having more video and other kinds of media outreach to try and bring awareness to participation in the survey. You can see it is a concerning decline.

Next slide. I put this up just because all agencies are experiencing problem with budget and personnel, but you do see there it is a challenge. Everyone is being asked to wear many hats. So we are facing a lot of challenges going forward.

Next slide. What is new is that the last quarter of 2018 is when we are going to be doing the split sample test. In 2019, fielding the redesign. That is going to be a significant change for HIS. The expectation is it will be more streamlined and that it will be something that will help both with response rates, but also with relevance.

For NHANES, in January of this year, it was determined that there was a need to potentially change the growth charts to add the percentiles for severe obesity because the current growth charts don’t allow tracking those severe obesity.

Also NHANES, late last year, August last year, put out a Request For Information on new ideas, innovative thinking, thinking outside the box for NHANES for the sample design in approaches to getting the data.

NCHS has been putting out some very relevant analytic guidelines and methods reports. The two that are coming to mind that recently have been published, we released Analytic Guidelines on Trend Analyses. Which just came out, I think last month. It is very valuable. A lot of years of work went into that.

Then another report came out earlier looking at the time lag of reporting the suicide deaths, to look at the week of the death and when it was reported, to better understand suicide is another form of death that has a greater time lag. Certainly, it was found that it was certain characteristics of the death itself. Firearms deaths were reported faster. Drug overdose suicides had had a greater lag.

There is a lot of work going on despite the challenges related to either the budget or personnel. People are doing a lot of hard work and I think there is a lot of dedication going on at the Center, I will say that. If anyone has any questions, please let me know and I will take them all back to Charlie.

(Laughter)

STEAD: We have had a rich set of questions as you have gone through this. Are there others or are we at a point – this conversation has actually been a nice setup for the next block on Data Access.

Vickie, do you have a question before we make that switch?

MAYS: Kind of question and a comment. First I want to thank you for coming and the work that you did to really put all this together. It was very informative.

For me one take away that is there that I want to think more about is the way in which the survey participation is dropping. We spend some time here this morning talking about data stewardship, or was it yesterday, and talking about issues of trust, et cetera. I kind of think maybe we should be talking, us, not you, but talking about whether or not there is some data stewardship kind of approaches that can help to increase the trust of the nation to participate in the surveys.

I think what you have on the other side is what you have is Google and every time something like that happens it makes it more difficult for you, the Agency. So this is very serious because you are also under a Congressional Mandate that your surveys have to come in at a certain level. You kind of have a bit of a crisis, I think. I think maybe we should try and think about ways that we might be able to help. Particularly building from the data stewardship framework.

QUEEN: Your point is well taken. One thing that the NCHS surveys face that many other surveys don’t is all of them are household interviews. People have known for years the gated communities, other places like that, places that are harder to get into to begin with, that makes it more difficult. So that is sort of another twist to it, too. Telephone surveys are experiencing similar things, too.
STEAD: Susan, thank you. Awesome update. Amongst all the supports of the National Center, we are clearly in that camp so thank you.
QUEEN: Thank you. I will find out answers to the questions you were talking about.
STEAD: If you have time, love for you to stay for this next half-hour, where we are beginning to talk about the data access issues. We are really trying to think through how we scope the possible work in a way that would be most helpful. So we would love to have your engagement in this conversation. Bob and Vickie will lead it off.

Agenda Item: Improving Access to Data

PHILLIPS: Susan, thank you. I almost wish we were talking about the Access for Federal Data Resources, not Access to Federal Data Resources. I wish we had some more help for you.

We are doing a couple of things. We are doing an update on what we kicked off at the last meeting in terms of Access to Federal Data Resources. Then also looking for your feedback in terms of our scoping and the work that is going to take place over the next year probably. I am intensely grateful to Vickie Mays, in particular, but to Roland Thorpe and Lee Cornelius, Bruce, Vickie Boothe and Rebecca Hines, for the development of the scoping document that you have in your agenda book. To Denise Love and Vickie for huddling yesterday over the slides that you are going to be seeing.

I say that to say you are going to have a little discrepancy. The scoping document is a little different than what I am going to present in the slides. That is just because there as been an evolution in the last 24-hours of what we are talking about.

This effort started with a couple o things. One was the Commission on Evidence Based Policy Making. The other was concerns that came to the Committee directly from stakeholders about loss of access to some data – federal data resources.

We started last meeting with a discussion of four health data systems in particular; the Community Health Status Indicators, the Health Indicators Warehouse, Behavioral Risk Factor Surveillance System, not that we have lost that but roll-ups in some of the statistical modeling for it, and then the Health Data Interactive.

They are not particular sources, other than BRFSS, most of them are aggregation and information systems to access an array of data elements from federal resources.

Then we have this issue that we just talked about a little bit, and we talked to Rashida Dorsey yesterday, some discussion about movement of data into the Research Data Centers and how it has affected access to some of those.

I am going to let Vickie talk to this a little bit more if you want.

MAYS: What would you like?
PHILLIPS: Are there ways either through technologies or statistical methods, that we can address some of the concerns about protecting what have been Public Use Files, but now are not public. Susan, thanks for talking about some of the reasons why they moved in.
MAYS: I guess the issue is as we get constituents coming and saying, it is harder, and this is at the community level. I don’t think we are talking as much about the researchers now, but the community level, like the change with BRFSS.

For some groups, if there is data on a website and you can use that data on a website, then you are able to write a proposal much better for state and county kinds of smaller type grants. So as less of that is available, people are feeling they can’t do what they have done in the past in terms of being able to make requests of local groups to fund very specific things.

Instead they now either need a scientist to come in, a researcher, or somebody, to help them to now access a more complicated data process. I think that is what is being raised here. I think there are questions about well, is there some kind of way this stuff can be rolled out again for them to have it. Particularly losing the data at warehouse. Are there ways in which technology can make it easier? Those are some of the questions that we are raising.

PHILLIPS: The last piece in this, and I really thank Denise and Vickie for helping me understand, is this issue of assessing data risk versus importance of the issue. In particular, around opioids, it is the idea that when there is a sudden epidemic or a need to access data, is there a way to assess the risk of exposing the data to public use versus their value for utility?

I think we have seen some of that in what you presented earlier, but that balance of value versus risk with certain data elements.

MAYS: Can we ask Denise if she would talk about that particularly in terms of front a state perspective?
LOVE: My perspective is with a single data source of public state data bases. We have vendors that do this but there is a risk score, a tolerance score, that states can apply and if they tolerance is very low then the methodology will either take away data elements, suppress them, aggregate them. If their risk tolerance is a little higher, you can give a little more granular data. It is kind of a risk tolerance within the legal framework.

We are seeing, like Vickie said, more need for more data at the local levels. So for instance, Part 2 is a recent experience that we are working with states because SAMHSA is very restrictive in how that data can be released, but we think we are going to have some success in doing a state research extract request that templated so that states can take it and get IRB approval. But it is more relaxed than if an individual requestor was doing that. We have done it with CMS as well.

I don’t, you know, there is no easy answer, but I just don’t know that many state users or community users, that would even use the Research Data Center for some of their ongoing needs or daily needs.

MAYS: I think one of the issues is whether or not we can think through this issue about the risk assessment and whether or not, depending on how compelling the issue is, either the population hasn’t had data and it needs it – something about American Indians. It is like the need is so great like with opioids or HIV, that we suddenly say, there is a compelling – like in IRBs, there is a compelling interest here that needs to be met and that the benefit will outweigh the harm.

Beginning to think about is that a way that we can get more access to data by balancing risk and harm, as opposed to everything is the same. Whether the problem is one where people are dropping dead or the problem is one where we have a lot on that – not a big deal. I think that is one of the things we are thinking to think more about whether that might help in this balance of making data accessible.

QUEEN: For the statistical agency, it is the requirements for confidentiality. You can think of that in terms of harm but the assurance of confidentiality that has been made in wanting to – there is the value of the information that is needed for addressing an emerging issue, for example. Then there is balancing that against the risk of the disclosure.

The risk assessment, you see some examples where it is like a formulaic process, but at least with some of the data sets that we have had, it is not that easy. We understand there will never be risk o zero. The question is trying to determine what is the likelihood that with this and this and this, you would be exposing the potential for reidentification.

The areas that are the most likely to be removed from a data set are going to be geography and dates. Before 1989, NCHS, the Public use microdata files for deaths included exact dates and counties. After ’89, it changed to, I don’t know, not exact dates. Maybe year – I don’t know the extent, so I don’t want to say.

Then counties and other areas of 100,000 more. Now I think it is more restrictive. It is just increasingly a challenge and you do have the concerns of wanting to make sure that you are abiding by your assurances that you made to those that have provided the data and the need for being able to use the data.

LOVE: And it is really at odds with the geospatial and we are trying to get at sub-county levels. There is a need to do that but there has got to be a solution. I don’t have an easy solution here, but the epidemiology needs to get down to the neighborhood level, that is actionable.

National or state level figure is not actionable for the communities and the providers. What are those trade-offs? If you want a geospatial data point, what else can be stripped and what is how low can you go? Because, when people say to me, well, if it is five cases and you put it on a map and then someone says, well, my mom might be one of those cases, I say to them, but you already knew that. You did not have to go to the map and say, oh, let us dig through other data sources. It just confirms that your mom might be one of those five.

I don’t think in the data point alone, without other data elements, the data released and pinned pointed are robust enough to identify a unique. That is where – how low can we go and what is that risk tolerance because we have to figure something out.

PHILLIPS: We may need some use cases to play off against the confidentiality of agreements to see what we can do.
MAYS: But it sounds like that when you collect the data with this the assurance of confidentiality, that then you can’t go back and undo that with risk/harm benefit. Right?
QUEEN: No, you can’t go back and undo it.
PHILLIPS: There may be spatial ways to protect the data.
QUEEN: There is data swapping, there is perturbation, but some of what you are talking about too, is if you have a survey that has all these characteristics about an individual, there is much more information that could potentially be
LOVE: And surveys maybe unique from discharge data because that doesn’t have as much, and we don’t have that patient consent component. But you do have the issue of you don’t want uniques in the data sets that people could back into.
PHILLIPS: This exchange is very helpful. Roland, did you have something you wanted to say?
THORPE: Not yet.
LOVE: one more question, I know NCHS uses swapping and some of those other things, depending on the application, so you could release peer county – like this county over here we are going to swap because it is very much like this. Then it mixes the identification issue up a little bit.
QUEEN: The data source that we have that would go down to that level would be vitals. Vitals are different. They really are. There is the agreement with the states. That is something that I think that NCHS would have to really be in negotiations with NAPHSIS. I told somebody at work, I found online that New Hampshire released some information that was quite a detailed level. But that was their purview to do that. NCHS can’t do that, but they did it. So states can do with their data certain things that NCHS would not be able to do.
LOVE: That is why I get involved because there is such variation. Why can New Hampshire do it and not this other states? I get asked that all the time. There laws are similar but they apply their policies very differently. It is just a big problem.
PHILLIPS: Denise, let’s hold onto that. Let us not lose that because in terms of our Committee thinking about this, that may become a solution as a difference maker.

Can we go to the next slide really quickly. I wanted the Committee to know that we did field queries out to a series of stakeholders. Very simple query about what data are no longer available, either reliably or where they have lost access. Then how were the data being used before? What decisions or activities do they support and whether there are commercial interests that are affected?

We sent this out directly probably to 15 to 20 different groups and then snowballed through some of them, I have no idea how many people they reached. I continue to receive feedback from folks that I did not reach directly, including today.

I just wanted to give you a sense of some of the feedback that we have gotten through this that I think helps us frame a potential panel discussion in the fall.

APHA highlighted specifically, CSHI and HDI. I will make these available to everyone. Then walk through how their members are using these for research. How pubic heath departments are using them, and what the impact has been for them not to be able to have those data reliably in the way that they had before.

University of Missouri CARES, who supports a number of efforts with CDC, with US News and World Report now, talking specifically about the National Notifiable Disease Surveillance System, which had been part of HIW at the county level. Then the BRFSS roll ups.

Then specific data elements within those and how it is affecting their ability to continue to support some of the services that they do, which is actually on the next slide.

Specifically, some of the community health needs assessments. At least the Missouri Extension Assessment Reports. A number of states and health systems with their CHNAs, in particular. It has gotten a lot more difficult because of the data element loss.

The Society of General Internal Medicine Research Committee, and then today, the Health Policy Committee, actually reached back to me to let me know that particular problems that we had not even listed initially in the fall. So with the Research Committee, it is about the fact that risk adjustment data used for Medicare Advantage plans had initially been offered and then pulled back.

Then this article in JAMA, just recently, by Austin Frakt and now Brennan, at the Healthcare Cost Institute, noting the access to Medicare Advantage Plans. Thanks to Bill letting me know some of this came up at Datapalooza, I went and looked and actually it was announced that these will be accessible, it just isn’t clear when.

So a completely different framework set of data elements, but a particular stakeholder group asking us for help.

Next, please. The Community Hospital Corporation, the Catholic Health Association actually reached out to their network and that is how this came back to us, but again, around community health needs assessments, and particularly CHSI data elements. It is really getting down to peer county data for hospitals to be able to compare their counties in thinking through what they might do in terms of their community needs and their three-year implementation plans, which now are a requirement through IRS.

Next, please. Then a proprietary group, Trilojy Integrated Resources, who actually support now, 27, soon to be 29 states, 250 local public health care sites that they funnel data to. 256 local health districts in 20 states that they are not only providing them data, but they are actually creating websites for the states to use and counties to use. And pointing to 219 different data elements from the National Vital Statistics, particularly around mortality and natality. Again, the BRFSS roll ups and the ability to drill down to county. The Chronic Condition Data Warehouses, being elements that they have lost and actually providing an entire spreadsheet of the data elements that they are struggling to get and having to work back through.

Even quantifying now that they are spending about $250,000 trying to rebuild their application to respond to the new data schemes where they can access the data but it is a much harder enterprise for them to do it. It is about reliability and timeliness of the data and those elements that they can get.

I have a spreadsheet going. I have all these comments. The slides are available so that you can dig through them if you really want to. But I want to give you a flavor of kind of the responses that we are getting back. What I would really like to get is get your feedback because we have got about 12 minutes on next steps.

Next slide please. This is even more updated than what you see in the scoping document. We started the preliminary fact finding and getting some perspective from data holders, but we feel like there is a need do get some more meat on table, a little framing of the specific elements and the use cases.

So talking about potentially having a panel conversation in the fall, where we will bring some of the folks that have given us feedback and maybe some others, to talk about the problems. I think this discussion that we just had suggests to me that we may need some folks on the panel that can talk about potential solutions we have not thought of.

There is a conversation that Vickie and Denise, in particular, helped me think through in terms of an environmental scan. Which was not part of our original plan, but the conversation that evolved was not just that we may need one but it may be when do we need one. Depending on whether it helps us set up the scope of the problem or is it more at the end where it is more about the scope of the potential solutions.

That is where we would welcome some feedback. I certainly welcome Vickie and Denise talking more about that particular piece, because our goal is eventually at the end of ‘18 and into ’19, to start to develop some recommendations at different levels of the problem scope and the solution scope.

MAYS: Let me just comment a little bit on the environmental scan. I was kind of in we should do that first, even though we are getting valuable data.

Part of what I think we need to be able to do is to know the landscape by which we are trying to offer suggestions and solutions. It is a complex one. I think at the end of the day I don’t want to write a letter to the Secretary where it says, these are the problems. But to be able to write a letter to the Secretary that says, here are ways to look at changes that can occur.

I don’t think it is going to work for us to just say, BRFSS needs to just be available. These things just need to be there. It is complicated why things have changed. I think for me the reason I think we need an environmental scan is I don’t think we understand that yet. I think we are more helpful to the Secretary if what we can do is to then explore those barriers and to figure out I think, better ways to cope with that.

It is not about money all the time. It is sometimes about regulations. Sometimes it is about do we need a will with these states to get the states to do something differently. I don’t think we know that stuff yet. I like these examples. I think these examples are like, oh, my god, we have got to do something. I guess I want to see us know our landscape and know what to do.

KLOSS: Maybe this has been contemplated, but is there a preliminary step here of having kind of a fact finding by working with the Data Council?
PHILLIPS: we had an interaction step in here.
KLOSS: How has the policy changed?
PHILLKIPS: We had an interaction step in here that involved some interaction with the Data Council and got some feedback that that might not be useful at least early on. We would love any feedback that might change our direction on that.
MAYS: Statistical agencies, too.
DORSEY: Rashida Dorsey. I would certainly welcome having some time on the agenda to have someone from the Committee, from the Subcommittee, talk a little bit about what the idea for this project would be to the Data Council. I think it could certainly help you with framing and scoping this project because if the Committee is going to develop recommendations for the Secretary, have an idea of what might be most useful for HHS, in particular the things that are under our purview, would be helpful to know. That discussion, I could certainly facilitate that clearly prior to this meeting.
PHILLIPS: Rashida, could we do it the other way, where we go to the Data Council?
DORSEY: That is what I meant.
COUSSOULE: I don’t know how this could be productive without a bit more back up detail on environmental scan at first. We say it doesn’t always come down to money, sometimes it is money and sometimes it is regulation, but people are making decisions about priorities on a regular basis. People aren’t making dumb decisions on purpose. People are making decisions with the best information that they have and it may not be enough information or they may not understand the impact of that. Even if they have to make it, it may not be easily communicated to what it needs to be.

I do think that there is fact finding or back-up information on environmental scan that will need to be done prior to even being able to scope a recommendation set.

GOSS: I was just going to make one little comment, but I think Roland was before me. I think the point that Nick is making is really important and I think the environmental scan – I kind of feel like we are doing in the TNV Model or approach, is really working. You are using that opportunity to convene your stakeholders. I am not sure we really mean the word “hearing” in that Q3 bucket. We may mean roundtable or forum.

I love the idea of being able to get the subject matter experts to validate where your environmental scan landed and then be able to use that as a solidification on what you build.

THORPE: I don’t need it. That is exactly what I was going to say.
COHEN: What I was going to say, I agree it should be a roundtable or workshop, not only for the data users who have found they no longer can access the data, but it would be great to also have the stewards of the data to provide us with information about the decision-making process about why the data are no longer available.
QUEEN: I would recommend having more than one, like a person from RDC, like Peter Meyer from NCHS, and perhaps somebody from another statistical agency. Whether it is the Census RDC. I know it may be a little in the weeds, but having someone from a disclosure review board, like a chair of a disclosure review board, say when they are looking at the files and whether or not they are going to release them, what it is that they are doing because at least with NCHS, we view our Disclosure Review Board as an independent entity. There is a lot of back and forth and a lot of saying, so you can identify your mom but you can’t identify her with certainty. There is a lot of decision making, but we respect the decisions of the Board ultimately.

It would be valuable to even hear their approaches. I am sure that they are different across agencies, too.

LOVE: Maybe a state data oversight board, too, because you may get similar concerns but also maybe different approaches.
STEAD: My challenge remains what I think of is pre-work around sufficient definition of the scope that you would know how to charge environmental scan. I think we have had a rich conversation and what that is doing is informing a very large potential landscape. If you play back to the journey that we have been on with TMV or Beyond HIPAA, and other things where we have done an environmental scan, we have started with these kinds of discussions and used that to frame how to bring together a draft of what we think the scope is. Actually, some pieces of the environment – almost becomes a version 0.01.

Then we have used that to pick the people to bring to panel here, in a substantive block of one of these meetings where we get some depth and different snippets that we think are going to be very important.

That then let us refine the scope and the charge to the environmental scan. What questions are we trying to get an answer? I very much like where you are headed but my own gut is we need to do more work. Maybe if we work within the Subcommittee and the Work Group, because you have now got some fairly active people in this work group, you have got broader participation through your run-up to this meeting, that maybe we could have that panel be a one-and-a-half-hour block – possibly up to three hours, if we wanted two panel. My guess is one panel the first time, in September, that might let us then really have a clear scope, a clear charge, do the environmental scan, and then figure out whether that sets up a hearing.

I sense we are one step early to contract for an environmental scan.

KLOSS: I think going back to the goals that your developing. It seems to me that this work falls squarely in the overarching charge to this committee, which is to develop a policy framework. What might come out of this is what that HHS Data Access Policy should look like, including stewardship. That is something that I think could be short-term, but it could have some long-term lags. It just seems to me that what is happening is that there is a vacuum in that policy and what this Committee could be working on is framing something going forward.
PHILLIPS: Thank you, that is very helpful. Some of what we are talking about touches on the other committees, as well. At some point we will want to invite you into that space to help us figure that out too.
HINES: Other subcommittees here?
PHILLIPS: Yes, other subcommittees.
STEAD: Linda just triggered another thing. When you think about a policy framework for data access, a key piece of this panel that might be in the fall, would need to be Mona Siddiqui because they are in the process I believe, of building such a framework. The briefing I heard at Datapalooza implied that work was aggressive, and that the majority assessment would be done this calendar year and the governance framework would be done in 2019, implementation would take longer.

But my sense, and I am looking at Susan, my sense is as a train moving, and if we don’t align what we are doing with that train moving, it is not likely to be helpful.

DORSEY: I would just say that if that is what you are thinking about, just so the Committee is clear, that the data governance piece, it is a part of Reimagine that is being led through the Office of the Chief Technology Office. It is for HHS governance.

If the thinking for some of this work is how to provide more access to states and at the community level, that is not the scope of what the work of the Reimagine team is doing with data governance. Just as an FYI.

So if you are thinking about developing doing some work that would be aligned or in some way maybe useful as part of that effort, it is happening pretty quickly. Just have an understanding of the scope of that work, which is a little different from what I am hearing that you guys are proposing.

STEAD: You are much more likely to be on the ground than I am. What I have described is what was said. That doesn’t mean I heard it right and it doesn’t mean it was said right.
DORSEY: No, no. If that is what you are thinking so you understand it is really for HHS.
QUEEN: I was going to ask you Rashida, what the governance is for would include access to data externally, from others accessing HHS data?
DORSEY: I can confirm this but I have had some conversations, starting with CTO and Mona Siddiqui, in particular, the primary focus is to facilitate sharing across HHS and to break down silos across HHS. That may ultimately help other forms of access but that is where they are starting. That is why I am just sharing whatever this group is thinking, just so that you have an understanding that the work of the CTO under the Reimagine efforts, which I talked a little bit about yesterday, is really designed to facilitate sharing across HHS.
PHILLIPS: Rashida, I would ask for your help if we can, in figuring out if there is an external piece to it that we should be plugging into just in the next few months.
COHEN: Or at least aligning these.
MAYS: One of the things I wanted to say is I think what I have been arguing for in this group is that there are the federal efforts and then there is the community side. I think we need to – because the solutions are not always the same. I think if we are going to do alignment, then we need to be clear about this is a federal issue, this is national data. Because I think, and I had written down, one of the issues of why did you do this is Jim Sorace used to say this all the time, we need to be able to use our own data better. We actually have to pay people outside to help us use our own data.

I think we want to be able to help but I think the alignment, as I understand what Mona is doing, is it has to do with HHS being able to use HHS’s data, as well as HHS being able to have the ability to share among themselves. This is my understanding of what I have been told of what is going on.

So I think that is a part of what we are doing, then we have another part. I think the alignment is necessary. I agree with Bill, that train – they are going on that train. But if we want that, then we should bring them everything. We need to do the piece that aligns with them.

PHILLIPS: Thank you. We are out of time, but the feedback and the help has been incredible. I think I have a better sense of where we need to go.
STEAD: Thank you very much. We will now break for lunch. We will be back at 1:30 p.m.

(Recess For lunch)

AFTERNOON SESSION

Agenda Item: New Medicare Card Project/SSN RI Monitoring Update

STEAD: We are ready for a touch base on how the new Medicare Card Project is going since the cards began to go out to new registrants at the beginning of April.
GOSS: DO we have Monica Kay on the phone by chance?
KAY: Yes, Monica Kay, along with Lois Ferriol(phonetic), with other CMS colleagues, are on the phone.
STEAD: Outstanding.
KAY: Are we ready to proceed?
GOSS: I think we are. I think we are all getting orchestrated here and we’re glad to know you are on the phone with us. I think we want to do a two-part approach to this. One part is to get an update from CMS on their new Medicare Card Project, and the second part would be to take comments and questions and observations from committee members based on what they are hearing from their respective parts of the world.
KAY: Great, it sounds like a plan. I will go ahead and get started. My name is Monica Kay and I am from the Office of Information Technology. I am going to bring you an update on the new Medicare Card Project. The first thig is we are now live. We have started mailing out the new Medicare cards with the new Medicare beneficiary identifier, or MBI who are beneficiaries.

We have also started mailing, if you go to our CMS.gov site to our source page. Just to remind you about our timeline of events, again as I said, now April first through 2019, December 31, 2019, we wanted to ensure that all systems and processes are ready to accept the new MBI and that we have started mailing the new Medicare cards to our newly eligible people with Medicare, as well as our wave one of card mailing.

We also began our transition period, which is a 21-month transition period which will allow people to use either the health insurance claim number or MBI for processing. Again, as we continue to do our mailing throughout that time period, we expect that June of 2018 will be the launch of our provider look-up tools, where you can begin searching the MBIs if you need to have it from the provider perspective. We will also begin returning the MBIs of remittance advices in October 2018. Of course, to meet our deadline for April 16 of 2019, which is our deadline for issuing those new Medicare cards.

Then, as a reminder, January 1^st, 2020, will be the end of our transition period where we will no longer use HICNs on most of our data exchanges externally. But we will be using it for internal purposes.

Again, a couple of reminders about the solution concept for our new Medicare cards. Our plan was to generate our Medicare beneficiary identifiers for all of our beneficiaries. We also plan to issue our newly redesigned Medicare cards, as well as modify our business and systems processes to accommodate the receipt, transmission, display and processing of the MBI, all of which we are doing right now during our transition period.

Again, as a reminder of some of the new Medicare number characteristics, it is the same number of characters as the current health insurance claim number, which is eleven, but it is suitably distinguishable from the health insurance claim number. It contains the upper case alphabetic and numeric characters throughout the eleven-digit identifier. It occupies the same serial as HICNs on transactions, and the biggest point of note is that it is unique to each and every beneficiary. A husband and wife will have their own MBI. It will be easy to read and we will exclude characters, or have excluded characters S, L, O, I, B and Z. It does not contain any of that intelligence, special characters or any inappropriate combinations of numbers or strings that might be offensive. Again, in terms of MBIs, they can be changed if they have been compromised or deemed to have been used for fraudulent purposes.

Let’s talk a little bit about the transition period. Again, that is April first, 2018 through December 31^st of 2019. Right now we are mailing cards out to our 6 million beneficiaries. That is due to complete in April of 2019. From April 2018 through December 31, 2019, we will both accept and process HICN and MBI on transactions. As of January 2020, those HICNs will no longer be exchanged with beneficiaries, providers of plans with some limited exceptions.

I will talk about using that new Medicare number during our transition period and some of those instances for our different areas. Again, the CMS will accept usual processing and return to stakeholders, either the MBI or HICNs, whichever is submitted on the claim during the transition period. Any stakeholders who sent or received transactions containing the HICNs should now be free to submit or change the MBI. CMS is actively monitoring the use of those health insurance claim numbers and MBIs during our transition period to ensure that everyone is ready to use MBIs only by January 1, 2020.

Let’s talk about MBI usage as it relates to providers. The first thing is when a provider checks the beneficiary’s eligibility the CMS HIPAA Eligibility Transaction System or HETS, will return a message on the response indicating that CMS has mailed that particular beneficiary their new card. Then in October of 2018, as mentioned earlier, through the end of the transition period, when you submit a valid and active health insurance claim number on a Medicare fee-for-service claim both the HICNs and the MBI will be returned on the remittance advice. During the transition period we will process all claims with either the HICN or MBI, even when both are in the same batch.

Again, for providers, starting in June 2018, providers and suppliers can look at the MBI on the MAC portals. We share this information with all our providers in terms of how they can log on to the portal, what instructions need to be done, in order to effectuate the look-up of the MBI MAC portals. Providers and suppliers need the following beneficiary information to look up MBI’S. They will need the patient Social Security Number, the patient last name and first name, the patient date of birth, and first their provider and supplier NPI. Just a point of note, providers and suppliers can use any MAC portal to look up that physician MBI.

The key point for providers is that you should be ready now, and there is a 21-month transition period from April first, 2018 throughout December 31^st, 2019. Providers have three ways to get the new MBI. A patient can present it to you at the time of service; the provider can obtain it through a secure web portal with the MAC; or you can receive it through the remittance advice. And we have a lot of resources that providers can use to learn to help people and talk to people at Medicare about the card. And they can be found on our web site.

All Medicare Advantage plans receive the HICN to MBI crosswalk file prior to the start of our transition period. The MARC user interface will show both HICN and MBI during the transition, but only the MBI when the transition is over. Prescription Part D may submit either the health insurance claim number or the Medicare beneficiary identifier both during and after the transition period. CMS has provided information regarding the systems changes via our health plan management system. Memos have been released for the systems that are impacted.

The beneficiaries enrolled in Medicare Advantage plans, once they receive new Medicare card and number, they may use it to enroll in a Medicare Advantage or prescription drug plan. All Medicare beneficiaries who are enrolled in a Medicare Advantage or prescription drug plan will still receive an insurance card from their plan that they must continue to use when obtaining services while enrolled in that plan. Also, our E1 transactions for pharmacy, both the Part D, A and B, E1 transactions will return the MBI. Pharmacies may submit the HICNs or MBI during and after the transition.

The new Medicare card, as it relates to other stakeholders. For accountable care organizations, the accountable care organizations are receiving MBI information for their assigned Medicare beneficiary population. Both the health insurance claim number and the MBI may be included in the claims and claims-wide E-file starting April 20, 2018 through December 2019.

In terms of our crossover claims process, the health insurance claim numbers will continue to be used for crossover claims processing and will also have access to MBI starting in April 2018. During the transition period, CMS will process and transmit Medicare crossover claims using either the health insurance claim number or Medicare beneficiary identifier.

For our Medicaid agencies, are dual-eligible Medicare and Medicaid beneficiary. State Medicaid agencies are receiving both the health claim number and MBI in their system exchanges with CMS. And state third party buy-in dual eligible will continue to use the health insurance claim number.

For our private payer. For non-Medicare business, private payers won’t have to use MBI. They can continue to use the supplemental insurance unique number to identify their customers, but after the transition period, supplemental insurers must use MBI for any Medicare transaction where they will have used the health insurance claim number or HICNs.

For our third party group payers, third party systems will allow the submission of either the MBI or the health insurance claim number throughout the transition period. But again, after the transition period ends, third party payers must use the MBI for any Medicare transaction where they would have used a health insurance claim number.

For our remittance advices you have a number of information regarding remittance advices. For our Medicare Remit Easy Print, starting October 1, we will update the MREP, of that it gives you the MBI, submit a claim with a valid and active HICN.

With our PC Print, starting October 1, 2018, we will update PC print so that it also gives you the MBI when you submit a claim with a valid and active HICN.

We are currently changing the PC Print Admittance Advice table to MID and adding that MID-corrected label and field. Your MAC can give you access to free PC Print software so you can see and print that remittance advice information.

Lastly, for our Standard Paper Remit. Starting October of 2018, we will update the Standard Paper Remit so it is also includes MBI when you submit a claim with a valid and active health insurance claim number.

To see all this information and also get examples of all of the information that has been provided, please go to our CMS.gov New Cards website.

Let us talk a little bit about our Railroad Retirement Board Beneficiary. The Railroad Retirement Board will continue to send their cards with the RRB logo, but you won’t be able to tell by looking at the MBI if the beneficiaries are eligible for Medicare because they are Railroad retirees. We will return a message on the MBI eligibility transaction response for RRB patients. That message will say, Railroad retirement Medicare beneficiaries, which can be found on the 271, Loop 2110C segment method.

Medicare providers must program their systems to identify those RRB beneficiaries so they know to send those claims to the Specialty Medicare Administrative Contractor.

So in terms of the usage of the Health Insurance Claim Number after the transition period, there are a few exceptions. That includes claims, our appeals, our CMMI models, specialty models, as well as our premium payments, and certain reports, which include our quality reporting, historical reports, care summary notices, and ACO reports.

In terms of our claims, there are also exceptions for our mandate claims, for in-patient hospitals, home health and religious non-medical.

Of course, our information requests deals with our disproportionate share of hospitals, Medicare secondary payors, and any general increase that may be received, and of course, policy reporting.

If you need to see this or other, please go to our CMS.gov new card website, where all this information is contained.

Now I will turn this over to Lois Ferriol, who will talk about our card mailing that began –

GOSS: Monica, can I interrupt, this is Alix. We appreciate this update and the details of the transition. I think the point of today’s session, at least my perception was, that you were going to give us an update on how the actual transition itself, was occurring, what feedback you were getting, and we were going to solicit more especially, some points from our fellow colleagues at the NCVHS table, to help give you some further input about what we were hearing in our neck of the woods as it relates to the new Medicare card initiative.

I am not sure if Lois’ update is still needed or considering that we have about 10 minutes or less left for this portion of our agenda, I want to give you a chance to hone in on feedback that you are getting from your world and then give us a chance to survey our members, whom we asked to do some outreach in their own neck of the woods before they came to the meeting this week.

KAY: I would say this, let us hear from Lois around the card mailing process. I do think the information about what you need to do and what we are doing currently is vital for the conversation. So let me turn it over to Lois, who can talk about some of the highlights of our card mailing process and that will greatly influence probably a lot of the discussion today.
FERRIOL: I will keep this very high-level so we can get into the questions from all of you. Our new card mailing did start on April first, and that was for newly eligible people with Medicare across the country. We are now starting wave one and two, which are occurring this month in May. And we will update our chart as we go through all of the different mailings.

The Railroad Retirement Board will be issuing their new cards to Railroad Retirement Board beneficiaries shortly, along with the wave. Just keep in mind that beneficiaries who are newly enrolled or who newly enrolled throughout this process, will receive their cards outside of these waves. So it is important to know that across the country people are going to be receiving the card starting now.

We sent out the new cards along with a letter of instructions, on how to destroy their old cards. They also have a specialized communications for those with alternate format needs, and also the people with alternate format preferences are on the same schedule to receive their cards.

Some of our communications and outreach importantly, is that we have a medicare.gov new card campaign page. On that we are allowing and asking folks if they are interested, that they can get an e-mail for when their new cards are mailed to them, which we thought was a very important piece to give to the beneficiaries so they can type in their e-mail address and we will start sending them notifications when CMS starts mailing to their state.

Also, as part of our medicare.gov campaign page we have a map of the US that people will be able to go into and see how the mailings are progressing across the country so that they can know if they have received cards in their state or when they should expect them to arrive. Along with each wave we are having local media and radio tours, along with public service announcements, to highlight and alert the folks in their area that their card is on its way and please expect it.

We also have the education outreach materials on CMS.gov for folks, and most importantly, we have ways for beneficiaries to receive a card. Those who are signed up on mymedicare.gov on the dashboard page, they will get alerts of when the card is in the mail. They will also be able to view or print their card, so they can go in and click the link and their new card will show up on their mymedicare.gov. They have the ability to print that, to write the number if they haven’t received their card yet, if they need it sooner rather than the card mailing, if there is an issue. It is also going to be able to do that on their cell phones

We are just trying to reinforce the beneficiaries that mailing everyone new cards takes some time, and they are going to arrive at different times than their friends and neighbors. Make sure their mailing address is up to date. And beware of anyone who contacts you about your new card. So the mailings are moving forward and again, we just point everyone to our medicare.gov website for these new updates.

CORNELIUS: We thank you. We are going to move forward.
GOSS: At this point what I would like to do is ask committee members to raise their tent cards if they have any feedback they would like to offer to CMS on what they are hearing about the new Medicare card project from their neck of the woods.

Rich.

LANDEN: I don’t have feedback, but I have some questions. You did a superb job of presenting what you are doing as far as pushing out, but I was hoping to hear more about how that has been received. So three questions for you.

How many cards or what percentage of them have been returned as undeliverable? Second, what is the number of phone calls you have received from the new card recipients, and what are the key issues that they raise? And third, some sense of the volume of phone calls you are getting from providers offices calling for assistance with what to do when a patient presents a new Medicare card?

KAY: Thank you for those questions. Let’s turn a portion of the question to someone who can address the provider inquiry side of the house.
CHUCKNOT (phonetic): This is Rich Chucknot. Just to give you an update on providers. We are in the MAC call centers and we are receiving very few calls related to the Medicare Beneficiary Identifiers. For example, for the month of April, about 75 calls from providers across the country about MBI. There have been very few calls. With the HIPAA eligibility transactions, the 270-271 transactions we are seeing an uptick in the number of MBIs coming in on that transaction. About 1.2 percent of our transactions last week can take an MBI on that. We are very happy with the progress we have seen with the eligibility transaction.

The look-up tool for providers to be able look up the MBI based on Social Security number and name and the date of birth, that has been rolled out to a number of our jurisdictions, Palmetto JM, NGS JK and J6, WPF J5 and J8, CGS J15, and we are watching numbers of the look-ups and we are seeing very good progress on providers actually using that tool and getting back a good response in regards to the MBI.

So far, with the provider community, we are very happy with how things are working with regard to eligibility transactions, the look-up tool, and helpline calls coming into our call centers.

GOSS: Rich are you okay with that? You asked two other things. I see we have some other tent cards up and we have less than five minutes left on this agenda item.
LANDEN: Yes, if they want to respond to the other two either now or offline, that is fine.
KAY: Let me take that back and respond to that offline. Just to give you a general sense, is that most of the feedback has been positive to date.
MONSON: I just wanted to provide feedback. One of the questions that came up from the committee is to get feedback. So I checked in with my organization as well as others in Northern California, and what we are seeing happen is that the beneficiaries don’t know what to do with the new card. So they are showing up at our clinical locations asking for assistance, which is as you can imagine, very time consuming, and not something that we are equipped to manage.

The last time I checked in was just before I left for DC, and it is just overwhelming how many beneficiaries are coming into our clinical locations asking what the heck they are supposed to do with it and what it means? I think any kind of help that you can provide in equipping the providers with what they might be able to give the beneficiaries versus our clinical staff, who are supposed to be registering patients, are ending up spending los of time, one, trying to figure that out on their own and, two supporting the beneficiaries. So that would be very helpful. That is the same feedback that I received from other large provider organizations in Northern California.

KAY: Thank you for that comment. Can you please put out directions for our CMS new card mailbox? It would be helpful to have that information. Just a couple of points to note. There is a letter that accompanies the card instructing the beneficiary what to do and to start it immediately. The first thing that we would ask the provider —
GOSS: Thank you for that guidance. I think we understand that those resources are available, and I want to really make sure that the people sitting around the table who have done their homework and are trying to bring questions and feedback, have an opportunity to present that. So I am going to go to Nick Coussoule next.
COUSSOULE: Just one question. I am with a health plan in Tennessee and we have actually started getting claims flow through COBRA claims and then through the Medicaid operation. One of the concerns that I have are calls that are coming into our call center are not so much about the new cards, but there are some scams that are running where people are trying to get the to give up their information and claiming that they are for your new Medicare card, can you give me your Social Security number and some other information.

We have had a number of those calls into our call center. I wasn’t sure if you were getting any of that or any feedback from that around the rest of the country?

KAY: Again, if you could send that information to us we do have an office that are handling such those incidences. If you could contact us.
GOSS: We understand that request. I think we were thinking that this was a two-way dialog that you were going to take some feedback from us today, and that is why you wanted to be on our agenda. But I will give you one last committee members commentary before we move on to the next agenda item.
LOVE: This one is for Jacki. In California are you finding with the state hospital discharge OSHPDs data collection – I think they do capture Social Security. Has that created confusion at the front end admitting staff?
MONSON: It hasn’t yet. But they are trying to find a way to reconcile that data on the back end.
STEAD: Thank you. We appreciate the update and will bring this agenda block to a close and move on to the next agenda topic, which is the update from our colleague on the Community Health and Well-being Framework and the effort to develop measures.

BLAIR STARTS HERE

Agenda Item: Update on NCVHS Measurement Framework Implementation Work

STOUT: Good afternoon, everyone. For those of you who do not know me I am Soma Stout. I work at the Institute for Healthcare Improvement. I have the privilege of leading the 100 Million Healthier Lives Initiative.

Just to give a little bit of background, my hope is actually most of our conversation will be a dialogue. I have only a few slides just to remind people about where we are in the – how we got here, where we are in the process, and where we are thinking of going so this can be – I have several questions and some guidance that I would love to ask you for in terms of making sure that the process that we are following aligns with the vision that you have and to introduce some of the places where, as we have been in conversation with a broad stakeholder group of federal and non-federal partners, things that they have brought to our attention and proposals they have made that we wanted to feed back to you for your thoughts.

So, the way we got here, if you remember, NCVHS, of course, created the Framework and published it in January 2017. There had been extensive look at sort of the landscape of what would be measures for population health. The idea of a population health framework that was guided by an overarching understanding of the social determinants of health and their contribution under an overall framework of wellbeing was one that I think had a lot of traction. After a number of rounds of feedback, the Framework was published with a set of 10 domains and 30 subdomains that, on the next slide, just as a refresher, you can see.

One of the things that was important was that there were several overarching design principles that I know we really took to heart. One was that it needed to be flexible enough to both meet the needs of national stakeholders – so, very rigorous evidence-based measures that are benchmarkable and things that you can follow over time and measure anywhere, but also meet the needs of local communities.

Meeting the needs of local communities was sort of a twofold enterprise. The first was that nationally available data be cut down to the subcounty level so that local communities could really use them to drive improvement. The second was that there might be a set of distinct measures that are not what is being measured at the national level that might be priorities for local communities. We really took that design principle to heart in the way in which we created the process.

The second was that there did need to be a parsimonious set of multi-sector measures, measures that truly reflected the overall view of what we know creates health, which is both physical dimensions to that, but also a number of mental – we say mental, physical, social, and spiritual wellbeing for people, but also for communities as we think about all of the determinants of health and all of the agencies and sectors that contribute toward that.

These were just as a reminder of your work. This is where you were. I will assume you remember your work. I imagine your work in voluminous. It must be hard to remember all of it.

As that process ended that the NCVHS team was leading, as we moved to implementation, there was a desire for handing off the process as we move toward implementation, to a group that could steward it. 100 Million Healthier Lives agreed to be the group that created a facilitative leadership role in bringing together federal and non-federal partners, both for the process of measure selection and then for the process of measurement implementation.

Naturally, if you go to the next slide, we convened two workgroups, one focused on measure development, one focused on measure implementation, and then a stewardship group that would sort of guide the process as a whole.

We have had two in-person meetings, one for measure development and one for measure implementation, fairly logical. We have a third – our third gathering coming up on Friday, which is after a Delphi process to look at where things are. We try to time our gatherings fairly carefully. One was after the dust settled after administration change. It just wasn’t going to be easy to engage federal partners when there was a lot of shifts. So, we sort of waited until the dust settled. A second was we wanted to be sure as we were going into the Delphi process, for instance, after those first two meetings, that we waited long enough for the U.S. News and World Report, which was already on its way and had a timeline to end in March – that we would be able to see what was finally selected in terms of metrics so that rather than creating a lot of duplication in the field, we could try to leverage and align with existing resources like that, as well as other aligned initiatives that were happening in the field.

As we did that, sort of what emerged in the first two meetings was the sense that the framework was incredibly valuable and would be very useful and people felt that there was a gap in thinking of what we called the wellbeing of people. So, there were two – essentially two domains that were added to the framework. One was about the overall wellbeing of people beyond sort of physical health, which is sort of how people more narrowly interpreted that, and one was that equity needed to not only be a lens through which all of the other domains were examined, but needed to have its own domain. I have to admit equity is the price of admission in 100 Million, so we might have had a priority around that, but it was also something that I think was widely embraced and supported.

As we sort of have gone through a process of doing a detailed landscape analysis of existing other efforts in the field, what measures they are thinking about looking at, part of what I think we have come to is what we need to develop is something that looks more like a measurement ecosystem. There may be these national measures – and then depending on who you are and what you are trying to accomplish, you are able to navigate to different kinds of tools and measure sets that are appropriate for what you are trying to accomplish. By doing that, part of what we have been able to do is think about how we might knit in some of the existing efforts in the field and point to those. So instead of them just being a set of here are 20 different ways you can measure population health or social determinants, which then leaves, in our experience, people in the field choosing none of them and/or not creating the kinds of synergies and alignment that would allow us to get large-scale data and benchmarking, but we can have a few things that would be that parsimonious set and other things that are part of a larger ecosystem of tools and resources that people might be able to go through.

We have been in the midst of a Delphi process that just finished for three rounds now.

So, in the Delphi process, there were several aims and key considerations. One, the goal was to come up with, of course, both some recommendations of measures for each subdomain of the NCVHS framework to come up with what might be a national set. Actually, we structured the Delphi process so that people could recommend measures both for a national set and for a community set. We sort of said let’s just take a step back. If we just ask those as open-ended questions, would we come up with things that are the same or would we come up with things that are different.

What has been really interesting in terms of what has emerged in the process is in some cases, they are exactly the same. They want the same data across both. In other cases, communities prioritize things differently or even people who are looking at measuring things at the community level prioritize different measures for the community than they would prioritize at the national level. That was really helpful to us because it really supported the idea that what we need is not one set of five measures to rule them all or something like that, but rather a set of tools that are available and useful to people who are creating change at different parts of the ecosystem.

There was a strong push at every step of the process, in both of those first two meetings as well as during the Delphi process, that we needed to really balance existing measures, which are benchmarkable and evidence-based and already collected, with developmental measures that people – that had strong promise and could help shift people toward your broader understanding of what creates health.

Even though there was a full acknowledgement of the implementation challenges of new measures, there was a sense of every system is perfectly designed to obtain the results it gets. If we don’t change anything about the way we measure because we are just using the same things we have accessible to us, then we are unlikely to get – if we see measurement as a tool for culture mindset and system change, that we are unlikely to change the system or have different results, if we don’t change anything about that.

As people looked at what would be valuable, a number of alternative kinds of developmental measures that still had adoption for significant national initiatives or could be adopted emerged as potential candidates with strong support behind them.

One of the things that I do want to say about the process was that we were very careful to include a fairly wide range of people. So, it mattered a lot to us that we had national experts. I think we had a pretty strong breadth of national expertise from many of the different groups that measure and care about measurement at the national level.

We also had a strong breadth of community measurement experts, as well as actually community members who were going to be more affected by inequity or social determinants who had developed enough capability during a number of processes we have in 100 Million to be quite expert in giving feedback into this kind of a process. It really led to a richer, I think, conversation, as well as more of an ability to hold a tension between the different points of view that we went into the process valuing.

So, three feedback cycles. The first one was we realized there has been a small group behind the scenes. We have been so grateful to the members of the NCVHS committee, who have given some of their own time to support this process, like Rebecca and Kate and Vicki. They were joined by several others who really rolled up their sleeves and came up with a first set of measures that were aligned with the landscape from the OECD wellbeing measures to things from the Minnesota Compass to NQF to others to really looking at that and proposing an initial set of measures. That group acknowledged that their view was also limited.

The first round of the Delphi was to ask anyone are there measures that are missing here that you strongly recommend. People got to choose which subdomain they felt comfortable commenting on and then commented.

The second part of the Delphi was a prioritization of those to create a shorter list. There was another round of getting that next shorter list, which still left us with something like 57 measures in some subdomains down to now I think about 5 measures per subdomain at most at the national level and the community level.

So, this is where we are at in the process. Part of – that third cycle just closed last Friday. We have been in the process of assessing it. This Friday, we will be looking at where we are. What we say is this isn’t a popularist democracy to create measurement. It is not just that all votes win. But some of what emerged in the process were where do we need to do some additional developmental work. Some of the additional developmental works appear to be where is there a desire for more measures of what we call people-related wellbeing for specific subpopulations. So, the workforce, as a group, with significant numbers of employer coalitions in the mix, in terms of what would employers value for measurement of wellbeing in their workforce, children and adolescents, and then older adults, have emerged as three places.

Equity is one we held to say we know it is going to be its own subdomain. We know that there are measures that are within each of the subdomains. But we wanted to see for its own domain what would be additional measures that would be needed to reflect that that wouldn’t be duplicative of what was already in the subdomains.

An example would be there are measures of the GD coefficient or income inequality are already in the economic subdomain, a vitality subdomain. On the other hand, things like measurement of everyday racism as a question – I am not proposing that or saying that is the one – or racial segregation isn’t.

So, how do we make sure that key overarching things that we may think are important get added? There will be a process that continues after this to flesh out some of those domains further.

The conversation on Friday will focus on where are we in terms of – taking a step back, looking at the big picture, understanding where we think the opportunities and gaps are in where the process has brought us to date, understanding sort of our process, as well as is there a national measure set that is emerging of parsimonious measures for each of these that are based on things that are clear winners from the Delphi process? What would implementation look like?

An example of what gets modified with the implementation filter would be if our preference was a measure of one way to report walkability or measure walkability, but a slightly different one was implemented already in say the U.S. News and World Report or other tool, then we would probably go with the one that is already implemented if it was a minor difference. Versus if it is a substantial new area, then we would really need to look at who might be adopter groups for the developmental metrics, what are the tools and technologies that would support that implementation, where might that investment come from, where might there be testing groups to try these out in the field, and what would be the investment and the pathway for national adoption if on testing these measures proved to be useful, valid, et cetera.

That is the brief synopsis, I think, of where we have been, where we are, and perhaps where we are going. I would love to leave the rest of our time for conversations, questions.

STEAD: Thank you very much not just for the wonderful update, but for the unbelievable effort in helping this make traction.
COHEN: You stole my first sentence. I have had the pleasure and privilege to be working with Soma and all of the phenomenal folks who are truly committed to seeing this measurement framework into reality at a variety of levels. I want to personally thank Soma for the incredible leadership she has shown in pulling this together and really moving proactively, incredibly quickly, herding the cats. It has been fantastic.

I actually had – my feedback is more kind of a question for you and this effort to consider. What role do you see us playing in helping your efforts move forward probably at HHS and at the federal level?

STOUT: We would love your help. The reality is in order for these to move forward, especially as we identify some high level – that parsimonious set that we would want to recommend for national adoption, really thinking about what does national adoption mean and where are there opportunities that will work to advance those measures in the current climate?

So, an example would be I can imagine the Veterans Administration study taking a part of this and saying we will integrate this in the National Veterans Survey. In fact, they have already proposed doing that and are assessing some tools for that. I can also imagine that there are things that could be added to existing tools in that kind of way, where there may be more or less political will or more or less priority.

The leadership that this committee could play in helping to steward the implementation of this at the national level and/or guiding what that could look like, which might be accompaniment, but might actually be using here, to the right people, here is something that our committee has helped to produce as a framework and guide to the field. Here is the ecosystem that is recommended. Here are things that we would really prioritize. I think that kind of support would be invaluable.

MAYS: Thank you for coming back and sharing. It is always good – you send the child off to first grade and it is like how did they do there? You just come back and it is doing wonderful. That is good to hear.

Maybe you can help fill in for me like some of that process. Usually, in order to get groups to take it on, the first thing they want to know is was there an evaluation and who does this apply for. It sounds like what you are saying is that groups should take it on, but they often come back and say did you talk to – you know – and they will come up with very specific groups. It would help to kind of get a sense of who this was road tested with.

STOUT: Go to the next slide. You can see over 500 metrics were considered. There was a set of criteria developed with a team that helped to fill in some of the evidence details, for instance, behind each of the measures to the extent that they could in a short timeframe for 505 measures that were initially proposed in this process.

The 38 initial contributors to that were really diverse. We did something that we do in 100 Million a lot, which is use a social networking strategy for people to recommend who else do you think ought to be a part of the process and were very inclusive of those groups, both at the federal level, but also at the non-federal level.

So, we network mapped sort of who are the nodes of people that are really thinking about employer-based measurement, for instance, because we could imagine if you could get employers and healthcare delivery to adopt and integrate into their own measurement systems and make that data publicly available in a personally deidentified way, that could actually be a whole other driver for data collection that we currently don’t have as much leverage with. If they could be part of the process from the beginning and saw real value in it, they would be more likely to be adopters. We made sure to include who we thought we wanted to be implementers in the future into the process of metrics development from the beginning, if that makes sense.

Then the criteria – we adapted this from NQF’s criteria approach. The basics – what is objective and effective, important, feasible, usable, and valuable or useful, were some of what we – groups did to sort of assess. Within each one of those, we had a process where people proposed what were the sub-bullets under these that mattered to them. For instance, for valuable or useful, there was sort of a different – is it valuable to communities? Valuable to researchers and national stakeholders? Valuable to people/community members with lived experience of inequity for whatever that was? Currently used by or could be used by who? Who already finds this valuable? Is there potential for use in a national dataset within a parsimonious set of indicators?

The initial process had some scoring for these that the small workgroup did. We decided that was way too laborious for a large group to do, but we just broke it down into those kinds of indicators. I think the process of coming up with the decision criteria was useful. And I think part of what we are saying is before we roll this out as here it is, tied with a bow, part of the next step is we are beginning to, starting Friday, plan for implementation, which will mean that it will go through several more rounds of iteration with people in the field with much more opportunity for people to begin to say – not just ask the questions about who is it valid for and where has it been tested in, but to actually take it back to their stakeholders, bring in their measurement people, et cetera.

Usually, for the key adopter groups, we are at least three deep, in terms of who is involved, if that makes sense. So, the person who actually does their data reports, they are part of the mix by now. That will continue. I am sure we will find more groups that will meet that.

ROSS: Soma, thanks. Like Vickie, it is nice to see some of our work get used. That is great.

I thought you were going to say that we could endorse, but you said help encourage the adoption. Given this committee’s charter advising the Secretary, do you have further thoughts about – were you suggesting that we could encourage – do some activities, workgroups, discussions, hearing, whatever would be the right vehicle, that would involve federal agencies that you would want to see adopt this?

When you use the word adopt, who would you like it adopted by? Do you have any thoughts about what we would do to make that happen?

SOMA: Dave, it is clever of you to notice that. I assume you wouldn’t be personally investing your time into supporting anything you hadn’t endorsed. I just slid in the endorsement right in through there. We would, of course, want the endorsement of this group and your help with communicating and disseminating.

I think we would really want to think with you about who you think the federal groups are or the federal tools are. That might be most amenable to change. I think part of what we see is there are some agencies that are particularly motivated around this who we are already working with and know well. There may be just a chance to try things.

I think different agencies have different priorities. For instance, HHS has a huge priority around the opioid epidemic. I can imagine that getting some of the measures around mental health and substance abuse and wellbeing might be easier. What I am really thinking of are those more specific engagement conversations where we understand the context of that agency, where they might be able – if we think of a measurement ecosystem, if a different agency took on a different part of that, but we could find a way to connect to it still into a common thing, we might be able to get more done than if we say this whole thing in one bulk gets done.

On the other hand, if you think it could be done all in one fell swoop, by all means, we would love it. We will come back to you with a proposal.

STEAD: I will make one slightly tangential comment just to put on your radar screen something we are thinking about as another use of the Framework, in case you come across things where that might be useful.

We are thinking we will be developing – trying to identify terminologies that the National Library of Medicine might use to expand the Unified Medical Language System to fill gaps related to public health, population health, and social and behavioral determinants. One of our thoughts is that we might use the framework and its domains and subdomains as a – to just look and see whether there are, in fact, existing terminologies in those domains or subdomains that might be accommodated – that might be incorporated into UMLS in a way that would sort of help harmonize.

Through this process of identifying measures, if there is a way – Gibbs’ report had a good starting point. If there is a way to note terminologies that are already a part of the measurement process in some way, that would be – it would be extraordinarily helpful to sort of harness that because then that would help us all bring it together, if that computes.

STOUT: Absolutely. I wonder if there might be any connect with another process that we are involved with, which is around creating sort of common data models that can underlie, for instance, technology platforms. So, one of the things that we have been working with a number of partners on is – again, sort of a landscape analysis to look at, for instance, how are people talking about sectors. What we saw was the CDC versus – we had maybe 10 or 12 different groups. Each of them described sectors in a slightly different way. What is at the World Health Organization or the UN or others? How did they identify sectors?

There has been a process for maybe now eight different categories where we have come up with sort of a common classification that is built on existing classification where they already exist – that are already adopted. What does the Census say or for age groups and their cutoffs? What are the different measurement tools used that are in large-scale adoption? There we have actually held space for the NCVHS Framework to be the way in which impact in understood for population health – for common data model for population health.

If it is of interest to the committee, we would be happy to, at another time, share where that is and see if it might make sense. We have just been integrating the Framework into everything we do in 100 Million because it is a natural adoption.

STEAD: That would be great. Any other questions or comments?
COHEN: Just a general comment and observation for my colleague committee members. I would like us – as we move forward in developing our workplan for 2019, there seems to be several items related to following up here with the Measurement Framework, a whole variety of activities, whether it is NLMS-related or whether it is thinking about Soma’s suggestion about endorsing and looking for opportunities to use it at HHS or our other federal executive agencies. I would like us to begin maybe thinking broadly about having a focus on Measurement Framework follow-up for our upcoming workplan.
STEAD: Good point. Thank you. So that, I believe, brings us to the point of seeing if there is public comment either in the room or on the phone.

Agenda Item: Public Comment

HINES: Yes. The slide is up for those of you on the WebEx broadcast. First, let’s begin in the room. Do we have any visitors in the room to make public comment? No. Anyone on the WebEx chat? I don’t see any messages coming into the NCVHS mailbox. Nothing into my mailbox. Nothing into the WebEx broadcast. We will wait 30 more seconds. If you want to make a public comment, send comments to NCVHSmail@cdc.gov or use the WebEx broadcast dashboard.

We did get one comment from someone asking whether copies of today’s meeting materials – they will be posted on the website in the coming week.

Nothing from the WebEx. With that, I think the public comment period is complete.

Agenda Item: Closing Comments

STEAD: I will thank everybody. The work to get ready has been awesome. The teamwork by Rebecca, Debbie, Marietta, Eva, and the contract team has all been excellent. The effort by the committee has been outstanding. Thank you very much. We are adjourned.

(Whereupon, the meeting adjourned at 3:00 p.m.)