[This Transcript is Unedited]

DEPARTMENT OF HEALTH AND HUMAN SERVICES

NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS

SUBCOMMITTEE ON QUALITY

THE MEANINGFUL MEASURE SUPPLY CHAIN –
BUILDING MEASURES THAT MATTER FOR OUR NATION’S HEALTH

October 13, 2009

National Center for Health Statistics
3311 Toledo Road, Auditorium A
Hyattsville, MD 20782

Proceedings by:
CASET Associates, Ltd.
Fairfax, Virginia 22030
(703) 266-8402

CONTENTS

P r o c e e d i n g s 9:10 a.m.

Agenda Item: Introductions

DR. CARR: Good Morning. Welcome to NCVHS. The NCVHS Service is a statutory
public advisory body to the Secretary of Health and Human Services in the area
of health data and statistics. In this capacity, the Committee provides advice
and assistance to the Department and serves as a forum for interaction with
private sector groups on a variety of key health data issues. Within the NCVHS
Charter there are a number of roles and responsibilities. The Subcommittee on
Quality focused primarily on the first function outlined in the Charter which
is to monitor the nation’s health data needs and current approaches to
meeting those needs, identify emerging health data issues, including
methodologies and technologies of information systems, data bases and
networking that could improve the ability to meet those needs.

A second area of focus it to identify strategies and opportunities for
evolution from single purpose, narrowly focused categorical health data
collection strategies to more multi-purpose integrated shared data collection
strategies.

So I would like to pause here and ask the members and guests to introduce
themselves and then I will give an overview for the next two days. When you
introduce yourselves, could you also add a line about your role in your
organization or field as it relates to today’s subject? Please also state
if you have any conflicts.

I am Dr. Justine Carr, co-chair of the Quality Subcommittee and member of
the Full committee. I am an internist and hematologist and chief medical
officer at Caritas Health Care System, the largest community based health care
delivery system in New England. We are in year two of our EHR implementation
across six hospitals and 1200 physicians. Other than that, I have no conflicts.

DR. TANG: Paul Tang, chief medical information officer of Palo Alto Medical
Foundation. We have had an EHR implemented for ten years and I am co-chair of
this Committee on the Full Committee and no conflicts.

MR. QUINN: I am Matt Quinn from Agency for Health Care Research and Quality.
I am staff to the Sub Committee.

DR. MIDDLETON: My name is Blackford Middleton. I am from Partners Health
Care in the Brigham Young Women’s Hospital. At Partners, I am the
corporate director for Clinical Informatics Research and Development and I
chair the Center for IT Leadership and in both roles have an abiding interest
in quality measurement and the measurement of IT impact on quality. No
conflict, member of the Population Sub Committee and the Quality Sub Committee.

MS. JACKSON: Debbie Jackson, National Center for Health Statistics,
Committee staff.

DR. FITZMAURICE: Michael Fitzmaurice, Agency for Health Care Research and
Quality, senior science advisor for Information Technology to the Director,
liaison to the Full Committee and staff to the Subcommittee on Quality.

MR. REYNOLDS: Harry Reynolds, Blue Cross Blue Shield of North Carolina,
Chair of NCVHS, a visitor to this Committee. I spend a lot of my time on Health
IT policy and Health IT in general, both at my company in North Carolina and
throughout the country.

DR. CARR: So the ability to measure our nation’s health care system and
health will be crucial to addressing such national priorities as improving care
coordination health statistics. Today’s hearing is about meaningful
measurement, the supply chain of functions of choosing and building a measure
and the gaps that exist in the measurement of our four major national
priorities including coordination of care, health care disparities, health care
value and efficiency and population health.

The ability to measure key aspects of our health care system will be crucial
to addressing such national priorities as improving care and reducing
disparities, driving efficiency and value, and fostering population health. Too
often, however, ease of measurement has taken precedent over measuring what
matters. The availability of new data sources including electronic health
records as well as the heavy reliance on measures of meaningful use to allocate
distribution of funds under the HITECH provisions of ARRA, underscore the
relevance of understanding supply chain for measure development and
improvement.

So NCVHS Quality Subcommittee seeks to gain feedback from measure
developers, endorsers, system developers and reporters to articulate the
process whereby meaningful measures of quality are created, introduced to the
field and refined. In addition, we seek input and recommendations on the state
of meaningful measures in national priority areas.

So the goal of our hearing over the next two days is to address the
following four questions. One is, How do we approach building meaningful
measures? Two, What is the current process for developing measures and does it
adequately address measure development for key national priorities and
sub-populations? Three, How do we introduce new data sources, clinical data
from EHRs, user generated data, et cetera and how do we exchange them for old
measures based on administrative data? Four, How do we maintain and update
measures and what are the health IT system implications?

So based on the testimony, the NCVHS Quality Subcommittee will summarize the
findings, identify gaps and develop a set of written recommendations for
consideration by the Full Committee for transmittal to the Secretary.

I am grateful to our speakers for sharing their time and thoughts with us.
We are also very grateful to Matt for putting this all together behind the
scenes. So I encourage the Subcommittee members to listen carefully to the
speakers and engage in dialog that will bring meaningful recommendations on
meaningful measures.

With that, I will turn to my co-chair, Dr. Paul Tang, to share his unique
perspective. Since I read everyone’s bio, I will give yours. Paul is an
internist and chief medical information officer at Palo Alto Medical
Foundation, consulting associate professor of medicine at Stanford, vice-chair
of the federal Health Information and Technology Policy Committee and chair of
the Meaningful Use Group. He also chairs the National Quality Forums Health
Information and Technology Expert Panel and is a member of NQF Standards
Approval Committee.

DR. TANG: Thanks Justine. I thought what I would do is go into a little bit
more detail on the HITECH context for our hearing today.

As everyone knows, the Recovery Act provides an estimated maybe up to $46
billion in incentives to accelerate the adoption of HIT, in particular EHRs.
And there are four criteria to meet in order to earn the incentive. One is that
you use a certified EHR. Two, that you use it in a meaningful manner. Three,
that you exchange health information among EHRs and Four that you report
clinical quality measures as opposed to quality measures that are defined based
on administrative and claims data. So the HIT Policy Committee that was set up
in the statute is to recommend to the national coordinator, criteria against
which we would evaluate whether hospitals are eligible professionals would
qualify.

So we have a number of options. One is that we could take a structural
approach, meaning does the EHR have these features? Two, we could take a
process approach; are you using these features? And three, we could have more
of an outcomes oriented approach; are you getting any benefit from use of this
technology?

Because one of the criteria says that you need to use a certified EHR, we
left to the certification program to decide the structural approach. That is,
are you using a product that can quality you for meaningful use? And we focused
on the Meaningful Use Work Group of HIT Policy on two and three that is,
process wise, are you using these features and more importantly, outcomes wise,
are you getting any benefit from the use?

We focused on these “clinical” quality measures. We found that
there is a paucity of clinical quality measures meaning most of the measures,
and there are over 500 that are endorsed by NQF at this point, are defined
using administrative or claims data. So in fact, they are not using clinical
data that might come out of an EHR.

Another point is that most of the measures apply primarily to primary care
and do not cover specialty care very well.

Third, although the statute asks us to measure quality and stratify by
characteristics of an individual that would allow us to assess disparities in
care, often times we do not collect that information or properly report on
using that information.

Another problem is that there are national health priorities such as
coordination of care, or assessing and improving population health, yet we have
very few measures that focus on those aspects of care.

Finally, efficiency is very important, certainly in the context of health
reform and there we also have a lack of quality measures or a way of assessing
that. So in short, we lack meaningful measures. So that was sort of the
motivation for having this Hearing and hearing from hearing from the very
stakeholders in this quality measure supply chain. What is the current state of
the practice? How do we encourage more measures to be developed that use
clinical data from EHRs and how can we assess the things that are so important
for health reform or for the current health priorities.

So that is sort of a setup for how we invited folks to come and testify on
these matters.

While Helen is getting setup, let me introduce her. Dr. Burstin is the
Senior Vice President Performance Measures at the National Quality Forum and I
think all of us recognize that NQF is the endorser of quality measures.
Currently they have, as I said, over 500 measures that are endorsed and I am
sure Helen is going to cover the criteria by which measures are endorsed and
how they are trying to encourage measures that pertain to the health priorities
of the country. Prior to joining NQF, she was the Director of the Center for
Primary Care and Prevention in Clinical Partnership at AHRQ and she oversaw the
HIT portfolio which invested over $166 million dollars on research at the
intersection of HIT and Quality.

I think most of us know Helen quite well and appreciate her taking the time
to present to us. Thanks.

Agenda Item: Setting Priorities For Measurement

DR. BURSTIN: My pleasure. You guys were early. You are throwing me off. It
is so unusual to start five minutes earlier than unusual — apologies for the
delay.

So I think I am actually doing two talks. I will talk about criteria for
measurement and our industrial criteria second. I think what we want to do with
setting this up is talk a little bit about the national priorities at a broad
level. Where we would like the measurement field to go and as you have already
seen, obviously, from the Meaningful Use documents that came out of the Policy
Committee, a lot of emphasis already on at least several of the big national
priorities. So I will just go through the process and leave a lot of time for
questions.

So those of you who do not know NQF, in addition to endorsing national
consensus standards, which has been a traditional part of our role for 10 years
— actually this is our 10th anniversary — and publicly reporting
on performance over the last couple of years, has added to our mission
statement to the goal of setting national priorities and goals for the nation,
thinking that if we had a set of aligned goals, we would perhaps make more
progress than we do with the 500 measures, for example, that continue to grow
each year.

And lastly, trying to think through the piece around how we would actually
get these measures used and the fourth piece, and you will hear from Floyd
Eisenberg, Senior V.P for Health IT this afternoon, is we really see ourselves
also playing an important role as the bridge between the quality measurement
community and the HIT community and thinking through EHRs can really be brought
to bear to get us closer towards useful HIT based measures.

So about a year ago, NQF updated our evaluation criteria and the first one
here is the one that I just want to highlight. I will talk more about this in
the second session with David specifically focused on what makes a measure
meaningful? But one of the important things we did was really make the case
that not all measures are equal. We really want to get at measures that are, in
fact, truly important to measure and report. Some I would say meaningful, or
perhaps that term is used at the moment. But really trying to get at the fact
that we only want to measure things for which we can actually make a
significant improvement.

So what is the level of evidence for the measure that has to obviously be
clear? Is there an opportunity for improvement? We do not want to be measuring
things that are tapped out or fairly close to tapped out. We really want to be
measuring things where the act of measurement can actually be an important
force in terms of improvement. And that would either be in an absolute gap or a
significant gap across the providers who are being measured by the measure.

And then lastly, is it related to a priority area and the MPP areas I will
talk about in a moment are high impact areas of care. And the high impact area
of care, we are getting far more structured on in the short term with some work
we are doing under our HHS contract, which is evaluating all the top 20
conditions that are currently considered the highest priority for Medicare and
aligning them across nine different criteria to begin understanding issues of
cost, prevalence, impact, morbidity, mortality, complications, the whole gamut.
So from our starting point, that has got to be an absolute and, in fact, this
is an absolute criterion and if you do not pass this criterion the other NQF
endorsement criteria do not even apply, we will just stop evaluating the
measure.

So that was a lot of the basis of the relation to the national priorities.
It is that same thinking of only measuring what is important. So I will come
back to these other criteria in the next session.

We have been trying to move the field also towards higher performance and
Paul knows this well from sitting on our ultimate approver of endorsed
measures, our Consensus Approval Committee, that we have been trying to push
the field away from very narrow process measures that feel like interim steps
to an outcome, often very distal from the ultimate intermediate or final
outcome and drive towards higher performance. Really what is more proximate, if
it is a process measure, proximate to the ultimate outcome? And I will talk
more about that.

Definitely a shift towards composites trying to think about a comprehensive
view of measurement, measuring disparities in all we do. Paul mentioned that in
his opening remarks and I will talk more about that towards the end.

Harmonizing measures across sites and providers -– we continue to have
measures that are not harmonized between the physician level or the hospital
level, plan level and we really see that as an incredibly important role for
NQF and the field in general.

Then promoting this concept of shared accountability, the right measures
often times cannot be assigned to a given entity. For example, readmissions are
perhaps the best example here. Hospitals who argue they cannot be held solely
accountable for reducing readmissions and yet community providers will also say
the same thing, “I cannot be held accountable, I do not get any
information from the hospital that my patient, was first of all was even there,
which I can attest to for most of my patients or that even if I got it, I do
not have adequate information for a really effective transition and hand off.

So measures that really get at what is the right measure for the population
regardless of how it ultimately gets attributed or accountable to an individual
provider has been an important focus for us as well as beginning to think
across patient focused episodes. This is not necessarily episodes in terms of
groupers of, you know, you stopped having billing for this particular event,
but actually from a patients’ point of view the longitudinal view of where
we are going.

And so we are increasingly moving towards seeing measurement being, in a
sort of two dimensional framework, across these high priority conditions across
episodes to get the most comprehensive view as well as across the national
priorities and goals as I will mention.

These episodes are increasingly moving towards getting us closer to
measuring outcomes, measures of appropriateness, and then in the next six
months or so, also adding to that cost and resource use measures coupled with
quality measures.

This is a schematic of one of the initial forays that we did into this
episode framework of beginning to understand how, if we just focus on the way
we have done measurement to date, all of our measures are in Phase II. They are
all, for the most part, in the acute phase bubble. Did you get PCI within a
certain period of time? — Things like that, but we kind of miss the larger
picture.

And so it begins with the concept of a population at risk, so the population
health goal that has also been embraced by the Meaningful Use listings. A
population at risk for whom an MI could have been prevented, and then moving
across the concepts of, well you know there is an acute phase, a post acute
phase, secondary prevention, but then also recognizing that not all patients
will wind up equal at the end of an MI.

Some will have had an early intervention and wind up needing quality
measures that would look quite different than a patient who winds up with
congestive heart failure or for whom may have multiple co-morbidities that
complicates it, in which case you would want to be sure to include measures
that get at functional status, quality of life, advance care planning,
appropriate for all, but especially important for the patients who have that
trajectory, as well as certainly for the relatively healthy person at the end
of the day — a strong emphasis on those secondary prevention population health
kind of oriented measures.

We would also like to begin seeing care that is longitudinally assessed. So
a patient with an MI perhaps you would see their care in those first critical
30 days to get at that acute phase in acute rehab as well as perhaps looking
one year out, to really begin getting a really robust picture of the care we
are providing rather than these very narrow slices that we have been doing to
date.

So that has been the background for the way we have been thinking about the
conditions piece, but there has also been a strong emphasis on seeing the
cacophony of hundreds and hundreds of measures is not getting us where we want
to go. And so MPP served as the convener of a group called the National
Priorities Partnership. And the logic here was that if we could agree that
everybody would focus on the high leverage areas, where we think that this
harmonization across the multiple groups, these effecter arms as we like to
think of them, around common goals for improvement, could actually
significantly and fundamentally drive improvement in a more rapid way.

And so the goal of this group was to establish national priorities and goals
for public reporting, to focus measurement improvement efforts on those goals,
and it was lead obviously quite ably, by Don Berwick from IHI and Peggy
O’Kane from NCQA. There are now 32 leadership organizations who sit around
the table.

The initial set of priorities and goals have been completed, as I will show
you, and the next steps are also thinking about what they could collectively do
across a set of drivers, of which measurement is truly just one driver.

We often talk about it because it is the easy one to kind of grasp. But in
fact, regulation, accreditation, paid for performance, the various ways we
could drive improvement, are what these groups are trying to do by coming
together.

So as they set the national priorities, the goal was saying, what are those
high impact areas? They focused on the areas for which we would achieve these
four key aims: providing effective care, eliminating harm, removing waste, and
eradicating disparities. And although disparities are not one of the six
national priorities and goals, it is considered the fundamental cross-cutting
area that we want to ensure across all the national parties and goals.

So let me just run through the six national priorities here. So the first is
patient and family engagement, one that has also been included in the Health IT
Policy Committee list of important areas to move forward on. And the idea here
is we cannot make significant progress until the engaged patients and their
families, in managing their own healthcare, are making better decisions about
care.

And so the specific areas of focus, the goals underneath this priority area
would be: patient experience of care assessment in all settings of care, with
feedback to the providers of how they are doing. I think we have made
significant progress on the measurement side here and have endorsed patient
experience of the care measures in almost every setting.

The second is what is really involved in patient self management and some of
the measures should really get at patients having the tools they need to better
manage their care. It is a whole lot easier to manage your diabetes if your
hypertension, if you in fact know what your readings were from your clinic and
when you were seen. So huge opportunities there, I think, for a bi-directional
flow of information to dramatically, hopefully, improve patient chronic care
management.

And the last one that, I think, is going to be in some ways the most
challenging, because the measurement field is probably at the earliest, is the
idea of making sure every patient has an opportunity for shared decision
making, before they make a decision for a treatment or a procedure.

And the field of decision science is still evolving into what that looks
like. There are some very nice condition specific or procedure specific, ways
of doing decision support for patients. But this is probably the area where I
think we are going to see the greatest growth over the next couple of years
because we just have so little to date.

The second one is population health, which so much of our measurement focus
has been in the silos of our healthcare providers and entities and here it is
really taking a more global view. How do we improve the health of the
population? Another really important area, I think, that Health IT can just do
dramatic things for, in a way we have not been able to do.

The three specific foci here are; improving healthy lifestyle behaviors,
really focusing in on those behaviors that we know have a significant impact.
Ensuring that all Americans get, at least as a starting point, all the
evidence-based preventive services that are indicated for them by age, gender,
and risk factors. And the third one, which is, I think, probably the most
innovative and the one I find most exciting in terms of thinking about the
potential for Health IT, is this concept of a community index. Such an index
would allow us to assess the health of a community, which would likely be a
composite of many different kinds of indicators of community health. But the
kind of thing very few of us — even as a practicing doc in a community health
center for decades, I have never seen, for example, the health of a community
that I serve in although I know it is pretty poor — it would be very helpful
to be able to get that.

And the ability, for example, of using geographic information systems, GIS
and other indirect methods, to in fact even be able to target in and say it is
this particular sub-part of this community for which A1C’s are alarmingly
high, figure out what the issues are and that is what I think the capacity of
bridging some of the community registry data, the population health data to the
personal healthcare system. Quality measurement data is very exciting.

Safety, obviously one of the foundational elements of the healthcare system,
has to be there. A lot of focus specifically here on healthcare associated
infections. We have been trying to think about what that looks like across
sites of care. The ability, for example, to have Health IT connected facilities
here is especially exciting.

If you think about the example of surgical site infections and how hard it
is to be able to do the 30 day assessment, or if you have a device, the one
year assessment to see if there is an infection, can really be driven by this
ability — again focusing in on some of the key issues, like serious adverse
events that we are beginning to look at, as well as mortality and then a
broader view of the different causes of mortality.

Care coordination is probably the one, I think, could be most significantly
impacted by a really interoperable Health IT system. Ensuring patients receive
well coordinated care across all provider settings and levels of care, with
specific foci here on ensuring that medication reconciliation can be done in a
way that is logical at the appropriate transitions in care and not a huge
burden without impact.

I am still the kind of doc who likes the brown bag at every visit filled
with every pill at home because I still do not trust what I get most of the
time. But hopefully the systems will catch up so that in fact the systems will
allow us to do what I can only currently do with a — actually they are almost
always Target plastic bags now because so many patients get the $4.00 generic.
So emptying them out all on the table –- but boy to be able to do that in
a way that is IT enabled and sharable across sites is really the goal here;
preventing hospital readmissions and preventable ED visits of trying to think
about how to use care better.

Palliative care is one, I think, that is probably the one, I think, may have
been the most surprising for folks who saw the national priorities and goals.
This was the idea of looking and ensuring that we have appropriate and
compassionate care for all patients with life limiting illnesses. And that is
not just end of life, that is not just hospice care and the end of life but
patients for whom relief of physical symptoms, whether it is from COPD or
dementia, are getting the appropriate care they need as well as help with the
psychological, social and spiritual needs that patients and families will face.

And then finally, thinking about what is access to high quality palliative
care and hospice services and how we ensure and measure the quality of those
services.

Overuse is a critical one, I think, that all of us agree and anybody
following the healthcare debate, it is hard not to focus in on this one –-
we want to ensure that we are eliminating waste, while ensuring the appropriate
delivery of care –- the delivery of appropriate care.

So there are specific areas of foci that the national price partnership
identified including inappropriate medication use, unnecessary lab tests and
the data is just resplendent with evidence of repeating laboratory tests,
repeating diagnostic tests, because we cannot get access to what was done at
another setting; again an opportunity, hopefully, for some of the IT to come to
bear.

Unwarranted maternity care interventions, diagnostic procedures, other
procedures, unnecessary consultations, preventable ED visits and
hospitalizations, inappropriate non-palliative care services at the end of
life, and then potentially harmful preventive services with no benefit.

And this is the classic example of the D-list from the U.S. Preventative
Services Task Force that I used to oversee at ARHQ, where we know the risks
exceed the benefits. These should be the easy ones to try, to at least as a
starting point, move forward.

We do have a lot of work to do on convincing patients in the communities
that more is not always better. Certainly not something I have ever convinced
my mom of, not for lack of trying for years and years and years, who views if
she comes home with several referrals and procedures to be followed up, that
that is a really good thing.

Lastly, those are the six national priorities and goals. I also just want to
at least emphasize, I think, it is really important, although disparities was
not listed as I mentioned, as one of the six national priorities and goals
–- there is a huge opportunity for us especially in an IT enabled
environment to ensure that disparities measurement is not an afterthought, is
not something we do after the fact to see if there were disparities. But in
fact, we should be routinely assessing our quality of care by race, ethnicity,
language, and socio-economic status as part of routine measurement. And we are
obviously exploring both the direct methods for collecting these data from
patients in a way that is patient centered and effective, but also some of
those indirect methods I mentioned earlier using GIS or coding.

And finally we understand we probably cannot stratify everything, but we at
least want to ensure that measures for which we know there are known
disparities get stratified.

So we have come up with an initial set of measures that we have classified
as disparity sensitive in the ambulatory care setting, where a set of criteria
were applied to say, if you are going to stratify any measures, at least start
by stratifying these with a focus on the prevalence of the condition, the
impact of the condition for the disparity population, the impact of the quality
process –- are there known interventions we can do to, in fact, reduce
these disparities and the size of the quality gap?

Our plan moving forward is to, in fact, go through the entire portfolio of
measures, at least the ones we think are important, to say which of these
across all settings of care should always be stratified for disparities.

And so just putting it together very briefly, this is the last slide, just a
visual for us of where we think we are going here. It is that same slide of the
acute MI across an episode, but with the overlay of the cross-cutting national
priorities here in gold.

And so you would, for example, assess those patient preferences at the acute
phase of what they would like done. You want to ensure care coordination across
the entire arrows that cross our various sites of care. You want to get at
issues of overuse, in terms of cardiac imaging or procedures; you want to get
at population health at the starting point to ensure that your population, who
is potentially at risk or could be at risk, reduces that risk for an MI. You
want to understand and ensure we have got the palliative care for the patients
coping with end of life, or if nothing else, just coping with the need for
relief of symptoms. And then you want to have safety, obviously, as a system
property that goes across the entire thing.

So it is this vision of where we are hoping measurement will go, that allows
us to get that comprehensive view across conditions and multiple conditions for
many of our patients, which I think will be one of the challenges in a
measurement way of trying to think about how you take these episodes across
patients who have multiple conditions. And then finally, overlaying these
national priorities and that is, at least our vision, of where we are hoping we
can move forward in the field. And I will stop there and take questions.
Thanks.

DR. CARR: Thank you, that was terrific. Could I start off with one question?
In terms of socio-economic disparities, I am wondering if incorporated into
that, is the factors that have to do with low income folks who cannot take time
off from work. In fact, let me refer in particular to the Massachusetts
experience, where health insurance is held by most folks, but that health
insurance may have a very high deductible or co-pay. So have we found a way to
think about the kinds of disparities that result from — I want to get
preventive care, but I cannot take time off, I cannot afford the co-pay, I have
no one to babysit my kids, things like that?

DR. BURSTIN: It is an excellent question. I think some of this is the
limitations of the dataset, if we have the capacity to include those kind of
data and I think those are probably difficult to incorporate into some of our
IT systems, for example. But patients’ ability to self report, some of
this might be particularly important.

I think the other thing is that this is one of those areas where that might
be an appropriate way to at least be able to pull out — and I will talk about
that in the next session — understand the issues around exclusions. And there
has been a lot of discomfort, for example, about excluding patients from the
denominator of a measure for which there were financial issues.

This is a classic example for me, it is very difficult to get ARBs although
I can get Ace inhibitors for $4.00 from Target for my community health center
practice. Getting an ARB is pretty difficult and if somebody has a reaction to
Aces I cannot get an ARB unless I try to go through the prescription assistance
program. Should I have an exclusion, for example, that allows me to say I
cannot get my patient this drug?

I think this is a philosophical issue and I think it is one of those issues
where I think, from our perspective, the key here is transparency. So if you
are going to be excluding patients like that, at the end of the day we should
be able to see the percent of times I am excluding patients based on
patients’ ability to pay, whatever the case may be.

So first of all, we can begin to understand where the issues really are and
second of all, the last thing you want it to do is become sort of an easy way
out. So I view myself as a safety net provider. When I see patients, I should
be kind of going that extra mile to try to make sure I can get those things
done. But we also want to have the transparency to allow it, so that if I can
routinely not get my patients in for colonoscopy’s, which is quite
difficult to do in D.C. if you are uninsured, there is at least a way to track
it and see it over time.

DR. CARR: Right, yes. I think your point is well taken. Not necessarily
excluding, but even –- and actually we were speaking last night just about
saying — what is someone’s deductible. If you have a $50 deductible and
you are in one cohort and you have a $3,000 deductible and you are in another
cohort, that that represents two populations.

DR. TANG: Thanks, Helen for really an articulate description of the national
priorities partnership and the way it is folding into the measurement
developing is really very nice.

Do you have a sense of the timeline of getting from where we are with this
big bulk measures to the kinds of measures that you are talking about? What is
the timeline for the development and what is the timeline for adoption? How
long will it take to get there?

DR. CARR: That is an excellent question. I tend to be a fairly impatient
person, so I hope not very long. We are fortunate in that we did get a sizeable
HHS contract that allows us to do some very wide-ranging projects. We have just
launched one with the Steering Committee next week on outcomes across 20
conditions.

I do not know that we are going to get all the outcomes that we want, but we
see it is something iterative that at least we can identify where those gaps
are. What are the most important measures and kind of provide a menu of what
needs to be developed going forward.

We are going to do the same thing shortly around resource use. I think some
of the measures around care coordination are in development. I know you will
hear more from some measure developers later, who have those in the pipeline,
at CQA for example.

So I think we will fill out those national priorities within the next one to
two years. I do not think it is going to be a very long period of time. I think
the challenge is going to be if you overlay that with what we are trying to do
in Health IT, it is not clear how many of those can be jump started, by
developing them from scratch in an EHR enabled environment.

So we often have this concept of developing a measure and at least as you
will hear later on from Floyd, we have been trying to think about how we then
retool them to make them work in an IT enabled environment to get at issues of
meaningful use for example. I think the challenge is going to be to have
measure developers kind of reframe it in their mind and say stop, if you are
going to build for the future here, build a measure built off of these
interoperable HIT systems, even if we do not have them in our hands yet, so
that we do not go through that secondary step of retooling. That, to me, is
most exciting.

Somebody just told me about a very exciting safety measure and he said,
“Well the only problem is you can only do it if you have an EHR.” I
was like, great, bring it in. That is the way to go. We do not need to just
wait and say, do this on pen and paper. Bring the right measures in and I
think, hopefully, if the other incentives move us forward we will be able to do
that.

But the capacity to pull in registry data and things like that, I think,
especially you asked the question earlier about specialty measures, I think,
one of the biggest incentives to move the specialty measures forward is going
to be the capacity to pull in registry data, good clinical data that clinicians
view as being critically important to understanding outcomes with the data from
an EHR. If we can do that we can really move on specialty measures in a way
that we are not going to be able to do quite as easily with the typical data
that is within the EHRs we have now.

DR. FITZMAURICE: Helen, that was a great presentation that really gives a
good grounding to what NQF is doing and why. I noticed an awful lot of the
demands site, here are the priorities, here is what we consider important, not
an awful lot on the supply site. Can these quality measures be supplied? What
is the cost of producing them? Are the data available? Does it take a lot of
physicians’ time, expensive time versus staff time? Is that a
consideration in the quality measures that you choose to go forward with?

DR. BURSTIN: Absolutely. And actually it is a little strange, because there
is a second presentation I will do shortly, on what makes a measure meaningful
which would have been logically pulled together, and in there I will
specifically talk about an online what is feasibility, for example. What are
the bars for which we would set a measure to do in terms of how difficult it is
to collect the cost of measurement, the feasibility of measurement, the
capacity to pull it off of IT enabled systems?

But I think the other important consideration is the fact that there has not
been a lot of dollars out there for measured development. So we can only go so
far, we can help with some off the retooling, but there still is a gap in terms
of some measure developers wanting to move forward with some of these newer,
more complicated measures. Most outcomes require pretty significant risk
adjustment, for example, that is not inexpensive to do in tests.

So I think the ability to pull that stream of funding in, which is part of
some of these proposals going forward, would be very exciting.

DR. FITZMAURICE: I remember when NQF was first set up, it was set up as a
consensus body a standards body so that, among other good features, a federal
agency who wanted to adopt quality measures, could adopt them by reference.
They did not have to go through and develop them themselves since it was a
United States standard, could put them into regulations. Indeed, I would expect
something like CMS maybe VA and DOD, to be very interested in that feature as
part of the National Technology Transfer Act and OMB’s has got good
directors. We are all executive directors; we are all encouraged to do that. Do
you see a lot of feedback from that in interest of CMS, DOD, VA for just that
reason?

DR. BURSTIN: It is an excellent question. It is actually a huge part of what
we do and it is particularly on the CMS side. We have not had a lot of
interaction specifically with VA and DOD. But CMS, for example, routinely looks
to NQF endorsement as a requirement to move forward, unless there is a
compelling reason not to with a measure set.

DR. CARR: I think why don’t we just move into our next section and then
it is you again.

Agenda Item: What Makes a Measure Meaningful?

DR. BURSTIN: So just continuing on that theme, I will skip some things that
may be duplicative. Their specific question was what makes a measure
meaningful?

It is amazing how many layers of definition meaningful have taken on in the
last couple of years. Specifically, I have already gone through this concept of
trying to move towards where we are hoping measurement will move us to.

I think it is important to consider where we are. We have had a huge growth
of measures, based on several important drivers. The need for measures for pay
for performance, specifically at the individual physician and clinician level,
disparity sensitive measures, patient experience of care measures,
cross-cutting measures, but we still from our perspective from where we sit,
have a couple of key questions. Do we have too many? Do we have too few? And
are they the right measures? And I think that is a lot of what I talked about
in the first presentation.

The availability of data sources for measurement becomes critical and then,
obviously, this whole transition to EHRs, I am hoping will be transformative in
the way we look at what is a meaningful measure.

Just to focus in the criteria a bit, I talked already about importance to
measure and report and I will go through each of these in a bit more detail.
The three other criteria that are especially important are scientific
acceptability of the measurement properties, which is really about the measure
itself. What is the reliability of the measure going forward? Usability, can
the intended audiences understand and use the results for decision making? The
ultimate be all and end all for NQF endorse measures, is to be publicly
reported and used to make better decisions, something that we are certainly in
a transition phase for at the moment.

And lastly, feasibility, can that measure be implemented, Mike’s point
earlier, without undue burden, capture it with electronic data or use EHRs to
capture it?

So I will run through each of these because I think at least the lists of
bullets are identified. Some of the key issues are already captured within
these criteria. So just to compare the old and the new, the key issues that I
mentioned are importance to measure and report is now a must pass criterion and
then feasibility now has a much stronger emphasis on Health IT. And it is
probably not a surprise that Paul Tang has his hand in developing these
criteria as well for us. Then, lastly, the issue of usability.

So importance to measure and report, I mentioned some of this in the earlier
talk, but essentially, is the juice worth the squeeze? Is the effort extended
to produce these measures worth it because it allows us to get it measured and
reporting in an important area in which improvement is possible?

The specific sub-criteria’s I mentioned is it related to a priority
area? Is there evidence to support the focus and opportunity for improvement?
And this, I just pulled this in, I have seen this certainly many times in the
Meaningful Use slides from the Health IT Policy Committee, but this is really
what we are attempting to do here with these new criteria of trying to move
towards the outcomes piece. The advanced clinical processes here, I sort of
view as the process measures most proximate towards the improved outcomes, as
you will see in the way we think about how measures that are meaningful are
developed.

So specifically, we would want to have evidence for each of these kinds of
measures, and measures are very, very different so on an outcome measure, for
example, you want to have evidence that that intermediate outcome, for example
blood pressure control, leads to improved health or avoidance of harm.

On a process measure, as I mentioned earlier, we want to specifically know
that that process measure is proximate enough to an outcome, that it actually
has an impact on improving the desired outcome. And those can be intermediate
outcomes. But we have had measures submitted to us, for example, that say, was
the patient assessed as to whether they needed a flu vaccine? As opposed to,
did the patient have a flu vaccine? So we no longer want to deal with the
measures of assessment and things like that, unless they are closer to the
ultimate end game of being able to track the outcome.

Structural measures continue to be very important and actually Blackford
co-chaired a committee for us just about a year or year and a half ago,
specifically thinking about Health IT structural measures. So what is the
evidence that that structure ensures consistent delivery of effective processes
or access to get to avoidance of harm or improved benefit.

And lastly, efficiency, an area of increasing emphasis for us, is thinking
about the association between the measured resource use and the level of
performance. So again, as I mentioned, we are only interested in resource use
or cost as it is coupled to quality, so we can see the two of them together and
get at the concept of efficiency.

I want to talk a moment about clinical guidelines because I think as we
think about the evidence base here; so much of what we do is driven by the
clinical guidelines. As much as we can complain about the state of quality
measurement, in fact much of that is driven by the state of clinical guideline
development.

The clinical guidelines are often not developed with quality measurement or
clinical physician support, our ultimate goal of improvement in mind. There is
often a lack of specificity, for example, we may know what services we should
be doing, but there is no emphasis on the periodicity of that testing because
often the evidence base is not as clear to make that determination.

There is lack of precise definitions. High risk patients are a classic
example, if we cannot specify that sufficiently, a good quality measure cannot
follow it. And then as a corollary there, the decision support rules cannot, in
fact, pick up the right patients either. And then the lack of imprecise action
terms, you know, may consider doing something does not lend itself well towards
translating to a measure.

So the consideration of appropriateness, for example, makes this very
difficult. It is hard to take the three inch tolm of an appropriateness
guideline developed by some of the specialty societies, which may be elegant,
beautiful work, but it is hard to distill that into what becomes a meaningful
measure.

We also tend to have a focus on those measurable branch points. They may not
be the most important, but they are the ones we can kind of grab the data on.

I used to have the pleasure of working for John Eisenberg for three years
and John used to love to talk about the drunk looking for his keys under the
lamplight, you know under the lamp post. And I think, unfortunately, a lot of
the way we have been doing measure development has been focusing on, well I can
see those data on the lamp post; let’s make measures out of that. As
opposed to saying, what is the most important thing to measure and then trying
to find the data to do that? — Again, obviously, a pretty big IT proponent. I
think this is where we can hopefully get at the right data to get to the better
measures.

And then, ultimately I think, as we think about standards on the quality
measurement side, we equally need standards on the clinical guideline side to
ensure we are getting clinical guidelines that are computable and useable for
both measurement and improvement.

This is just a slide that Danny Rosenthal, who works with us who is a
medical informaticist has put together, just making the case that all of what
we do is a case of shared evidence. We all rely on the same evidence in terms
of measurement and improvement. Guidelines really become the trunk of that tree
and ultimately those clinical decision points are those branch points. Clinical
decision support should be focusing in on the things that, if you can remind
somebody to do something or encourage a different clinical path, you can
improve outcomes. Ultimately, I think we are hoping that quality measures will
move from those very early narrow branches where there is something measurable,
towards those leaves at the outer point of getting towards outcomes.

Scientific acceptability of the measure of properties, we want to ensure
that a good measure going forward should have precise specifications so that
you could replicate it from site to site and do effective comparisons. You want
to ensure that there is some level of testing.

This has been a challenge for us, as measures have moved into the field with
such rapidity to meet the needs of various programs. We are not getting a lot
of measures that have been adequately tested. We are now in the process of this
year beginning to see all the testing data for that first year of measures that
came in under our time limited endorsement option. We will begin to see how
well they work going forward.

We need to see some demonstration of comparability of different data sources
we are using. I do not think we are going to get this is the short term. I
think one of the real challenges for us going forward, will be that as we have
EHR enabled measures, it is not clear that they could be compared to measures
off of administrative data or compared to measures off of chart review data.

And I am hoping some of the research work will help us begin to see, in
fact, how often we have measures that allow that comparability across different
data sources. We often think of the quality of the data sources, again some
work Paul led with us in our Health IT Technical Expert Panel, of thinking
about the quality of the data that is within the measure itself.

I think as we begin thinking about measures that may be based purely on
administrative data, probably the quality of that data, certainly on the
outpatient side, is not great. But as we move up that path towards getting at
clinical data from an EHR or other clinical registries and moving up towards
that, we are sort of assuming that chart review data and EHR data should be
fairly similar. We do not expect a whole lot of comparability to measures built
purely off of administrative data. But the jury is still out.

We want to make sure that the specifications should allow us to look for
disparities, as I mentioned, should have risk adjustment, certainly if it is an
outcome measure, and this issue of exclusions is the one I think is going to be
a really critical issue for us moving forward.

We all know exclusions significantly increase the complexity and the
measurement burden on what we do. It limits our ability to use electronic
sources, we often times have measures, for example, especially at the hospital
level, that require you to go to a chart, or pull an EKG, or pull a vital signs
sheet to get at exclusions. And it is a real barrier towards getting at
harmonization.

So we in our updated evaluation criteria, and this is an area where we have
not been as strict as I think we need to be going forward, need evidence
presented that the measurement results would be significantly distorted without
the use of that exclusion.

We oftentimes have measures where there are 25 or 30 exclusions, and if you
actually did a sensitivity analysis, one could argue probably 20 of them may
increase the confidence of the provider being measured but, in fact, in terms
of the actual impact on a result, are really quite small. And we need to have a
better sense of what happens with those going forward.

We need to have this issue, we talked about earlier that Justine raised
about SCS, but also patient preferences. We need to have transparency to
understand when a patient preference is potentially the reason for an
exclusion.

And we have very strongly made the case that the last thing we need to do,
as we are trying to move towards measures that are more feasible, is require
additional data sources beyond what is needed to do the actual measure itself
to get at an exclusion, unless really without it, you would be significantly
hurting the validity of the measure. This is a challenge for all of us.

And I think going forward, it is not clear what the best approach is going
to be in an EHR, in fact, from talking to folks about, do you want to have a
great deal of specificity on some of the clinical exclusions, for example, that
are contraindications and embed those into the EHR? Or do you want to allow for
more open ended exclusions with the ability to do back and audit those fields
to figure out what were the most likely exclusions and what was appropriate?

Usability, I mentioned really from our perspective, requires evidence that
those measure results would be both meaningful and understandable to intended
audiences. And we really do, again as I mentioned earlier, focus on measures
that are appropriate and usable for public reporting and in forming quality
improvement.

So measures purely used for internal QI do not need to come through NQF.
Those are great. But if they could not pass the four NQF criteria, we do not
necessarily need to bring them through our process. They may be very
meaningful, but they are not going to be meaningful for the comparisons between
providers and public reporting.

And we have also now specifically honed in on this idea that we have to have
measures that are harmonized, to ensure that we are adding measures that have
distinct or additive value. This is probably the toughest to implement. But I
think is the one, in some ways, that is the most important.

CMS supported our project for us about a year and a half ago on
immunizations for flu and pneumococcal vaccine. Because we had this cacophony
of measures across nursing homes, and home health and clinics and hospitals
and, in fact, when we began looking at the project, we had 35 candidate
measures for flu and pneumococcal vaccine.

One could immediately go – that makes no sense and so the idea is, we came
up with a set of what we thought were the appropriate specifications that we
think all the measures should align to. We recognize there may be different
data sources, oasis or MDS, but at least the science should not be modified or
different, based on the kind of measures you are using going forward.

Feasibility, obviously probably the most important from the perspective of
where we are sitting today, in terms of thinking about getting at the data
without undue burden, and as much as possible, trying to use data that are
routinely generated as part of care. So again, EHRs, clinical registries,
whatever the case may be, and we want to ensure that the required data elements
that are there, and I will mention some of the work that we have been doing on
our quality data sets shortly, are either in electronic sources or there is a
least a credible near term path towards getting to electronic data collection.

We are probably expecting, I would guess, within the next one to two years
depending on who you ask and I am not sure exactly what that curve will look
like, but we are going to require specifications for EHRs at submission.

As well as the fact that all measures are now going through measure
maintenance, thanks to our HHS contract, we were able to do a much more
rigorous job on measure maintenance. And I suspect many of those measures that
are currently endorsed, will not pass through measure maintenance going
forward.

And the requirement will be, going forward I suspect again and probably
around that same time period of one to two years that all the measures up for
maintenance, will have to submit their EHR enabled specifications. Again the
appetite for truck based measures except in very limited areas has really
dropped considerably.

And the last issue from our perspective on feasibility is since we are now
applying proprietary measurement systems to submit to NQF, one of the
feasibility considerations is the cost associated with the use of that
proprietary measure system.

This is Paul’s slide, you have probably seen it, but again as we are
thinking about the HIT enabling of this and we want to get at that shared data
element, that sweet spot, as Paul likes to call it, between quality measures,
decision support and clinical guidelines.

And so I will just end with just a couple of thoughts about some of the work
we have been doing around this quality data set. And trying to make the case
that from where we sit, the way to make measures more meaningful going forward,
is to ensure there is harmonization and ensures we are getting at the right
kind of clinical data. This would enable us to, for example, be able to very
clearly always know the code set, the code list that is required. For example,
if it is an active diagnosis of diabetes that is required, we will specifically
indicate, that has got to come off a problem list — that cannot come off an
ICD9 coding in the outpatient setting.

So this is a real transition for us. And ultimately, as you will probably
hear from Floyd this afternoon, we are envisioning this quality data set
getting built into a measure authoring tool, that will allow measure developers
to have again a publicly available measure authoring tool, that will allow them
to immediately pull up the right code list. Pull up where you would find the
data within an EHR so that again we are trying to build measures that are more
consistent and get at what are the clinical areas of importance.

This was a list of some considerations we had come up with early on as part
of our Health IT Expert Panel around what would make measures especially
meaningful from the IT perspective. And we have already talked about national
priorities or high impact, but we also talked about, does the measure reflect
leverage of something really important in Health IT?

Are you using your system to get at what is most important? Can you, for
example, pull aspirin off your medication list in your EHR? Something
notoriously absent from most things because it is an over the counter drug, it
is the most important preventive services indicator for adults in terms of
impact and prevalence and yet most of the time we cannot get at it now. So if
you leveraged your IT system to get at that, would that be another
consideration of a meaningful measure?

Are you getting a more credible representation of quality because you are
using good clinical data as opposed to assuming what comes out of
administrative data for example would work adequately.

The next one here, I think, is more future tense but I was really pleased to
see it in the 2013 and 2015 considerations for Meaningful Use, which is the
measure of innovative patient centered data sources. So as patients begin to
submit some of these data, do we get a different set of measures potentially
through their input?

And lastly, is the measure sensitive to effective coordination of care and
data sharing across sites and providers. We know just having the measure within
your own setting is not sufficient if you really cannot share across settings
of care.

And this was your work, again some of the HIT Policy Committee had done, but
I think we are moving towards trying to think about what is a meaningful
measure, I think is in evolution. And I think getting to the point where are
have those advance care processes with decision support on the path to
outcomes, we really see much of our work focusing in on getting us towards that
path of the more meaningful measures at the end of the day. We have already
talked about the goals.

And the last slide I wanted to show is a wonderful slide that the RWJF
Aligning Courses for Quality group have put together about thinking about the
comprehensive data that is needed to generate performance information. And I
think if we really begin thinking about what makes a meaningful measure, we are
going to have measures that are a whole lot more meaningful if we can, in fact,
have these data streams that allow you to pull in the information from
pharmacies, labs, EHRs, hospitals, registries and I would also add to this
patient report, PHRs.

That is the way to get at that comprehensive integration of data, to get at
what is truly a meaningful measure. With that I will stop and take questions?
Or whatever you would like.

DR. CARR: Thanks, that was great. David, why don’t you start now? David
is the Archstone Foundation Chair and Professor of the David Geffen School of
Medicine at the University of California, UCLA, and chair elect of the American
Board of Internal Medicine Board of Directors.

DR. REUBEN: Let me introduce myself a little bit. I know a number of the
people in the room and some I do not. I am Dave Reuben and I am geriatrician.
All my patients are Medicare patients. My oldest is now 100 and my youngest is
probably in her late 60’s. So that is one of my pediatric patients. Most
of my patients are in their 80’s and 90’s. Part of my day job is
running the division of geriatrics at UCLA, but another part of my day job is
working on both quality measurements, developing quality indicators and
improving physician performance on quality indicators.

I have been part of the ACOV Team which won the Eisenberg Award last year,
so I have a little bit of experience with this and I will draw in some lessons
from that. My nights and weekends job, and where I am actually supposed to be
right now, is in South Carolina because I am the chair elect of the American
Board of Internal Medicine. And Cris Cassel apologizes for not being able to be
here. She sends her regrets and as a public disclaimer, I will say, I am no
Cris Cassel. But, glad to be here.

So I am going to talk a little bit about meaningfulness criteria. My take on
this is a little bit different and I think complimentary to what you heard from
Helen. We could call it the gospel according to Reuben. But I will talk a
little bit about validity, importance and longevity of measures. Then I am
going to talk a little bit about physicians’ organizations and how they
relate to quality measurement and improvement. And then specifically, I am
going to speak about efforts of the ABIM, with respect to board certification,
rote learning into quality measures and unrelated quality measures. And then
finally I will close with talking about how these board measures align with
other efforts.

So in terms of validity, one of the things you always want to be concerned
about with validity, is does the measure capture what it is intended to? So,
for example, a pretty common measure is smoking cessation counseling. Indeed,
if your patients are smoking, you want to tell them to do smoking cessation
counseling.

So, I have two patients left that still smoke. All the other ones either
have died or I have convinced them to stop smoking, so I have two that are
left. And every time I see them, I tell them to stop smoking. I just tell them
to stop smoking -– if they are still smoking five or six cigarettes a day,
stop. And sometimes I sit down there and I go with a very scripted routine of,
let’s set up a start date, here are the different patches and different
ways of doing it, I will call you on certain dates, and that is more or less
what is meant by smoking cessation counseling. Other times I say, you know, you
ought to stop smoking. It is really important to your health. And that is it.

For smoking cessation counseling, those would be valued as equal in some
senses. I would satisfy the measure either way. But it is not always the same
thing. So does that measure really capture what was intended? What was intended
by the measure, is me sitting down and spending about 10 minutes with a patient
going over how to stop smoking.

The second, does measurement discriminate performance among providers? And
here you have to decide, what is a reasonable sample to distinguish one
physician from another physician in terms of how they are behaving? We at the
American Board of Internal Medicine and I will show you some examples later,
think that probably 25 is a reasonable sample size, with 25 diabetics, you can
get a reasonably decent measure of somebody.

Then the question is whether you are measuring at the individual level, in
other words, the individual provider level versus in the practice or the
system, so some of the things that are measured, in terms of quality measures,
are things that are very easily measured, because they are physician behavior.
Some are actually really dependent upon how good the infrastructure of the
practice is, so what are you really measuring, the individual provider or the
system?

The third question about validity is does improvement on the measure result
in the improved outcomes? So what you really want, and this is the way a kind
of measurement works in general, is if you have a randomized clinical trial and
it shows that a new therapy is effective — Beta blockers for MI. And then you
will have a professional organization say this is a guideline now. And then you
will develop quality measures that reflect whether that guideline has been
implemented. Then you will do a quality improvement step to improve the quality
on that. And then, what you should see is better outcomes. So you are linking
RCT data to better outcomes.

And we have a fair amount of information on this first part up to about here
but the back translation going from clinical trial data to quality improvement
to better outcome; sometimes we do not have those data. In fact, sometimes we
can improve the process of care and not improve the outcomes of care. So this
is the linkage that you really want to have but we always do not have
especially this final link.

And finally let me just say a word about forced responses to move on to the
next screen. In a lot of the electronic health records, to get to the next
screen, you have to answer something. You cannot get out. You cannot escape.
And there is some data actually on pain control where they have taken a look at
quality of care in practices that had to have a forced response. So you had to
click to get onto the next screen versus those that did not. And the quality
was actually worse when you had to click to get to the next screen. So people
who would do it would say, get this out of my way. I need to move on and do
other things and they actually did not do the process.

Importance, how much impact does satisfying the quality measure have? So
here you have two quality measures. One would be weighing the patient at every
visit to see if they have weight loss. And the second would be providing
nutrition counseling. They may be two quality indicators that are both valid,
but they are probably not of equal importance. You could say that weighing a
patient, which takes about 10 seconds, may not have a whole lot of importance
compared to providing nutritional counseling which may take 20 to 30 minutes,
but they may, in fact, be treated equally.

Related to this also is the value of individual measures versus composite
scores and I think Helen was mentioning that the field is really moving much
more towards composite scores. Some of the work we did with ACOV, we actually
showed that improvement of quality resulted in better survival. This is really
a nice thing to find. So we said, gee we can take it down and find out what
were the real drivers of that. We dug and we dug and we found that the one
thing that consistently showed better improvement in survival was pneumovax.
Now rationally, it does not make any sense. But, in fact, it was that surrogate
marker.

So trying to rely on a couple of markers and say if you move these, it is
really going to make a difference. It is probably not the way to do it. You
probably need a composite outcome.

How long does a measure remain current? Well one of my colleagues, Paul
Chakel looked at this a number of years ago and said that based on the evolving
science that a guideline lasts about three years. So if you build in quality
measures and they are there forever, guess what? They may be old and may not be
measuring the right thing. Witness the example of estrogen. One year it was a
quality measure that if you did it, you got credit for it. The next year, it
was a quality measure that if you did it, you got penalized. So in fact, the
science moves on.

How long does it take to game the system? There are a lot of electronic
health records now that are tailored towards quality measures and if you do not
do anything, actually the quality measure is satisfied because the default goes
to satisfying the quality measure. This is some scary stuff –that if you do
not do something, you can actually satisfy it.

I am also very impressed by the marked capacity of the American free
enterprise system to quickly respond to economic incentives. Indeed if it is
being measured and if you are being paid for it, they will find a way to solve
the problem.

So I am going to shift gears entirely here and talk about physician
organizations. This is the view from 30,000 feet. The first are medical
societies such as the American College of Physicians, the American College of
Cardiology, the Society that I belong to is the American Geriatric Society and
these are membership organizations.

And they are designed to advance the field, to advance the field of
discipline that would help professions and also for the general good of the
public. They promote education, they typically provide CME, they publish
clinical guidelines, and they publish journals. They are clubs and they are
somewhat parochial. They really represent their members.

The second is licensing boards and licensing boards are generally state
based and they are required for practice. And they are very bureaucratic, they
are just state regulated.

The third level is certifying boards. The overall certifying board governing
body is the American Board of Medical Specialists. But within that, you see the
American Board of Surgeons, the American Board of Family Medicine, the American
Board of Internal Medicine, Orthopedics, there are about 27 or so boards here.
These are not for profit. They are oversight organizations and they are not
membership organizations.

You do not become a member of the American Board of Internal Medicine. You
become a diplomat of the American Board of Internal Medicine. They do not
accept support from any kind of pharmaceuticals or device companies and one of
their responsibilities is to define the field. What is a cardiologist? When
does a new specialty become a specialty? So they are in a sense, a very trusted
agent.

So a little bit about the American Board of Medical Specialists, actually 24
boards — The American Board of Internal Medicine is the largest of these. We
count for about a third of all practicing physicians, many of the
subspecialists, the cardiologist, and the endocrinologist, the rheumatologist,
the geriatricians are all internists. An important fact is that about 85
percent of American physicians are board certified.

So whatever the boards do and whatever you have to do for maintenance of
certification has a lot of teeth, a lot of reach.

The ABIMs mission is of the professions and for the public. We feel that we
are accountable to the public. And we work through the profession to benefit
the public.

So how do we improve quality? Improving quality is a very important aspect
of the mission of the American Board of Internal Medicine. We have something
that is called maintenance of certification. And basically, in lay terms,
maintenance certification is keeping up. It is keeping up. It is to make sure
that the physician you are seeing is as capable on the day you are seeing them
as on the day they first got certified. This has been required since 1990.

So the majority of internists out there have to maintain their certification
continuously. There are four parts. The first is maintaining a valid license.

The second is a process called self-evaluation. And the self-evaluation is
basically, you would receive questions you would have to answer, you would look
up the material, you see how you do, if you do not meet a certain threshold you
have to take the test again. But it is a self study program. It is designed
that you would evaluate where you are in terms of knowledge and improve it.

The third component is a written examination of knowledge. This is a test
that is a high stakes examination. The way it currently works is that you drive
or fly to a testing center, you have to strip naked essentially, you cannot
bring a pencil in there, you cannot bring a piece of paper in there, you have
to get fingerprinted to get into the room, you cannot bring anything with you.
And you sit there for four to six hours and you take an exam on a computer
-– very high stakes. And you can pass or you can fail it. There is no in
between.

The fourth is something that may be the most relevant to today’s
discussion, and that is evaluation of performance and practice. Looking at what
you do in practice and improving it.

So board certification is not to be taken lightly. It is important. It is
important on the quality landscape. A series of studies has been published on
some of the value of being board certified versus not being board certified.
Better outcomes and more reliable care, better quality of care, patients who
are being treated high blood pressure. Actually the time since your
certification correlates with worsen care. So if you were certified a long time
ago and you had not maintained your certification, the quality of care you
deliver for high blood pressure declines.

15 percent lower mortality, myocardial infarction, higher rates of
preventive services, lower rates of mortality for colon resection, and fewer
low birth weight babies. So by in large, maintenance certifications is a
quality measure in itself for the physician. And those of you who have
physicians, next time you go there make sure they are board certified.

So let’s talk just very briefly about parts two and three which is the
knowledge assessment and the examination. And these are very complimentary to
the performance measures. First of all they test diagnostic acumen. So one of
the things about quality measures, if you have the wrong diagnosis, if you do
really well on the quality measures, it really does not make a difference. So
if the diagnosis is acute MI, and you get all the quality measures right on
acute MI guesses what? You probably have not helped the patient much.

So this is really important that you are making sure that you are working
with the right diagnosis. The other things that the examination and the
self-evaluation modules do are they test clinical judgment which is really
important. And also allows us to really explore conservative management, things
that should not be done maybe.

That said we believe very strongly that performance measures matter. So the
ABMS as the parent organization requires all the boards to implement assessment
to performance. And we will give you an example of the ABIMs practice
improvement module, which is web based, uses NQF measures when available and it
includes a rapid cycle PBSA to address areas.

Also included in these practice and improvement modules, are patient
experiences. The voice of the patient is, we feel, exceptionally important. We
also assess practice infrastructure essentially using the NCQA PPC to see what
is available there. Also included in this are peer surveys of how a physician
is doing? So these are very broad.

So when you think about the overall landscape here and where these fit in,
if you go down quality measurement. That first of all national priorities are
not set by the ABIM. They are set by governmental agencies the IOM, NQF this is
the goal line. This is where we are headed. These are our priorities. Then
guidelines are done frequently by medical societies, researchers, develop
guidelines, volunteer health organizations like ADA develop guidelines. And
they are operationalized and endorsed. NCQA, PCPI, think tanks like Rand and
then NQF endorses them.

The assessments are developed to see whether those measures are being
completed appropriately. NCQA does that and the boards do a lot of that as
well.

Finally providing reports and feedback, NCQA does as well as the boards and
finally reassessment to see whether improvement has occurred and to our
knowledge only the boards are doing that.

I am going to show you kind of an interesting example of that. So this is
the anatomy of a practice improvement module. It has three components to it. It
has the patient survey, it has the practice survey and it has a medical record
review, actually digging into the medical records. Most of this is done now
manually through manual, hand-written records because that is what mostly are
out there. But these can be adapted to using electronic health records. There
is the performance report. The diplomat has to have an improvement plan which
is a plan, do, study, and act. And then impact on what was learned. That is the
basic principles of driving a practice improvement module.

So let me give you an example of one. This is the Diabetes PIM and a lot of
work has been put into the Diabetes PIM. And I am going to show you kind of a
testing sample of these. This was a sample of 957 physicians, most of whom were
general internists. And to do this practice improvement module, a total of
20,000 patient charts, or roughly 21 patients per physicians and almost 19,000
patient surveys, about 20 per physician were required.

So one of the things that was done here was the composite score. How well
the docs did on these diabetes measures. And the way that was done was to
convene an expert panel –- what we mean by an expert panel we mean
internists, ophthalmologist, who actually are practicing docs and who are
academic experts — who review the performance on the measures.

How well did these docs do? We review the reliability of individual
measures, selected clinical and patient measures, weighted the importance of
these measures, reviewed the reliability and reproducibility of the composite
measure, looked at the actual performance on the composite measure, defined a
“borderline candidate”. So what is a borderline candidate? A
borderline candidate is the person who is the cut point between a pass and a
fail.

We do this all the time when we certify physicians on the examination. We
use a process called the Angoff Process that asks the people who are on the
committee or the expert panelists to say, what would the borderline candidate
do on this? And finally, setting a standard for performance.

So this is an example that goes from NCQA data and these are measures for
diabetes. An eye examination — the criteria for passing this would be 60
percent of their patients, and they would get a pass for that if 60 percent of
their patients had an eye examination. Similarly, a foot examination, 80
percent of their patients would have it. And then these have been weighed in
terms of how many points you get based on how important they are. So there are
100 points possible and those are the measures.

So here is actually some data, using the data on those 957 physicians and
their patients. And this shows both the mean and the reliability of the
abstraction, but the mean score with 58 percent of them, had eye examinations
done for 60 percent of their patients or more. And you see the overall clinical
measure score was 73 percent. And this is simply going through the NCQA types
of data.

And you see how they did. They did pretty well on some, like smoking
cessation, of course you do not know what they said was smoking cessation. But
they did not do so well on others like the foot examination.

So the reliability, how that was done? No, I am not going to invent
something. Rebecca Lefner(?) can tell you about that.

PARTICIPANT: (Comment off mic.)

DR. REUBEN: I cannot tell you the details of the reliability measure, I am
sorry.

So here is where the really interesting stuff comes. Those were simply using
the NQA, how they would score it. Here they used the expert panel to actually
do a couple of things. Here is to set the criteria for the minimum acceptable
candidate. What would you expect the minimal competent internist to be able to
perform? And on an ophthalmology exam, 28 percent would be the pass rate
essentially.

And then they weighed the relative importance of that, which are a little
different than the NCQAs, but they still add up to about 100. So this is the
minimal threshold. So that if you overall score on your diabetes patients with
less than 47 percent, you would fail this threshold. You would be below the
minimally acceptable candidate.

And here is how these 957 physicians scored. So you will see that the mean
score is about 66 percent here. And that the standard, the minimum threshold,
was 47. You have these physicians here who scored below what we considered
minimally acceptable. So 89 percent were considered competent and about 11
percent were considered not to be competent. And so this tells us, even with a
self-abstracted instrument, that you can differentiate physicians who are doing
pretty well with those who do not do well.

And guess what? They looked at these physicians, who we like to call the
bottom feeders; these are the people who are not doing so well. And they looked
at their exam scores and they looked at everything else that is measures in
maintenance of certification, and they do not do very well there. So this tells
us, this is a population that needs to be focused on.

So this is just the Diabetes PIM, the most popular ones are diabetes,
preventive cardiology, and hypertension but there are a whole slew of these
PIMs. And other ones that are under development. So just about any condition,
any specialty, there is a PIM for.

And PIMs make a difference. There are 11 articles published in print, that
show the validity of these and they have clinical meaning. I will show you a
couple of them.

Five studies, including two controlled trials, have demonstrated positive
changes in care. And guess what? The re-measurement aspect, the one we talked
about there before, is that the PIMs make a difference. If you get your data
back and you develop a plan to improve it and then you re-measure, these are
for hypertension, but this is the mean 28 percent improvement when they
re-measured — for blood pressure or lipid control, medication adherence,
non-pharmacological treatment, 50 percent improvement. So, in fact, this makes
a difference.

So what about the docs? How do they respond to the PIMs? The experience of
over 5,000 physicians, 73 percent with completed PIMs, says it changed their
practice. 82 percent would recommend it to a colleague. Now it does take time.
Let me just tell you, it takes time to do a PIM. And doing hand abstractions of
your records is painful to say the least. But in fact, the docs are doing them.
If you would like to test drive a PIM, the slide here has the demo site where
you can walk yourself through it.

So finally, I would like to end a little bit with board alignment efforts.
How are we working with other organizations? How do you change behavior? You
can regulate, you can use economic forces, or you can rely on professionalism.
And the professionalism is where ABIM comes into place.

So how have we aligned? We have aligned with health plans. Names of
certification have been integrated into some reward and recognition program.
So, for example, Aetna, Cigna, a number of the health insurance companies, if
you are board certified, they recognize you as being so. You are on their
preferred list; sometimes it actually includes increased payment.

The second are Bridges to Excellence programs. What the Bridges to
Excellence program has done, is take data from our PIMs, and we are working
with them in terms of providing some help, but they are using that to provide
bonuses to physicians for better performance, using the PIM platform.

Alignment with other quality improvement efforts -– now let us just say
that you have a great electronic health record and you are generating these
kind of data all the time and you are giving feedback to docs, there is
something called an Improved Quality Improvement Pathway, where we can actually
certify a healthcare organization, like a Kaiser Permanente, to do this
themselves.

And in fact, for Mayo Clinic, we have given them; essentially, a five year
blanket to continue their quality improvement organizations and those will be
recognized for Part 4. So the idea here is that, if an organization here is
really stepping up to the plate with this and doing it well, we are not going
to force them into this widget where we are going to make sure that, in fact,
those standards are being met.

With the public sector, we have been very aligned with PQRI, the board
modules function registry, and in the Senate Finance Bill, the participation
and maintenance of certification is currently included as a pathway to PQRI.
And we strongly believe that it should remain there — this is a way of really
putting some teeth into it.

Discussions are underway currently about alignment with Meaningful Use. Once
again, we believe, as Helen was alluding to earlier, EHRs that are going to be
marketed need to have the components of maintenance and certification in there
with the measures that are going to be used. Because what we have to do and
this is critically important for physicians, is to reduce the redundancy. We
cannot have them doing 50 different things to get credit for 50 different
stakeholders.

So those are some people who have, including the VA System, they can do
these practice improvement modules of data collection component in just a few
minutes. Because those measures are in the electronic record and can be spit
out so they can see what they are doing.

Modifying and building new MOC assessment tools that align with and support
meaningful use goals — the MOCs are always a work in progress.

So just to sum up, where ABIM is and where the boards are -– they are
aligned with where the quality field is headed. The efforts at ABIM are
complimentary; they are consistent with everything that you will hear from NQF
or NCQA. We are all in the same direction.

The board requirement for maintenance certification, reaching those 85
percent of physicians which is going to even be higher, engages physicians in
improving quality of care. We have this hook, a very powerful hook to
physicians, to get them to participate. So we are really part of the solution
here.

Names of certification and other tools are comprehensive. They include
performance measures, but more, the judgment, and the diagnostic acumen. We
have data now showing that PIMs change physician behavior for the better. They
are readily adoptable and not too burdensome. I am going to say not too
burdensome -– they are somewhat burdensome at this point, but in the
future, hopefully, they are not burdensome at all.

And finally, public and private payers can leverage this existing, well
regarded infrastructure to align QI efforts and accelerate improvement. We want
to work with other organizations to make this work. So I am going to stop here
and ask for any questions. Just do not ask me about the reliability.

DR. CARR: Thank you that was just very, very thought provoking, very
interesting, very great work, so I think there are a number of questions. Mike,
do you want to start?

DR. FITZMAURICE: A very good presentation and thank you for coming. I really
got a lot out of it. The first one is — are PIMs equivalent to board
certification or are they separate from board certification?

DR. REUBEN: They are a component of board certification. So there are four
components of maintenance and certification. The PIMs are one, the test is
another maintenance of licensure, and the fourth is a self-assessment.

DR. FITZMAURICE: You mentioned 85 percent of physicians are board certified
and so immediately it comes into my mind, why aren’t the other 15 percent?
And when I look at docs, I know I can go online and find out if they are board
certified. So why aren’t the other 15 percent of the docs’ board
certified?

DR. REUBEN: So there are a couple of reasons. One, are there some that are
not eligible to be board certified? A lot of the international medical
graduates cannot sit for board certification. Others not so much now, but in
the past in the 50’s and 60’s and early 70’s, did not think it
was very important, and never got certified. And then there is that last
category of people who wanted to be certified (actually two other categories)
and could not pass the exam –- just could not get certified. And then
there is another category and that is the lapse certification. So as I said
from 1990 on, and in family medicine this has been since the inception of the
field, is that you have to maintain your certification and if you do not you
lapse and you become uncertified.

DR. FITZMAURICE: Did I see that a physician has to be recertified every four
years now?

DR. REUBEN: Well that is a really good question. It varies by board and
where the field is moving, is instead of saying that you have to re-certify
every ten years, every five years, every seven years it is really what they are
calling continuous maintenance certification. So you have to do something all
the time. So, for example, you would have to do a PIM every two years. You
would have to do a self-assessment piece every two years. But you might not
have to do the exam but once every ten years.

DR. FITZMAURICE: So if you do a PIM, then you have got something like an
automatic reminder. I was low on this from last year, so every time one of
these things comes up, have a reminder somewhere to let me do this so I get a
good score the next time. Plus, it is better for the patient.

DR. REUBEN: Yes. And the way I would think about this, is that every couple
of years I would do a different PIM. So for a couple of years, I am going to
work on my asthma treatment and then I will work on my heart failure treatment,
because you are not only making a quick fix, you are making a permanent fix.
You are doing this PDSA cycle to change how you approach these patients in
general. So if you do five over ten years, you have got five big conditions
that you have addressed.

DR. FITZMAURICE: Thank you.

DR. CARR: Just one question. Do you have an assessment of what you said that
85 percent are certified, and that includes those that are grandfathered in,
prior to 1990? Do you know how many there are in the group that are
grandfathered in?

DR. REUBEN: I think it is 25 or 30 percent. Yes, 30 percent.

DR. CARR: I think that is powerful because basically it lowers that number
down significantly.

DR. MIDDLETON: Thank you, David, Blackford Middleton from Boston Brigham
Women’s Hospital. I guess the one sort of obvious question on the table
is, if we look at certification, 85 percent being certified, and then we look
at Beth McGlynn’s data, suggesting 54.6 percent of the time, we are not
actually doing what we think we know we should be doing. So where is the
disconnect? If we are certifying, getting up to snuff on the knowledge base,
but it is not being applied, how do we address that disconnect?

DR. REUBEN: Well, essentially, that is where I spend most of my time. The
same thing if you take a look at the ACO table, the exact number for older
people. And if you look at geriatric conditions like osteoporosis, and falls
and incontinence, it is even worse. So why does it happen? There are a variety
of reasons. One is knowledge but many times it is not knowledge. It is not
knowledge. It is systems and time.

So there was a study that was published five or six years ago by one of the
family docs, and they said if you followed guidelines for ten conditions, just
followed guidelines, and you did not do anything for anybody else, it would
take you 10.2 hours every day just to meet those guidelines for ten conditions.
And that does not include what the patient came in for. So that tells us a lot
— if you are a primary care doc, then you are on this continuous treadmill.

So what we have to do, we have to keep physicians’ knowledge up but
even more so, you have to make it easier for them to do the right thing. And
part of that are actually better systems. It may be electronic health record,
its team care, it is delegation, and there are many, many solutions to that.

But the PIMs really get at that. The PIMS get at how you are going to change
your practice, not just how you are going to prove your knowledge? Because
knowledge generally is not the issue, it is generally being able to implement
appropriate behaviors.

DR. MIDDLETON: I think you are right on target. I guess the challenge is,
that when we think about kind of overcoming the inertia of clinical practice in
the heat of the battle, and you are acknowledging it may not be a knowledge
based issue, how do we actually address then the clinical requirements for the
physician to be a knowledge broker, more than a knowledge manager if you will.
So on behalf of the care team, on behalf of the patient, he or she becomes more
expert in processes of information management, knowledge access use and
application, as opposed to the repository of knowledge. Where will a
certification go to address those kinds of issues?

DR. REUBEN: Yes, that is a terrific question. And I have to tell you, based
on my own practice, we have a group practice, there are some docs in our
practice who are really great docs but I will never be able to teach them about
systems. They are never going to be leaders in system redesign. And I think
what is going to happen is that there are going to be docs who are good at
that, who are going to be leaders within groups to do redesigning of systems
and the others are going to be just good docs and they are going to follow
these systems. But you cannot expect every doc to be a system change leader.

DR. MIDDLETON: Maybe just this last one, if I may, you know, I think in this
line of reasoning, you have to think about what are we measuring with quality
measures and certification? You eluded to this I think appropriately in your
talk, but I think sometimes about the concept of sort of attributable qualities
across a care team, borrowing from an attributable risk in epidemiology, how
would you actually say well, this internist versus this nurse versus this care
manager versus this sister, brother, mom, dad, cousin at home, is actually
contributing to the diabetes outcomes of interest? It strikes me that, in a
way, we have kind of a myopic view in the traditional, paternalistic medical
model. We are assessing the physician component, but perhaps ignoring,
currently, the rest of these components.

DR. REUBEN: I think you are absolutely right. I think the way we have to
think about this is working backwards. So working backwards is — what is the
outcome that you want? You want the process to be done, you want the better
care and how you get there. And it is both. You have to have a competent
physician working in a well-functioning practice.

Sometimes you have a really good doc working in a system that is not so
good. So the really good doc can offset some of the deficiencies. And sometimes
you have a really good system that can offset some of the deficiencies in the
doc. So you have this kind of wiggle room here. But when you get to the
extremes, when you have a totally dysfunctional healthcare system or practice,
or you have a really bad doc, then you have got huge problems. Then this doc
has got to be either remediated or eliminated or this system has got to be
scratched. But within that kind of a little bit out of range, you can
compensate one way or the other.

DR. FITZMAURICE: Thank you.

MR. REYNOLDS: Now this is for both of you. Excellent, I am not a physician
and so I learned an incredible amount this morning, so thank you. But as you
know, we were involved in a Meaningful Use Hearing and then I am sitting here
with a document in front of me, the latest one, and I look out at 2015 and it
talks about clinical outcome measures, efficiency measures and safety. And then
I hear these presentations on measures and then you have somebody who has got
to set them as a group.

In North Carolina right now we are working on all of this obviously, and
about 60 percent of our small docs do not have electronic health records, and
so I am sitting here thinking about them as I listen to this whole ecosystem.
So you go to accepted use and both of you mentioned understanding, people have
to understand it. And that is everybody involved.

And then it has got to get in their work flow and it has got to get in their
capture, it has got to get in their system that they are going to buy, which,
oh by the way, we are talking about where we are going to be in 2015 and they
may have to buy that now.

And then you talked about how measures might not last more than three years.
Now again, that was just a statement, that is not an indictment or anything, it
is just a statement. And so you find yourself in a situation that the measures
are extremely important, but also how we move this entire environment to a
place, and as we make recommendations and other things, it is almost like you
have a structure of measures so that the right information could be captured
this year and then next year and if one goes away, another one comes in, so
that you are not chasing a ghost, literally, in being an implementer of
systems.

I see a lot of ghosts flying around. Because about the time I figure that
that would be important, what you are saying is that this might be important.
And the way to grab it might be important and this percentage might be this or
that. And so help some of us that are not as attuned to this as you, how are
you guys going to be, when we think about really meaningful measures?

How do we build an ecosystem, that whatever you decide is a meaningful
measure can take it and use it and use it not in ten years and not use it in
five years. And oh, I got involved in this and got my incentive yesterday, but
oops, it does not work quite so nicely tomorrow. A long winded question, but it
concerns issues.

DR. BURSTIN: Incredibly important question though. I think two responses on
the big picture level. I think one of the keys issues is that I do not think it
is so much of an issue on embedding the measures into electronic health
records, as it is embedding the key data pieces, the key data elements and data
types. The key data elements and data types are not going to significantly
change.

We could add to that quality dataset going forward, but no one would argue
that the injection fraction with patients with congestive heart failure will
not be an important piece of a measure going forward, regardless of whether
there are varieties in the kinds of medications you may use going forward. So I
think that is the first thing.

It is less about embedding the measure of logic into the system, as much as
it is about making sure you have got the right data in your hand. And I do
think that quality dataset should evolve to take on the additional important
measures that come forward to capture that. I mean outcomes for example, may be
difficult if it is just the EHR on my desk as opposed to an EHR that is
interoperable to other systems and registries and things along those lines. So
I think that is the first piece.

So I think the second issue is that you know we increasingly are moving
towards the outcome measures piece of it. So I think the less we rely on the
narrow processes, I think increasingly what we are going to see, is that those
process steps become clinical decision support. At least that is my hope.
Remind me to do the key steps along the path.

Build that into decision support systems, the knowledge management piece
that Blackford was just mentioning, but in some ways allow the measurement
systems to rely on whatever the data source may be. And I think we are going to
have to accept that fact that probably depending on your optimist/pessimist
view of the world, the next five years, for example, I am an optimist, we are
going to live in an environment that is going to be uncomfortable. We are going
to have lots of different data sources measuring people, docs, and hospitals,
whatever the case may be. I think we are just going to have to live with that.

And I hope there are sufficient dollars around to, in fact, research and
help us to understand what those key differences are. But I have no doubt that
we need to build the right kind of measures we need for the systems that are
coming, even if we do not have them in our hands yet, Harry.

But I think that at the same time, we have to acknowledge the fact that we
are going to continue to have to have a library of measures that say, if you
can only do charts, this is what you do. If you have got administrative data
and you can pull in a couple of key pieces of clinical information
electronically, this is what a clinically enriched administrative measure may
look like. And this is what it is if you have an EHR.

But this next five, X number of years, I think, is just going to be a
difficult time period that we are just going to have to accept. And those 60
percent of providers in your community probably are not going to be ready by
2011 or 2012; even to turn the button on and immediately produce their EHR
based specs. But hopefully, they will be able to do a combination of clinically
enriched administrative measures or, as necessary for some of the outcomes, do
something short based.

MR. REYNOLDS: Yes, I think I agree with Helen in that perspective. Think
about a launch to the moon -– which is what this is, basically, you know
where you are at Cape Canaveral and you know where the moon is and you are
going to shoot for it. But there are going to be a lot of mid-course
corrections. You do not just set on one trajectory. And as long as the goal is
there, and the techniques and the equipment are there to be able to make these
mid-course corrections, we are okay. But we are not going to get it right the
first go around. There are going to be a series of revisions and revisions and
revisions until we get to the goal.

So as long as there are mechanisms and data elements, then you can revise
these things. If a big study comes out saying that estrogen is bad for heart
disease or bad for breast cancer, you can switch it from one to another. So
those are possible.

I would like to say a little bit of disagreement about outcome measures
because I am always a little nervous about outcome measures because the
relationship between process measures and outcome measures is not one to one.
It is not one to one for a variety of reasons. One it takes a long time to go
from a process measure to getting an outcome. And the second is that sometimes
they do not occur. Sometimes you can do everything right and get a bad outcome.
Sometimes you can do everything wrong and get a good outcome, so some of these
outcomes are beyond our control.

So, for example, my oldest living parent died at 62. My dad died of terrible
heart disease and this and that and he was doing everything wrong. He just did
everything wrong. And I am doing everything right. But there may be something
in my genetic code that at 62, I am going to flip off. And that I cannot do
anything about.

So outcome measures are a little less controllable. Process measures, either
by getting the physician to behave well and the system to behave well, you can
do something about. So obviously, you want to get them as closely related or as
distal as you can, but those are things that you can change. You cannot change
some outcomes.

DR. BURSTIN: I think the whole point that there is also an important interim
step as we get towards getting to look at more outcomes, which is also just the
process of assessing outcomes. It sounds a little odd, but for example, just a
clinical example from where I practice, you know, we have got a small community
health center, and just got an EHR about a year ago, actually. We now have our
nurses’ aides, who have at best a high school education, doing a mental
health assessment of patients, to screen for depression. They walk in the door,
it is color-coded right in my EHR, and it is color-coded in red if somebody
does poorly on their NHQ. I know exactly who I need to intervene on.

Again, the outcome may not be at the end of the day, that I have
significantly improved the outcomes for patients with depression, although
ultimately that is what I think we need to be held accountable for, but boy, if
nothing else, you are in fact doing the outcomes assessments of patients, the
screening, the functional status assessments, to allow up to even see what
patients are on that trajectory. Until we know what patients are on our
trajectory, it is just a difficult situation.

Our end-stage renal diseases project, a couple of years ago, had two
patients on the Committees who insisted, insisted, we had to have a functional
status measure for patients on dialysis, because you feel terrible. But a lot
of the docs say, I cannot be held accountable for how patients on dialysis
feel. But as an interim step, at least, there was a requirement that every
year, all patients on dialysis have a functional assessment done. So I think
there are steps on that path that get us closer towards what is meaningful to
patients, in terms of outcomes.

DR. MIDDLETON: Just following up that line Helen and David, one of the
things that is sort of akin to the attributable quality idea, is to recognize
that actually the terminates of premature death only are impacted to a small
degree by what we do in healthcare, of course.

Larry Green not being here, I have to speak on his behalf, that he would say
sometime about now, that 10 or 15 percent of healthcare has in fact a premature
terminates of morbidity, but the rest of the determination is based upon
community and behavior and genetics and, et cetera.

So I guess what I am suggesting is that one of the things I would love to
see research focus on, is how to then take this case-mix idea and really
account for the heterogeneity of the patient population, based upon all those
other determinates so the doc does not feel that he is doing the right thing
and getting a bad outcome. Or the patient is doing the wrong thing, but still
getting a good outcome — all those kinds of vagaries.

If we could better, actually tune the measure, to account for not the
typical connotation of case-mix, but a more subtle connotation if you will,
that accounts for genetics, for behavior, for community or social exposures, et
cetera.

DR. CARR: Thank you both. These have just been tremendous presentations. And
interestingly, they come from, in my mind, I am seeing them on two ends of the
continuum, where if we work hard and less is more to measure certain things
that NQF endorse, and yet I am impressed with the engagement of physicians in
this practice management. My observations are often that as we are pulling
together, whether it be in-patient or out- patient measures, maybe you are a
physician who is involved in one case or one off here or one off there, and
there is a little work around to kind of make sure that that discharge summary
does not go out without that input or whatever.

And I am impressed by the process of looking at your cohort of patients and
looking at what you wrote, and how you manage, because implicit in that, is
your ownership of the continuum. So that when you get to the one off measure,
you have a context to put it in. And I think there has been an asynchrony with
the physicians being involved in measurement and they are often just hearing
about what you forgot, what you did not do, what you should have done. And I
love this idea of really owning the cohort.

The second thing is the whole P4P campaign in Boston. People spend an
inordinate amount of work getting from 98 percent compliance to 99 percent. And
similarly, groups are tearing practices in Boston and those that fell below 98
percent might be tier 2 and those above 98 percent. So there is a, perhaps I am
exaggerating a little but not that much, the point is that we are wasting
effort worrying about that one thing that was perhaps not even preventable,
when in fact this kind of in-depth analysis is very rich. And I think as we do
these measures, we have to give some thought to how they are used in P4P,
because when we talk about waste that is a waste. And it is at the expense of
this kind of rich look.

DR. TANG: Well I think part of what Helen and David mentioned about the
goodness of a measure, was a reflection of an opportunity and 98 percent
represents very poor opportunity. It also, I almost think, defectively says
high gaming because it is just not humanly possible to achieve that kind of
quote performance. So can I go to the next topic?

DR. BURSTIN: One brief response and I think it is an excellent point, and
again something that does not have an option for improvement, would not at
least bring forward the NQF endorsement. It just would not be a measure that we
would consider important enough to make the effort to publicly report. You may
want to do it internally for QI to make sure you do not fall off the cliff, as
soon as you stop publicly reporting it. But at the same time, it probably does
not rise to that level.

I do want to make one another, I think important point, that I think has not
had a lot of discussion, which we often talk about just the absolute value of a
measure as opposed to the trajectory for getting there. And I think,
particularly for a lot of providers who may not be at the 90th
percentile perhaps, the trajectory of recognizing a huge improvement over time,
is something that should also be rewarded. I do not think we have seen a lot of
that. It is a bit in the value-based purchasing program from CMS, some
hospitals on that path towards the trajectory, the absolute threshold, get
payment as well.

And as a safety net provider on my day a week, I have the same perspectives.
I can have somebody walk in my door with an A1C of 15 and I can get them to
eight pretty quickly. Getting them from eight to seven can often be difficult.
So I think again, understanding the trajectory, as well as the absolute
threshold, I think is another consideration.

DR. TANG: Part of what I felt was so exciting about both presentations is
the direction towards better quality indicators of performance; let me put it
that way. So in the NQF way, it is looking at the quality of the data and what
David talked about, I think that is what is in the reliability column. But let
me check my understanding of the vision, particularly on the certification
side, the ABIM, is just like in CME we know how ineffective it is to plop
somebody in a seat in Hawaii. And there is a trend towards going towards
incremental CME, using adult learning methods of, gosh the best time is when I
have a question with a patient in front of me could I get credit for looking
something up right at that time? That would be reinforcing good behavior and
better care. Your counterpart was the PIMs, because in a sense, what you have
done is, I think, is created a notion of self-examination and doing something
about it, versus sitting down for a written test in front of some computer.

Now, I think you talked about continuous accreditation or certification. If
I interpreted you correctly, you are trying to instrument peoples practice to
figure out whether they are still practicing up to date, good quality medicine.
And if to the extent that we can both on the NQF endorsement site and the
accrediting site use the same measures that depend on the same data in the same
EHRs, to essentially like, Carolyn Clancy F7, would get me both my quality
reports and my certification, if you will. Is that sort of the vision?

DR. REUBEN: That is the vision. And the vast majority of the PIM measures
and we have I think 525 PIM measures or something like that, are NQF endorsed.
Now not all measures for all conditions have NQF endorsement yet. So there are
some that are not. But, in fact, we are trying to align with NQF, so that it is
the same measures. We are all working towards the same goal.

DR. TANG: So that is where I go to, is I really am interested in the
reliability, because it stuck out that that high gaming smoking cessation
counseling measure, I believe, has a low reliability and we should probably go
away from things like that. Because it is not clear to me that the evidence
says forcing something into the documentation about smoking cessation
counseling, improves the smoking rate.

And so why create the gaming because with EHRs as you know, it is instant
verification. You can cause it to happen. And the other things that I looked
at, I noticed in your diabetes PIM, I think you showed, is the A1C greater than
nine measure, so as the not bad measure. And in a sense we are trying to go, I
think, away from those kinds of measures in NQF endorsement process, because it
does not align well with what physicians think about their own performance. So
you do not think you want to be not bad. You know what the guidelines say and
you want to head towards those guidelines and to the extent that your mental
model of what it is to do good quality measures, lines up with the measure,
that, essentially I think, is positively reinforcing in terms of the good
behavior. So in some sense, it may be instead of going after the current NQF
measures, we want to go towards where the puck is going, for the measures that
are aligned with practice and the way physicians think, and I think can really
create a positive reinforcement move.

DR. REUBEN: Yes, we have actually thought about this a lot. Not so much with
ABIM, but certainly with the ACO measures which is really a floor. That is, if
you are not performing these -– they are not a ceiling, they are not the
real goal. But if you are not performing these things, you probably are not a
very good doc kind of thing. And as you get higher and higher up that
aspirational ladder of what you really would like to have, you get much more
push back. You get much more push-back because other things happen. You cannot
achieve that 98 percent, that theoretical 98 percent. But if you say, you know
this is a floor and if you do not meet this floor, that you are not doing a
good job. How can you argue if you are in that group that is the bottom
feeders, that less than 44 percent? How can you justify that you are doing a
good job as a doctor when the floor is low.

DR. TANG: So maybe that is my real question. Philosophically, is a
certification organization like APIM going after the floor to eliminate people
or going for the goal that motivates people. And I almost think that you are
going to capture far more docs on the latter.

DR. REUBEN: So you are absolutely right. And if you take a look at our
strategic plan, our goal is to continuously set the bar higher. So here we are
today in 2009 the bar is here. In 2011, the bar is going to be a little higher.
And the bar is going to be higher in 2013 so you keep dialing it up.

DR. TANG: So the question is are you setting the bar for the floor or the
top part? I feel better about getting 60 and then 70 and then 85 towards my
real goal, versus passing over the 20 percent goal.

DR. MIDDLETON: I think Paul and I have a similar vision or aspiration here.
In many ways, I would much rather be pulled up than smacked from behind. In a
way, the low bar is the smack from behind. The aspirational goal or the stretch
goal might be that in fact, certification takes on a new flavor. It is actually
continuously monitoring, continuously educating and providing instantaneous
feedback at the point of care, so that I actually know, is my diabetic
population within the guard rails or is Mrs. Smith actually within the guard
rails of her diabetes care.

There is a problem though, I think, that we have to sort of recognize here.
If history tells us anything about the Internet, I am concerned we may see some
cognitive substitution in healthcare, in ways that have happened in other
industries. That is, many more decisions may be made by many more different
types of people. So rather than raising one bar, I would suggest maybe a
spectrum of pro-active certifications, if you will, in the manner in which we
just described, that might actually delineate a range of professionals rather
than an increasingly god-like physician.

In other industries, of course, many different types of folks are making
many different types of decisions based upon tools and utilities and knowledge
and the access on the Internet, all the disintermediation stuff, et cetera. So
they can gain knowledge, rather than accessing the provider, per se. So it is
just a thought, thinking about the high bar and a range of professionals, all
perhaps warranting certification under the ABIM, but in different roles.

DR. REUBEN: We have talked about this. ABIM is famous for talking and being
very careful and ruminating and considering many things. And in fact, we have
talked a lot about whether currently it is a dichotomous threshold you are
certified or not. But the whole idea of caring physicians and having physicians
who are exemplary and fine and good and whatever, we have not crossed that
threshold yet.

It is not that it is out of the question for the future, but it is not thee
in 2009. And part of the reason, it is really interesting, this came up
yesterday at the board meeting, is do all physicians have to be superb
physicians or can they be fine physicians? And is it okay to go to a fine
physician, or do you have to go to that person who scores in the top 10 or 20
percent. For me, I am fine with my physician being certified and being
competent. I do not need that top one or two percent. I do not gain that much
from it. So these are really good questions, but the idea is to move the bar,
continuously raise the bar. And that is right in the strategic plan.

DR. BURSTIN: I want to follow-up on that. I think that beyond even
certification, the capacity of that magical F7 button, allows providers within
a group, and I say providers in the broadest sense of the word, Blackford. I
mean, all clinicians sitting down together at a systems level reviewing the
data out of your practice. I mean to really have that in real time obviously,
with my time at the Brigham, when I headed quite a measurement, I had that. I
would hand out to all the docs our entire internal medicine practice, here are
the measures we initially had it A through Z. That lasted about ten minutes,
because we all unblended in about 30 seconds because we wanted to see who was
best.

But that is that kind of real time feedback that allows for really dramatic
systems improvement and helps you figure out — it really should be the medical
assistant who just routinely does flu shots without an order from me. I mean
that is the kind of stuff that, I think, becomes a systems piece of it and
becomes so apparent when you have the ability to rapidly report. Not a year
later get the reports from an external body saying this is how you did. But in
real time and even without necessarily having to do your own chart review,
seeing this is how you are doing on a continuous basis.

DR. REUBEN: And that is the Holy Grail for us. The Holy Grail is that
doctors in real time can monitor the care for a number of different conditions.
That said that is not going to be possible without electronic health record.
You just cannot get there.

MR. QUINN: Thank you both for your presentations — This sort of gets into
one of the thoughts that I had. The PIMs and the MOC is a really compelling
lever for motivating clinicians. And many well-intentioned and well-designed
quality improvement efforts, have not really gained physician buy-in for a
variety of reasons.

It seems to me that if you can align these two, the PIMs as well as the
measures that gain buy-in, there is an opportunity for a synergistic effect. Is
it the measures that have to gain buy-in from clinicians themselves or how do
you design this so that it just makes sense, and that you do gain maximum
uptake?

DR. REUBEN: I do not think the measures have been a problem. The measures
are based on randomized clinical trials. They are based on guidelines. They are
not the problem. The problem is getting the docs to be able to do it.

And there are a lot of issues why you know, for example, if you have an old
person who has got 12 problems, for some reason or other, doctors feel
compelled to do a lousy job addressing all 12 at each visit and then spend an
entire visit addressing one problem, doing a good job. It is just somehow or
another, how we were trained as doctors.

And there is a tremendous amount of inertia in what goes on in a
doctors’ office. A little quality improvement project I did a few years
ago, I sat and watched every one of our docs interact with patients and saw
what went on in the visit. And there was a lot of time with talking and a lot
of time with counseling there was a lot of time addressing those issues, but
the kinds of things that would be measured under quality performance just were
not there. They did not have the time to do that.

So I do not think knowledge is the issue. I do not think what the measures
are, is the issue. I think being able to get that behavior, the knowledge into
practice is really the limiting step.

DR. BURSTIN: I agree with David. And just to add to that, I think that while
the measures themselves may be okay and they are evidence based, I think, many
clinicians we view, just do not go far enough. They are not even thinking about
the 2013, 2015 kinds of measures that are envisioned for meaningful use, care
coordination, patient and family engagement, does take us to a different place.

And those are not the traditional measures that most docs have had fed back
to them or even the ones that we have had as part of our assessments, other
than patient experience, which I think many would argue, has been really
transformative for a lot of health systems to have that HCAPs data or other
kinds of patient experience data.

But I think going beyond that to get at even other levels of how often do in
fact, do you get a discharge summary back in a timely manner? Are all the labs
followed up at discharge? I know what happened. I mean, those are the kinds of
things that I think providers, and I do not want to say docs, because I think
it is bigger than that, it is really the team, would find so meaningful.

So it is not just measurement for the sake of measurement, but it is
actually measurement that is meaningful to me as a clinician to do a better job
because it gets more at the systems.

DR. FITZMAURICE: You have painted a picture of really how difficult it is
and yet how far we have come from NCQA to NQF to the board certifications and
holding people more and more accountable. I sense it is not going to get
easier. We have the quality measures and I work with Floyd at the HITSPE
population technical committee, where we worked on quality measures and
surveillance measures and since then there is this continuous, this is not
precise enough, we need to send it back to NQF to the HITECH Committee that was
chaired by Paul to get more precise measures of just what was meant in the
quality measure.

Now as we come up into 2010, 2011, 2012, 2013, 2014, we are going to be
moving from ICD9 to ICD10 and we are going to be changing some of the claims
information. Beyond that, we may be moving more and more to SNOWMED as
underlying coding system. What that means regarding difficulty for quality
measures and for meaningful use measures as well, is that the kinds of things
that the electronic health records are massaging are going to be different.
They are going to be split into parts and some things will have to be
aggregated.

Is NQF working with the quality measure developers to bring some realization
to this? I have seen some really good work that Floyd has done in seeing a
piece of the ICD9 codes, these are the exclusion codes. It is all going to have
to evolve into something better and better as we get into better and better
coding systems. Is this on the radar screen, Helen?

DR. BURSTIN: It is completely on the radar screen and in fact we are also
doing some additional work under our HHS contract that is an expert panel that
is meeting shortly to begin understanding what are those transition issues and
how do we even envision them? When all the measures come up for maintenance,
for example, by 2013, they will have to be in ICD10 or SNOWMED, part of the
determinations of what happens with the HIT Policy Committee. So that is
definitely the trajectory.

And again, this idea ultimately of saying, if you are offering a tool from
an EHR to NOVA in 2011, 2012, 2013, and you have the capacity to go to a
measure offering tool, it will automatically pull up the appropriate code set.

I think part of the challenge is that we do not often have all the
crosswalks we need between ICD9 and ICD10 and SNOWMED. But I think, to me,
those seem like technological issues that are achievable, as opposed to, I
think, the harder stuff which is actually around capturing some of the logic
and are we actually staying true to what was intended through the guidelines,
through some of those transitions.

And I think some of the measures that have been developed, again, have been
based on clinical guidelines that are not as precise or pristine as we would
like, and so some of the logic of trying to translate some of that to an EHR is
actually the harder piece than the coding. I would argue, last time, known well
of when stroke systems began. It is pretty darn hard to put in an EHR. So you
have to really rethink from scratch, how do you develop a measure that gets at
the timeliness of stroke treatment as opposed to starting with the measure that
we have, which was developed for a very different environment that is not IT
based?

DR. FITZMAURICE: Could I ask another question? And that has to do with
preparatory quality measures. If I made a health plan a group of providers, and
I want to measure quality, I may say, this is really a good quality measure,
but do I have to pay 25 cents a patient every time I apply it to the quality
measure developer for 1,000 patients that is $250?

Is there some way to handle the economics? We did it in one case with
SNOWMED. We bought out the license for five years, here is a $5 million payment
and so much per year, to use it and then at the end, we can use it as it stands
in perpetuity. That is one subscription model that could be used for paying for
quality measures. There may be others. Do you see having to pay for the use of
quality measures to be a large or a fairly small barrier to the use of quality
measures?

DR. BURSTIN: It is an excellent question, Mike. To date we have all the
current NQF measures are available without fee. There is definitely a move
towards some more of these proprietary systems, especially if you look at the
complex proprietary systems where there have been years and years of investment
in a complex risk adjustment data base. Some of those coming through may have
associated charges. I think one of the challenges going forward, if for
example, the grouper methodology is something we want to move forward with,
maybe should that ultimately become a public sort of analogous situation to
SNOWMED? I do not think we know the answer yet. But I think you know,
increasingly, we are seeing that without the public support for the measurement
side, and that I think has been a challenge for us, you know as the clinical
registries have been developed, oftentimes by specialty societies with
specialty society funds, as they transition that to being for public reporting,
how does that evolve? And I do not think we know the answers yet. But I think
it is something we are all, kind of, keeping a close eye on. Hopefully, the
additional support, as I mentioned early on that is hopefully in some of that
health reform, for measure development will get us to maintain more of that
critical measure development expertise in the public domain.

DR. REUBEN: I would just like to echo that. I think the way to do this is
really to have these measures in the public domain. And to do that, you are
going to have to pay for the development and the maintenance of these measures.
And rather than doing that on the back end after they are developed as a for
profit industry, that this is a wise investment for the government, is to
invest in the development and the maintenance of these measures. It is just
going to make it much easier to distribute and much more widely accepted.
Otherwise you get into the terrible issue of the best measure may be
proprietary, but you cannot use it.

DR. CARR: Thank you so much for this very rich and exciting discussion.
Could I ask that we could get copies of your power points? Thank you and we
will distribute them. And so I think we are going to break now actually, for a
long lunch break. A number of folks are flying in and will arrive at one. So we
will reconvene at 1:00 p.m. Thank you.

(Whereupon, the meeting was adjourned for lunch at 11:32 a.m.)

A F T E R N O O N S E S S I O N (1:10 p.m.)

Agenda Item: Current Measure Development, Endorsement,
and Adoption Process

DR. TANG: On behalf of the Quality Subcommittee of NCVHS, I would like to
welcome you back to the second part of today’s program and we had really
good testimony this morning. I am sure we will be continuing this afternoon
with really excellent panelists. I appreciate them being here.

We talked this morning about the process of identifying high priority
conditions, NQF for example, and the vision of sort of continuous certification
from David Reuben, the Chair Elect for ABIM. Now we are going to move into sort
of the measure development. We talked about there are lots of measures out
there but maybe not enough good measures and so we are trying to hear from this
panel what is the process for getting a measure developed and endorsed and how
can we even encourage, promote, and pull some more good measures out of this
system so that we can ultimately get -– now that we have this window of
opportunity with the HITECH Act and saying we are actually going to pay for
some of these measures in a substantive way in terms of relief or compensation
for implementing EHRs. This may be a big moment of opportunity where we can put
some more effort into this area because there is money on the table so to speak
for the adopter side.

With us today we have Karen Kmetik who is the Director of Clinical
Performance Evaluation at the AMA, and works with Bernie the PCPI, Physician
Consortium for Performance Improvement. Sarah Scholle is the Assistant Vice
President for Research Analysis at NCQA, a major measure developer, and Dr.
Bernie Rosof is the SVP for Corporate Relations and Health Affairs at North
Shore Long Island Jewish. I think you are the hospital that made news saying
you even up the ante even more, almost doubled the potential incentive.

I think we are really at an inflection point where we can use if we had good
measures, we can really change the face of quality and quality improvements.
And Frank and we will let Justine do because you are such good friends.

DR. CARR: Frank Opelka superb surgeon and President Executive of the
Louisiana State University Healthcare Network, physician executive, and
recognized national leader and patient-centered healthcare for the surgical
patient. Thank you all of you for being here today.

DR. TANG: Is the order that we introduce you a satisfactory one or you have
a different order that would make sense?

DR. ROSOF: Thank you very much, Paul, for inviting me and for the
opportunity to provide some comments, which I hope will be useful to the topic
you have just introduced. From my perspective performance measurement as a
science has many purposes and you have probably heard a few of them already
this morning. It should be an integral part of all efforts to improve the
quality of care. It should encourage performance improvement with the ability
to benchmark individual or a group performance against regional and national
standards. It should advance efforts to support quality improvement at the
point of care specifically by integrating measures into electronic health
records and electronic health record systems, which you will hear a little bit
more about in the a few moments. And when the data is valid, when it is tested
and it is risk adjusted where appropriate, it can be used for public reporting
to help share decision making and ultimately choice.

Now the Physician Consortium for Performance Improvement and other
organizations, are beginning an effort to advance the alignment of performance
measures to be integrated into maintenance certification programs. You probably
heard that from ABIM this morning. All of this clearly will enhance our efforts
to provide patient-centered care, the object that we are all after.

The tools necessary to accomplish this include commitment from the
profession, further research into the science of performance measurement and
quality improvement, education of healthcare professionals providing care as
the value of performance measurement, adequate funding, health information
technology integrated into the work flow of the providers of care, and a true
commitment from the academic educational community to incorporate into the
curriculum of medical students’ learning about the principles of quality
and measurement. No longer is it appropriate to ask the question is it
necessary to teach quality. It is.

Assistance for some of this is already underway with funding appropriated
for comparative effectiveness research, information technology, and a clearly
stated commitment from healthcare professionals, from professional societies,
and of course from the American Medical Association.

How does one go about selecting a high priority clinical area to develop
quality measures? There are competing pressures to accomplish this; CMS and the
PQRI program, consumer purchases for public reporting, well-defined gaps in
measured development as articulated by many medical specialties, particularly
as they become necessary for PQRI and other pay for performance type programs,
and of course accrediting bodies and maintenance certification programs among
others.

Now recognize also that the smaller specialty societies perhaps have a
little bit more difficulty in this area having not had the resources that other
large specialty societies do and so this may add to the problem additionally.

The selection process must include the necessities of developing
specifications for multiple data sources including EHRs are protocol to test
measures, are compendium of clinical guidelines and best practices to
facilitate measured development, and truthfully a plan for implementation never
should forget the necessities as we begin this process for a plan for
implementation.

A hierarchy to enable decision making in this complex setting as determined
by PCPI and others, including the National Quality Forum, includes importance,
includes scientific accessibility and acceptability and the evidence becoming
available, usability, and not only usability but interpretability of that
usability, feasibility, and to determine where gaps in measurement exist.

Now the gold standard would obviously be that the measures would be
generated from a strong evidence base, that they be clinically rich data, and
that they would employ strong risk adjustment mechanism.

Now a real concern and somewhat of a danger that I ought to mention at this
particular point is that we move forward conceptually this kind of a group and
leave behind the essential drivers both absolutely require to affect and create
change and will maintain the public trust. Those are the practicing physicians,
nurses, and other healthcare professionals. We need to be certain in that what
we do we don’t lose them as we move forward in the agenda that we are
about to move forward in and be sure that we incorporate both education and
learning at all levels so they are involved in this decision making in addition
that we don’t make the decisions without them being involved.

If we look at the first slide, what we are interested in doing is increasing
these numbers. Only one physician in five receives process of care data. It
kind of builds on what I was saying just a few moments ago, and less than one
physician in five receives clinical outcomes data. We all know that physicians
like to respond to data. They like to have accurate data. They do not want to
be outliers and they respond very, very effectively to data if it is provided
appropriately.

Also, in terms of meaningful use of electronic health records meaningful use
needs to be meaningful to all healthcare providers and that includes all
healthcare providers. The goal probably would be to engage all physicians and
healthcare professionals from varying specialties in meaningful use of measures
and meaningful use of electronic health records.

Now the PCPI is convened and staffed by the AMA. Let me go through this
although many of you may know this already. Membership consists of more than
125 national medical specialty and state medical societies, ABMS and its member
boards, CMSS, the Council of Medical Societies, AHRQ, CMS, Joint Commission,
NCQA, and NQF, are also active participants and represented at PCPI meetings.
Experts in methodology are an integral part of the PCPI and 13 non-physician
healthcare professional organizations participate actively at PCPI meetings. In
addition we now have a healthcare consumer/purchaser panel, which I will chat
about in just a moment.

Our current measures portfolio consists of 42 measurement sets, 260 plus
individual measures, and approximately 70 percent of the measures in CMS PQRI
were developed by the PCPI.

In terms of the consumer/purchaser panel, fresh out of the press we had a
meeting with the consumer purchases, Peter Lee, Debra Nest, David Hopkins who
were part of that meeting. As part of that discussion we brought up the issues
of measures that matter, consumers and purchaser’s views of uses and
users. There was some consensus related to this, so I thought I would bring
that to you this morning and the consumer purchases felt and that would be a
good opportunity to make this part of the presentation.

The consumers and purchasers want measures that are useful for
accountability and for quality improvement, performance improvement. These
include outcomes, for example, functional health status, morbidity, mortality,
et cetera, composites of multiple process measures, resource use, care
coordination, patient experience, measures that taken together provide a
comprehensive picture of provider’s care that has become more and more
important as we move this forward, measures that show gaps and/or variation in
care, and also measures that show disparities among different populations. We
are the gaps in equity that impact the delivery of care and impact
specifically, outcomes. A very important point, maybe a little difficult at the
moment to include as part of all measures but has to be a goal as we move
measurement forward.

In terms of the criteria for a topic selection the required characteristics
I mentioned briefly, but important also are gaps and variations in care.
Remember that the measures we have been able to accomplish up to the present
really are measures that are more appropriate for large specialty societies.
The small specialty societies have been not involved as much, but must be and
those gaps are obvious and we will move forward to correct those gaps. Evidence
base and high impact.

In terms of high value characteristics, care coordination is extremely
important. Patient safety is involved where care coordination is eliminated,
specifically when it comes to transitions of care. So care coordination is an
important aspect, patient safety and appropriateness and overuse. As you know I
chair the Overuse Committee for the National Priorities Partners, and part of
the work flow for the PCPI going forward is to coordinate the effort between
the National Priorities Partners and our development of measures as we move
this forward.

Our portfolio of measures, again, to make mention as you can see. There are
measures across all specialties recognizing once again that there are clear
gaps that we will make efforts to fill as we move forward, but we have tried to
create measures that are cross specialty and cross cutting so that it can be
used by a variety of practitioners.

Now the current work plan is to fill those gaps to include specifically
appropriateness topics. Now we can say appropriateness. We can call this in
addition, overuse and appropriateness. For example, the surgical and
nonsurgical management of back pain, which is a target that we have with the
National Priorities Partners, a percutaneous intervention for chronic, stable
coronary artery disease, maternity care, for example, induction of labor and C
sections, sinusitis with antibiotic prescriptions and sinus radiography, and
diagnostic imaging. Diagnostic imaging seems to be a theme as we move forward
in the overuse arena.

Important to recognize when we discuss overuse, it is always overuse while
delivering appropriate care. It’s not overuse in itself as a single word,
but it is overuse in the delivery of appropriate care. We don’t want to
misuse the word overuse without understanding that this is for the delivery of
appropriate care.

Currently also in the work plan is care coordination. Phase I is transition
from hospital discharge to home or other facility, transition from emergency
department, this area of transition, which is so important, and phase II
transitions across ambulatory care, outside of the hospital environment where
transitions are as important.

I think if we look at the current work plan to round out our measurement
sets, let me give you an example for the Heart Failure Measurement Set because
that is exactly what Karen is working on or has been working on recently, they
want to round that out by including outcomes, intermediate outcomes, and
process. Once again we get into the area of appropriateness. But we can’t
always talk about appropriateness and overuse without considering under use in
addition. There are many areas. A quick example is in the treatment of
pediatric asthma where under use of medication is a specific concern and so
under use is sometimes as important as overuse. And also we want to be certain
that we include both the inpatient and outpatient arena.

As we move forward the current work plan is also to integrate into EHRs.
Karen is going to talk to you about that in a moment. We are working
collaboratively with NCQA. We have a very specific collaborative relationship
with NCQA in developing certain measures particularly the measures of
appropriateness, and with NQF once again to try to coordinate and align our
efforts with their efforts in the National Priorities Partnerships goals.

We are working with the EHRs vendors to incorporate the use of measures
within the electronic health records, and also we haven’t forgotten the
physician users of EHRs. Actually here is the group that is going to implement
and make things happen we hope.

That is briefly the overview I would like to present. Karen will go into a
little more specifics related to some of the work we are actually doing. If
there are any questions we will be happy to answer that in addition. Thank you.

DR. CARR: Thank you. That was a great overview. I think what we will do is
go through all speakers and then come back with questions.

DR. KMETIK: Thanks very much for the opportunity to share some further
information with you about what we are trying to put in place and
operationalize so that we do have those measures that are meaningful to
different stakeholders and we are getting them into electronic health record
systems. On the next slide is where I want to overview with you today is a
model that we put in place into our measured development process to try to move
forward in this arena.

I also wanted to throw out a proposal about how we might take some of the
lists that are out there now of measures and maybe make it a little more
tracking of progress, a readiness kind of list, and then lastly, to go into a
little more detail about Bernie mentioned about specific areas that we are
focusing on to try to fill the gaps.

When we thought about this activity of building on the core foundation of
the PCPI, which is we have all those specialties around the table. We have
those 13 other healthcare professionals around the table. We have a consumer
purchaser panel. We have consumers, patients, and employer groups on every work
group. We have that foundation. How can we move it forward in the right
direction?

We put this model in place where we said for thinking about the measures and
putting them into electronic health record systems and taking advantage of that
new rich clinical source that we have all been hungry for so long, we said
first we are going to start with our core base that we have these folks around
the table and we have this growing portfolio of measures across specialties and
subspecialties.

We want to make sure then for the measures that they work on that they are
covering those critical areas. We want to then make sure we are developing the
appropriate specification so that those measures can be integrated into EHRs.
We want to vet that with those physicians who already have the EHRs, as well as
EHR vendor community, sit down around a table, and roll up our sleeves. Does
this work? And then our notion as part of our model is to have a series of what
we call incubator groups. I will explain a little bit more about that. That is
sort of the model we have in place to make sure we have the right people at the
table where advancing, evolving the types of measures that everyone is looking
for and we are building them in a way that they can be integrated into EHRs.

If I just go to the next slide, this is just a visual representation. Now, I
am from Chicago and we lost the Olympics, but we are in denial. We are using
rings in everything that we do because we are just in denial. But this is just
a physical representation of the model I talked about where in the center there
the one and two, that is our body that we work with to develop the measures. We
specify the measures. We then vet them with the EHR vendor community in circle
number three, and then around the periphery there the fours, are these
incubator groups, which we think are so important and again I will explain
those in a minute.

In the next slide if I just take an example and I am building off the heart
failure example that Bernie mentioned. This is a measurement set that we
developed with the ACC and AHA and it has been around for a while now and
primarily up until now included process measures.

What you will see coming out soon then is a new set of measures for heart
failure that includes inpatient and outpatient, as Bernie said. It includes
measure of overuse. It includes a functional assessment type of measure. We are
building them out as we said and it is really gratifying to be able to pull
those groups back together who have learned and are now ready to go in that
next direction.

We are also benefiting from some of the researchers. I don’t know if
you have seen the article by Steve Persell from Northwestern who is saying,
let’s take of that data. Maybe it’s not just what is the blood
pressure but how many drugs is that person on. What are the doses of those
drugs? What have you done recently to try to control that blood pressure? We
are trying to take advantage of the research as well. If I look at one measure
out of that set just by way of example, is the ACE/ ARB measure for LVSD.

What do I mean by saying we are trying to develop the right specifications?
In working with Floyd at NQF, we are using this language so we can talk to each
other and not get tripped up too much. We have level 1 EHR specifications we
call them. That means we are going to take that measure in words and we are
going to turn it into all of the code sets, the algorithms, the calculations,
the rules, that one would need to be able to use that measure in an EHR
environment. And I will be honest. Right now we are going to provide every code
set option because the nation has not exactly landed on the code set nor have
these code sets been built into every EHR product yet. We are tracking that
with HITSP and others, but right now we are going to put it down there so
everybody can see. We are going to look at are there SNOMED codes that work,
are there the ICD-9 codes that we need, are there CPT codes in the area of
drugs. We have got to give both NDC and Rx Norm right now. Happy when everybody
jumps on one bandwagon we will be right there with everyone. We are giving the
CPT-II codes also, as an option as well as some of the codes there that can be
used for the exception reporting. That is what we call level 1. That is now a
product of the PCPI and we are doing much of that in concert with NCQA and NQF.

Level 2 specifications we call something different. That is putting it into
-– I just call it an IT savvy format and that is for the programmers at
the EHR vendor companies to be able to take our specifications and put them
into their products without having to rewrite code. We came up with a prototype
for that. It is now going to a standard development organization with NQF
sponsorship and we are tracking that and we are ready to take that step when
again everyone lands on what that should be. Those are two parts.

If you go to the next slide then, the evaluation. We think this is an
important part and that is why we built into the PCPI’s model. We want to
talk early and often with the EHR vendors, not that we want to be limited by
what the products are today, but to be able to have that conversation to say
you know, we really need to someday have in that system not only prescribed
data but dispensed data. If you are doing the ePrescribing can that be
connected. We are trying to have those conversations again toward being able to
move toward a next generation of measures. We are having those conversations as
Bernie said, with physician users right now to say how are you using the EHR
now? What pops up on your screen? Where in there are you reminded about the
aspect of this measure and can you report the data? We think that is an
important part of the model if this is going to be successful.

The next slide then talks about these incubator groups that I mentioned,
which are just something that I love personally because it gives us a chance to
really sit down with a group of physicians who have electronic health records.
We give them the actual national specifications. We give them those level 1
specs I talked about. We collect data. We have it sent to warehouse and we can
analyze where are the issues. And again, we think this needs to be part of the
model of measured development that it needs to be yes, the expertise, the
evidence base as Bernie mentioned to have the measures, to put them into the
specifications, to vet them with those who are going to use them, but then we
need some real live groups that can give us that rapid feedback to say you know
what, you described it this way and that ain’t working or how about this
way.

We feel honored that we put this group together actually four years ago now
with some grant funding, different practice sites, different specialties,
different EHR products and that is critical, different EHR products.

On the next slide just share with you a little bit about what we have been
able to find from that. Again, if you just stick with that measure as an
example, the ACE/ ARB measure, discrepancies between the NDC codes and our
measure specifications and the NDC codes that are in different EHR products in
those practice sites. Even if we could have the exact same coding, first of all
it depends on the EHR vendor sending the NDC update to the practice site, which
is on all different schedules, and then the practice site actually installing
the update.

We found that we had errors not because anybody was doing anything
intentionally wrong but because we may have had what we thought was the most up
to date NDC list for ACE and ARBs, but that list had not yet been uploaded in
those practice sites. That sheds a great light on something. RxNorm could
really help with that and conversations I know, are going along nationally
about that.

But we would never be able to articulate this without those incubator
groups, without actually trying it seeing what happens. In this scenario for
this incubator group for the cardiovascular measures we have data now being
sent quarterly from these practice sites to warehouse. They are able to use the
data themselves of course, first and foremost in their practice. We can
calculate performance rates. We can calculate exception rates and we can
validate the data.

In this scenario for exceptions reported to the warehouse, we went in and
took a sample, manually abstracted the data from the EHR, had a hundred percent
agreement. It was reported we found it. The same for if the data warehouse said
the measure was met, we were able to validate that 90.48 percent of the time.
Then we also wanted to validate what appears as a failure. When the warehouse
said this measure was not met, we went back in and actually there we see a big
disconnect. In only 19 percent of the cases when it was a failure in the
warehouse reported to the warehouse, was in fact a failure when we looked at
the data and the mismatch was things like the NDC code. The drug ACE/ARB
actually was prescribed, recorded in the medical record, but it wasn’t
exported to the warehouse because it didn’t pick up the right code.

These are things we certainly want to figure out and solve before we go
national with this and put different aspects of importance on the different
data. I wanted to share that with you and that is what we mean by incubator
groups. We have one cardiovascular care. We have one HIV/AIDS measures. We
would love to be able to have a dozen of those in different disciplines. That
we believe would be very powerful.

I will move now to another item I wanted to mention, which we want to build
into the PCPI process, which is tracking the progress. It is one thing to say
these are the measures that are important and we have covered all the bases of
a set and that is down the left hand column there. You want process. You want
intermediate outcome, cost/utilization so we can track that. But I would
suggest that we needed to also then track the readiness for sort of going live
in a national meaningful use program. This is just a simple way in which we are
trying to keep track of it to say we got the measures. Have they been
NQF-endorsed or is there a chance to go through NQF. Do we have the level 1 and
level 2 specs? Have they been vetted by the vendor and physician communities?
Have we tested it in the incubator group? And then are all the fields that we
need in X percent of EHRs today? There has to be some comfort level we have
that on the measures that we land on, these are ones we want, have all these
steps occurred and then we at least have a comfort level. I am suggesting that
maybe it would be helpful to us as we move toward defining more the measures we
want to define in meaningful use we start to track it this way as well.
Everybody is a comfort level and it’s not well that’s a great measure
but we are nowhere near. Well, let’s see. What pieces along this continuum
do we have or are missing?

Lastly then, there was a comment about we got a lot of measures but maybe we
still don’t have the compliment that we need. Again, we feel like we got
the people around the PCPI table. We are going back to all the groups and
saying where we have sets let’s round them out as Bernie and I use that
term meaning as we did for heart failure, let’s add what is needed there
to make that a full set, bring in the inpatient as well as the outpatient, et
cetera. Some particular areas that we are putting a focus on in addition what
Bernie said just to put a fine point on it is, the care transition measures
that we have done and are through NQF right now going through the process are
from the hospital side to another location or to home. And a big part of our
effort going forward are ambulatory care transitions, just the hand offs, the
referrals to specialists, et cetera. We think that will be very valuable to a
lot of different specialties and sub-specialties.

We are also putting emphasis on pediatrics. I think everybody would agree a
big shortage of measures in that area, and then again looking at the different
sub-specialties that still don’t even have that core yet that is going to
be meaningful for that patient population.

I will wrap up. I am sorry if I have taken too much time, but again our
thought is for the measure development side, to have a model in place to get
where we all want to go and we feel that it includes working with the
specialties and sub-specialties, building on the measures that we have,
rounding out those sets, new measures where we need them, making sure we have
the level 1 and level 2 specifications, vetting them with the vendors, testing
them in these incubator groups, and tracking this progress. I think it is very
transparent then for everyone to know where are we and where do we need to
focus next. Thank you for the time.

DR. GREEN: I am Larry Green. I am the member of the committee, member of the
subcommittee and no conflicts. Is that the routine?

MS. SCHOLLE: Good afternoon. I am Sarah Scholle from NCQA and I am happy to
talk with you this afternoon about the work that we have doing in thinking
about how we can expand our measurement opportunities using EHRs and health
information exchanges. What I wanted to talk about today were the steps in
creating eMeasures and some of the activities that we are working on to try to
develop new measures that take advantage of the capabilities of electronic data
sources and also our process for updating.

Today what kinds of measures do we have and what data have we been working
from? Our measures have focused on retrospective review where we are looking at
care after it has occurred, usually with a single point in time over a set of
period of time where we use a specific threshold; for example, is blood
pressure less than 140 over 90, and where we are thinking about multiple levels
of healthcare but that means different data sources for the same measures
because we are looking at different organizations or people.

The data sources that we are using are most often claims data, visits,
procedures, and labs. To some extent we have electronic lab results available.
Sometimes less frequently, we have clinical data, like the results of labs and
radiology or CPT category II codes or medical records data or patient survey
data. The measures that we have today are framed in our data sources today, and
what we could do with those existing data sources.

But as we look to the future we are thinking about a different measurement
setting and different capabilities, different data sources. We are hoping that
the measurement will be concurrent with clinical services so that we will be
able to influence care at the same time that we are measuring and monitoring.
It will be linked to real time clinical decision support so that we will be
giving clinicians an opportunity to look at the guidelines and say, what should
I be doing. We will be working with the data source that is not dependent on
who you are measuring. We are not using health plan claims data to measure
health plans and medical records for physicians, but trying to think about
electronic data sources that can build up and be used at multiple levels, and
it brings us the opportunity to look at more clinically relevant measures. We
can look at change over time. It is really hard to do in a chart review or with
claims. We can look at actual levels and the amount of improvement, instead of
trying to say did you pass the threshold and then if you are one point below
the threshold we don’t give you any credit.

We can look at multiple values. We can look at treatment intensification. We
can try to stay not just is the blood pressure at goal but also we can try to
take into account the different situations where well we tried. We did
everything the guidelines said we should do and this person just hasn’t
come to goal. We are giving more benefit and recognizing the efforts of
clinicians to really do well by their patients.

The data sources that are available in the future we think will be claims
combined from multiple health plans. There are some real advantages to claims
data. That is where you get the payments and some services that are better from
those claims data but also electronic records, electronic patient surveys,
personal health records. In the future we think the data sources are going to
be richer and that we can even dream of an environment where it is all linked
together and would allow us to really look at care over time, patient in a
patient-centered and population-based way.

What does this mean for measured developers and evaluators like NCQA and
PCPI? Immediately what we need to do is convert our existing measures into
measures that can be used in this electronic environment. At the same time we
need to be thinking about creating new measures that can really capitalize on
what electronic data can offer. And moving to evaluation models that take into
account the electronic data collection and outcome measures, so that we are
able to look at the full range of care.

But there are a number of issues that we need to be thinking about as we try
to move forward. What are going to be the formats for EHR-based measures? Where
do we look for the data in the EHR? Does the diagnosis come from the diagnosis
field in the problem list? Does it come from the medications that were
prescribed from the lab results? And in EHR you can use any of those. What is
the hierarchy for data searches? Does problem list trump medication list
whatever? What code sets should be used? Should it be concurrent or
retrospective? Should we be thinking about care for this individual patient at
this point in time? Did you do the right thing today or should we be looking
retrospectively? One of the reasons that will get complicated if everybody has
sort of a different look back period because we are looking at one patient came
in May and another patient came in December and then we are trying to put that
information together. Should it be visitor population based? How do we update
the measures, the codes? Research changes. The guidelines change. We need to
change the measures. What is that whole process? As we move forward these are
going to be some new challenges for us.

How does meaningful use in an EHR setting in this new world how does that
change our measure development process? This slide shows you on the left the
traditional measure development process from the review of evidence, develop
clinical logic, data sources, evaluate feasibility, field test, specs. In that
traditional measure development we have different specifications depending on
where it is being implemented. We have heat is for health plans and we have
heat is for physicians.

In the meaningful use environment a lot of the steps, the basic steps are
similar, but we are going to have to after we review the evidence and develop
the clinical logic we are going to have to identify the data elements that we
need, where we want to pull them, what are the source codes. We need to put
those into an XML format, a format that is machine readable, test with EHRs the
way that Karen described the work that they have done with these incubator
sites and then provide vendors standardized and encoded measure specification,
machine-ready specs. We are really looking at moving into that process.

There is a proposed draft standard for what this should look like, the
eMeasure or HQMF which is Health Quality Measurement Framework, and that is the
model for how we are would get to this electronic measurement. It is a
structured representation of the performance measures using XML to tag the
elements and this is what would allow us to import data elements and measure
logic into EHRs. This is the specification that needs to happen to allow the
EHR vendor to say I have taken this logic and now I can spit out a report of a
performance measure. What it will look like in practice is that you go from
what is on the left here, which is the HEDIS specification for A1c poor
control. It has all the instructions. And then on the right side that is what
the XML code would look like that. I can’t explain it to you, but Shane
can read it.

What is our path for retooling our existing quality measures? Right now we
have a number of quality measures that have been developed. They have been
endorsed. They are in implementation now. Our steps are we need to make these
ready for pulling from EHRs. We are actually working with PCPI and with NQF to
convert the specifications to basic EHR value sets and logic using the level 1
kind of EHR that Karen described. We will be reviewing those converted
specifications early next year with NQF and beginning to test those measures
and incorporate them into EHRs. This depends on vendors working to incorporate
those measures and for vendors to be able to report those measures out in a
standard reporting framework. That would be the path to get from the measures
we have today into measures that can be reported out of EHRs.

Right now this support from HHS and NQF we are actually doing this. We are
beginning to convert our high priority measures to EHR ready measures. We are
looking at 35 existing measures that would be available for use in 2011 going
through those steps I just mentioned.

That is sort of taking what we have and making it EHR ready, but what we are
really excited about are the new opportunities. With these electronic data
systems we have an opportunity to really think about measurement as enhancing
working on a number of different fronts. If you think about the measurement
process I said, it started with the evidence development and with guidelines
and then we develop measures. But what we have here in this middle of the chart
you have clinical decision support, performance measurement, patient decision
support, and patient education materials. With electronic systems our vision is
that these things would all work together and that the guidelines and that
information would be available to the clinicians and to the patients and it
would be used as the basis of performance measurement and that you would use
the information to track the results and track outcomes, track what happens
because a lot of those guidelines aren’t going to apply to every patient
and a number of subgroups of particular importance of low income or different
racial and ethnic groups are not going to be represented in all those
randomized controlled trials that are used for guideline development.

So having information that comes out of the electronic systems to help us
understand and learn what is happening in the real world, and then being able
to feedback that information into updating guidelines and again updating
performance measures, clinical decision support, patient decision support so
that you really get to an ongoing process of quality improvement that builds on
as we are learning.

There are a number of areas where we are really interested in trying to use
the capabilities of electronic systems to support new measurement and
development, overuse, care coordination, treatment intensification, and I will
talk about these in a minute. I am going to hold the discussion of care
coordination because I will be talking with you about that later this
afternoon.

But we wanted to talk a little bit about the priorities for meaningful use
measures that have come out and where our work or the work that we are doing
with the PCPI is trying to address these issues and where it is building on the
opportunities that EHRs bring to us. In imaging that is where we are looking at
overuse measures building on the kinds of appropriateness criteria that have
already been developing. Patient experience, how can we use electronic systems
for that? Some of these items actually may better fit in tools like the
physician practice connections, patient-centered medical home so they become
structural requirements for how EHRs are used. That applies to some other
things like home monitoring and comprehensive patient data. But we are also
working on readmission measures, care coordination, and things like preventive
services, having comprehensive preventive services.

I wanted to just touch on a couple of these activities that we think are
particularly important. First, overuse and appropriateness. With support from
AHRQ, we sponsored a national working meeting in June to think about the issue
of measurement of overuse and appropriateness. This is a difficult topic. Some
of the early work that RAND did on measuring appropriateness of care that was
done 30 years ago and it hasn’t led to measures primarily because of the
difficulty and the lack of feasibility of doing the detailed chart reviews and
then also the reliance on an expert consensus database and the challenges of
doing that.

Nonetheless given the concerns in the economy and the national priorities we
heard a lot of interest in moving in this direction but that we should proceed
with caution and focus on overuse measures in a particular area and we are
working with PCPI on that, but also thinking about how we might take those
appropriateness criteria, building on research like the work that American
College of Radiology and American College of Cardiology have done, that
required detailed clinical data, often detailed clinical data both from
ambulatory and inpatient settings and to be able to develop new measures and
think about the opportunity for measurement and decision support.

Another opportunity that we are involved in is looking at the Archimedes
model, which maybe some of you are familiar with. What is interesting about
this model and it is very different from the way our current HEDIS measures are
set. This approach combines clinical decision support with measurement of
outcomes. Instead of saying the guidelines say everybody should have a blood
pressure less than 140 over 90, you calculate a specific risk score for a
patient and you adjust and it provides decision support to the clinician based
on that patient’s particular characteristics. We are working with Kaiser
in Hawaii to test this.

Then some other opportunities for measurement relate to developing new
outcome measures that take into account risk adjustment at the physician level
and then looking at treatment intensification and the opportunities for using
electronic survey models to get to patient experiences.

Finally, I wanted to mention the idea of updating measures. In some ways
having electronic measures there should be some opportunities for making
updating more feasible because you can make it electronically available to
vendors but it depends on having a process for updating. NCQA formally
reevaluates all HEDIS measures at least every three years but if there is new
clinical evidence we might update it more frequently. We have updated our
diabetes measures every year for the past three or four years. It is going to
be important that there is support in opportunity to do that kind of updating.
It is an important role for metric developers. Thank you.

MR. OPELKA: For those I haven’t met, Frank Opelka with the American
College of Surgeons at Louisiana State University in New Orleans. Going last is
great because all the important things have already been said. I will try and
shorten this and hit some highlights that might perhaps open up some dialogue
in some other areas for measure development to enhance specific issues for
quality improvement and to help patients with patient decision making as they
move forward through the healthcare system.

I am involved with several different aspects of performance measurement at
home and I am in a 10-hospital safety net hospital system. We have a very
elaborate ambulatory patient measurement system that pulls clinical information
together on about 8 to 10 chronic diseases and then in house, in the hospital,
the inpatient side, we have a very elaborate system that looks at current
measures that are being used and that whole program is in a learning network
across all 10 hospitals for quality improvement. That is separate from some of
the other aspects I will talk about today, which is from the College of
Surgeon’s standpoint, where we have multiple registries, some of which are
very old. We have the National Trauma Registry. We have the National Cancer
Data Bank that the college runs in conjunction with the American Cancer
Society. We have a new registry that is out there, the Trauma Surgical
Improvement Program. We have an old system that is about 10 years old for the
college, but it is about 20 years old given its VA history, and that is the
National Survey Quality Improvement Program. Those are all measurement systems
that we have that are up and running today. Some of them have high validated,
highly reliable data. Some of them have very poor data entry points and thus
are problematic when you get to the point of performance measurements and
quality improvement because of the problems with the data itself.

I will speak more about the procedural-based care because we have talked
about global aspects of measurement and I want to just focus a little bit since
there is a whole different realm of I think, performance measurement when we
get into procedural-based care and it has a much harder drive for outcomes.
This is much more active, aggressive, short care that needs to have some kind
of aspect of outcome assessment. We are very strong proponents of those being
risk adjusted.

The National Surgical Quality Improvement Program currently over 30 days
captures about 130, 135 data elements. About 30 or 35 of those are just part of
the risk adjustment. We have now actually taken that system and deployed it. It
is in 300 hospitals across the country. We find a lot of problems with that
kind of intensity of measurement.

First of all we have learned that we don’t need 35 risk adjustors. We
could probably get by with -– in fact we have shown we can get by with
about six to eight risk adjustors in the care of the surgical patient. That is
a big help.

Secondly, we currently sample multiple procedures, but the sampling is only
around maybe 20 or 25 percent of the total procedures that we are looking at in
a hospital. One of the things we have done is said, can we assess a hospital
and its physicians by decreasing the number of samples we get and increasing
the number within a given particular condition. If we did 80 to 90 percent of
all colons, rather than 25 percent of all colons, and picked up a whole bunch
of other background noise of very low volume procedures, we have actually
intensified the high volume, high risk areas in their sample size and shortened
the number of data elements that we are collecting for risk adjustment, which
allows us to actually create a much more I think, detailed view and a less
expensive view from the hospital standpoint for performance measurement.

I think the second key area once you get into procedural-based care, and I
didn’t include patient safety on here because I think that one is already
out there, but the one area we haven’t put enough focus on is
appropriateness. There are huge challenges in appropriateness of care, but
perhaps we are trying to take too much of the elephant. Perhaps we need to take
a smaller bite and get down to something that is actually more easily
collected.

Then finally I think if you are going to put this into some meaningful
aspect on the patient side and perhaps other stakeholders in this, it is to
create a composite and put composites together to try and bring it all together
on a particular condition for a patient in the procedural-based world.

I have mentioned the NSQIP. It is facility level measurement at this point
in time. We have done some modeling at provider level measurement, but you
really run into sample-sized problems. If somebody has only done 8 or 10 of
something in a given year, can we reliably talk about performance and can we
reliably assess where quality improvement efforts need to be trained in one
individual? We have not found that to be successful. But if we take the entire
group of people in a given hospital who are doing this we can literally make
movements for huge efforts in quality improvement. We have demonstrated that
now by creating several different learning networks across many of our
hospitals that are in the same region. We have a learning network in Tennessee.
They all collaborate. They see their own data and then they see the group
aggregate data and it has made huge efforts towards quality improvement and
enormous cost savings.

If you were today to take all the complications that NSQIP has prevented in
300 hospitals, and put those in the major hospitals across the country, you
would be saving somewhere in the neighborhood of $50 billion to $80 billion per
year in this country in the cost savings from quality improvement. That is not
hitting every hospital. That is just hitting some of the major hospitals. The
potential savings are astronomical.

What comes from this though most importantly, is this is trusted and
meaningful to the providers. The reports they get and put in their hands are
actionable. In measure development we really have got to be thinking in terms
of actionable measures. We have measures out there now in perioperative
antibiotic use and then we try to correlate that with SSI, Surgical Site
Infection. Well, there are so many other drivers for surgical site infection,
perioperative glucose control being one of them, but the patients’ other
comorbidities, all of those factors, the surgical skill and judgment, which we
don’t have a good way of measuring today.

When we look at antibiotics and SSI and we try to make that correlation
since it doesn’t correlate to the provider there is not a huge effort in
there to correct it. It is more to the hospital get this standardized and
reliable fashion but it may not have the impact we are looking in SSI. In
looking at the current CMS data the fact it is showing there is no correlation.
Great antibiotics still had significant surgical site infections. Poor
antibiotic usage didn’t necessarily have significant site infections. We
have to make sure that what we are measuring meets that level of
appropriateness and meaningfulness at the provider level.

We talked about a lot of the comments have been about updating all the
aspects of measures and how we put them together. The risk adjustment actually
requires updating. We have looked at the NSQIP today and we went in and we
looked at the beta weighting of all these different data elements that are
collected. They were based on a VA population and there in the VA population
albumin was a huge clinical driver of outcome, but it is a poorly nourished
population that has a lot of alcohol abuse and it certainly does show up that
albumin is a key driver and it gets heavily weighted. So that beta weighting
needed to be changed to meet a particular market that is being assessed for
that improvement and those kinds of changes are ongoing if you look at the
different variables that are a part of the risk adjustment.

A couple of comments on appropriateness. I think the best way to approach
this is to be condition specific, evidence based. From a procedural basis I
have thrown up two examples just to think about this and Bernie already
mentioned this. Spine surgery for low back pain after failed medical trial in
patients with no urgent triggers. There is a definition of appropriateness and
we could enhance that through a panel of experts with the evidence behind it,
but that is the kind of thinking that if we put that out there as a measure of
appropriateness that we think that that will actually improve the quality of
care that people get at the right care at the right time for the right reason.

Cholecystectomy, a very common operation. But if you look from laparoscopic
surgery forward, there has been a change in cholecystectomy. We used to operate
on gall bladders that had gall stones and if you were diabetic and
symptomatical. If you were diabetic we did it if it was asymptomatic because of
the problems in discerning symptoms in the diabetic. If you were in the ICU and
you septically ill and you had gallstones and an inflammation of your gall
bladder, we called this acalculus cholecystitis and it was an indication for
surgery.

Today a lot of patients present with right lower quadrant pain and they have
biliary dyskinesia. There are no stones present. There is some mild thickening
of the gall bladder. They might or might not have a gall bladder ejection
fraction determined and they might or might not have that gall bladder ejection
fraction related to reproduction of their symptoms. But there is a lot of
cholecystectomies performed for this acalculus cholecystitis and in fact that
is an area that we should explore for –- is that most appropriate. Do we
have the parameters for defining that in a most appropriate fashion? How does
this level of appropriateness merge and fit into the issues of comparative
effectiveness and how do you then push that into patient shared decision
making. These measures are being developed for the entire downstream effect and
the point of care and I think it makes a clinical difference for patients and
it is a big gap in what we do.

Patient specific factors, however, really confound us. That is where
appropriateness gets to be a mess. If somebody has an inguinal hernia, you
could say well it is appropriate to operate on someone with an inguinal hernia,
but then you start to change the story. They have an ejection fraction in their
heart of less than 20 percent. They are 90 years old. They are bed ridden and
that hernia never bothers them. Perhaps it’s not appropriate to operate on
that hernia.

Those factors in trying to develop measures for that level of detail I think
will fail. There are just too many confounding variables to try to put that
together. But a good beginning is to start with condition specific evidence
base appropriateness. I think that can be done in a meaningful way.

I also think that when we start with developing measures particularly in
areas of procedures, we have to look at this as it all comes together so that
when we are beginning the measures we can see how it is actually going to
complement and fulfill what that patient needs and their decision making, what
other stakeholders are looking at and want to see for the patient. There is
value in structure from a procedural basis. That value may be volume driven. We
know that there are some procedures out there that have volume association with
them and we have to be mindful and respectful of that on behalf of representing
that to patients.

The process measures. There is not a lot of enthusiasm for these because we
can’t cleanly tie them to outcomes, but it is still a good important step
in creating standards and reliability and care. It has a role but I don’t
weight as heavily as I would risk adjusted outcomes. However, I still think
that measure is at the facility level. When you build the composite the first
two structure and process are provider level and some facility level but
outcomes are more facility level based.

We have completed and just put into the public domain or it is about to be
launched into the public domain through AHRQ, a surgical CAHPS, which is much
more specific for the surgical patient. It was developed by those patients
through a proper facilitator to represent those things that they are seeking
and not the general CAHPS that you see in the hospital, which if I tell you
much about the parking and the painting but not so much about the expected
outcomes and did we achieve that and were you apprised of what you thought you
needed to be apprised of when you went into your operation now after your
operation. And then the appropriateness I mentioned, and then finally
efficiency.

Currently under the NSQIP forum we can measure efficiency. We can add in
appropriateness either within the registry itself today or we can add it in a
parallel registry that is about to be launched in conjunction with NSQIP, which
is referred to as a Surgical Quality Alliance registry. But these are different
pathways than these other pathways we are talking about for measures, and yet
all these pathways are important. We have to bring all of this together to
represent the structure and process with these risk-adjusted outcome measures.

That completes my remarks about this. The last thing I would say is that I
think from a procedural basis as we develop the measures not just based on
priorities that we set but this has to be in conjunction with the payor
community. We have gotten a lot of information and help by sitting down with
payors and asking them specifically even market by market, where are they
seeing variance, where are they seeing gaps, and how do we partner with the
measures that they have to complement those with the clinical measures?

We are working with the vendors who run the current clinical databases and
all of those now are dropping patches into the EHRs that run through the
practice management system, to at least bring in some of the initial data. We
can pull in all the demographic data and reduce the burden of data aggregation
through those patches. That is working. To get it to the point where we can
pull in necessary clinical information such as in NSQIP, we have actually had
to use certified nurses who are certified to the process to do that data entry
so that we have auditable, reliable integrity in the database. Thank you.

DR. CARR: Thanks very much. A really excellent presentations and remarkable
synergy in terms of where we are headed. I would like to open it up to
questions now and I will start with Paul.

DR. TANG: Thanks to the panel for excellent discussion. Maybe starting with
Bernie in terms of some of the measures, what do you see the timeline or maybe
the process of transitioning from the kinds of – well, actually I think
Karen addressed that in terms of you are working on the EHR-derived measures.

One topic that came up earlier from Helen Burstin actually was the whole
ACE/ARB. Let’s say you have the ACE/ ARB measure. How would you propose we
account for let’s the ACEs on the formulary but the ARB is not and they
have an adverse reaction to the ACE? Is there a way to account for those kinds
of things or how are you viewing that in terms of measured definitions?

DR. KMETIK: I don’t have the specifications in front of me but I am
pretty sure we acknowledge that with ACE/ARB there is a way to document
intolerance.

DR. TANG: I really liked your matrix where you talked about checking off
where a measure was with respect to the meaningful use. Have you by any chance
done that for all of the MU measures?

DR. KMETIK: We are in the process. I would be happy to share that. I think
it makes an important point -if I could elaborate on that a little bit, which
is we all appreciate the need for deadlines, but to arbitrarily sort of and I
don’t know if it is arbitrary but to pick a date and say that’s when
we go live is troubling because we want to all be assured that we are there and
I think that tracking exercise helps to acknowledge where are we and we can
focus our efforts on the things that still need to happen and let that drive
the appropriate dates of things.

DR. TANG: I got to go back and say fine, that Medicare trust fund is going
bankrupt and it is fine to say let’s wait until we are ready. How do we
make that compromise with when do we think at the current pace we can be ready
versus when could we be made to be ready?

DR. KMETIK: I’m not saying we sit back and just wait until it all
happens. I am just saying that laying it out that way I think, makes it pretty
transparent to all of us and in some cases we are going to need to set some
dates and say this what we need to achieve by then. But without that
specificity, you tend to get people a little frustrated sometimes and they will
throw up their hands in frustration. I think it is better to articulate this is
the goal – this is the goal we want to reach. Here is where we are in
different things so let’s try to put some aggressive but reasonable dates
to each aspect of that because without the realities sort of tracking
mechanisms, it is hard to engage in a conversation.

DR. ROSOF: I can also say that this is a resource intense kind of work
especially with the extensive number of measures we have on our plate, plus the
measures that the consumer purchases would like to put forward, et cetera.
Adequate funding obviously is an issue to help Karen along in this quest.

DR. MIDDLETON: Let me join Paul in thanking you all for really great
presentations and thanks for all for coming. I guess I am sort of sitting here
wondering where is the break through. If each of you could respond perhaps with
what you see as the top one or two or three, break through opportunities for us
to develop measures, to accelerate the development of measures, to implement
measures. I am feeling a little under whelmed so far, in the sense that we have
a lot of activities going on. We certainly know that we haven’t impacted
to a large degree yet, behavior change and change quality in outcomes in cost
of care. Try to be a little bit provocative, but gently, what are the top three
breakthroughs you would like to see in your own work or the work of others?

MR. OPELKA: I think that is a great question and in fact I think we have
asked that of ourselves many times. I think that one very important
breakthrough that we have to have is harmonization across the payors. It’s
just not working to have multiple different payors hitting these providers with
multiple different quality programs. It is highly inefficient. If we want to
talk about removing efficiencies or improving efficiencies in medicine, let us
not be the cause of more inefficiency at this process. We lose credibility with
the providers immediately. They already see these unfunded mandates coming in
as one more burden and it’s not an opportunity for quality improvement.
They really want to give good care. I have never met anyone in medicine who
didn’t want to give the best care everyday and if these things come in and
they are just burdensome and they get one from United, from CIGNA, from Blue
Cross, from WellPoint, from all the different plans and then there is CMS. They
are not harmonized. That is not helping the American patient one bit.

I think that is going to take government intervention to tell everyone you
are going to get in a room. We are going to get this together. It is very
important.

Secondly, just from the surgical perspective, we do have –- it’s
not compelling evidence, it is overwhelming evidence, that we have improved the
quality of care in the NSQIP hospital. It is huge. It is not longer denial. The
return of the investment is also there. Business case. So the up from funding
is returned. Good quality care is cheaper care and not only that, there is
massive buy up that the surgeons love this. There is information that comes
back to them is scientific that they see risk-adjusted data that is being
shared. They see themselves. They see their hospital, and then they see here
are the high-performing hospitals, here are the low-performing hospitals, and
there is this gaggle in the middle. But that starts the learning network and
the quality-improvement process.

I would say there is an upfront investment to make that happen and I think
that is also an all payor responsibility. I don’t think it is a government
responsibility alone but I think it is an all payor responsibility.

MS. SCHOLLE: When I think about the breakthrough I think about two pieces.
One piece is having the information that we are collecting be something that a
clinician can use and can explain to a patient. That is the piece of trying to
look at how these measures can tie to decision support and could be reported
out to patients. Choosing a small set of measures that you can really build
into, get out of the problems of free text and other stuff so that you can
actually get a report that is meaningful to the clinician that there is
decision support that can help guide that clinician and that they can also use
that to help patients understand what their care needs to be I think is an
important piece.

That can only happen when you have an HIT system that can support the
collection, the use of these data elements into a measure and when you have a
workflow process that supports that and the work flow that staff that are
trained that have responsibility for printing out the report or for entering
the data and using that. Our experience as we have been looking at measurement
with the EHRs is that those are the critical first steps and so that happens by
having structural measures that say this is what the system should be capable
of doing and these are the tasks that the team needs to be able to perform and
need to be assigned so that you can get those reports that they can build into
your decision support that they can be shared with people so everybody is
working on the same page and maybe trying to do that on a few really good
measures that makes sense to everybody would help us to understand how to do
this bigger and better for different commissions.

DR. ROSOF: So as Frank said that is a very good question and it is a
question that I asked when we started doing the work on overuse not knowing
where to start. I started with the clinicians and specialty societies. Their
response to me was try to eliminate the confusion. Try to develop national
standards that would be agreed upon by the providers, by the payors, the
consumers, the specialty societies, and the boards. Yes, you can build that
into my workflow. That would be terrific so that I don’t have to do other
things during the day but it would harmonize current performance measures and
perhaps I could use the same for my maintenance of certification as I would for
my pay for performance, PQRI, et cetera.

That whole issue of confusion amongst the already overburdened clinician is
something that we would really like to accomplish. If I could look at one
thing, I would say let’s try to develop national standards agreed upon by
all those stakeholders.

DR. MIDDLETON: This is really terrific because I think you guys are laying
out kind of the inner thinking behind your presented thinking, if you will. I
wanted to follow up Frank with you. In a way we had a conversation this morning
with David Reuben from the UCLA and ABIM, and it was very interesting to think
about how in fact measurement of physician performance through quality directly
or indirectly through other means, process assessments, what have you, may
raise the issue of whether or not we are establishing a floor of competency or
whether or not the process can be turned on its head to establish a goal, an
aspiration or reach or stretch goal, which then changes the dynamics, not only
the human dynamic and the psychology, but also the measurement process in
responding to measurement and assessment as a tool for me to continue self
improving and lifelong learning and a learning network as you described, as
opposed to the kick in the butt when necessary if underperforming or over
performing untoward ways.

I wondered if you could comment on the learning network and whether or not
there is an individual component to this, which has this feature or this
characteristic or whether or not you think something like that might be
possible?

MR. OPELKA: I have had this discussion many times and with many of our
stakeholders and it is fascinating, one of the stakeholders Peter Lee, he and I
came to the same conclusion that at a national level we needed a baseline. We
needed the floor. But at a personal level, we needed the upper 10 percent, so
you have to do both. I don’t think we can escape that. I think all of us
want to know where the floor is, that you have to be above this. If you are the
outlier on the low end for a particular condition or treatment of that
condition, it has to be identified and the public has a right to know. This is
their health. And at the high end, we need that so that we can get into those
systems, look at those systems, and see what is happening.

What is so different about that system because most of those successes are
not one individual who is doing something absolutely extraordinarily great and
miraculous. It is the fact that they put together those systems that created
the standardization and reliability. If you look at what Geisinger does that is
so good, all their goals and all their targets are standardization and
reliability. It is nothing more than that and they let performance of the
individuals distinguish themselves beyond that. I think you have to have both.

The reason you have the learning network isn’t so much to ferret that
out because that is intuitive I think in everybody. Nobody wants to be on the
bottom. Everyone wants to be on the top. But we all are coming to work already
thinking we are doing the right thing. Medicine moves and changes so fast,
particularly in this day, that we have to teach each other on a continuous
basis going to a monthly or even a semi-annual society in listening to some of
the latest and greatest doesn’t translate into getting back into home.

I think the quality improvement efforts have to be driven by these measures
that these are specific to where our issues are and then we have to own them.
We need a team effort to own that to figure out how we are going to change. It
is very hard for an individual who fully believes in what they are doing to
change a law. You need the learning network.

DR. KMETIK: I want to just echo the themes and maybe say in terms of a
breakthrough. I do worry that we are missing in our vocabulary today in our
conversations about meaningful use et cetera, both these pieces that I think
you are getting at Blackford, and that Frank has articulated. I worry that we
have had past national programs where the emphasis has been on get the data
somewhere and we have done that. We have done that exporting.

That is done pretty little to get timely data into the hands of physicians.
I think the vocabulary needs to make sure we are giving those two things equal
weights. We got to get the data in the hands of the physicians. They are going
to use is as Frank said. If it is good data they are going to react to it. We
understand the need for exporting data and sending it to others but if that is
all we do and we haven’t built this first, we have missed the golden
opportunity but we have just replicated what we did with claims and that would
be so frustrating I think to all of us if we look back five years from now.

DR. TANG: I am picking up on Blackford’s and I really liked this last
discussion. I think Frank answered the question about why the floor because I
challenged ABIM on let’s stop raising the floor and let’s start
shooting because I find that individuals and is to your point of facility
versus individuals, maybe the floor is for the facility but the individuals
react very positively embracing on even transparent measures about themselves
and that has been our experience.

The breakthrough might be to make sure that our quality measures align with
the physician’s line, which is towards the individual. I am finding less
and I can understand why we developed failure quality measures. I just
don’t see much room for them anymore. An example is the hemoglobin A1c
greater than nine. It doesn’t tell me any way – it doesn’t tell me
how close I am to the goal of less than seven. It seems like we want more
measures that help individuals because I think the public reporting and that is
what the evidence shows is that consumers maybe because we don’t present
them in a usable way, but that is not what is changing their choice. It is
actually changing the docs behavior to more improve it. Maybe that is an
example of an improvement. Design the quality of measures to help docs perform
at a higher level.

The second potential breakthrough and this in reversed order because Floyd
is going to talk about the quality dataset and I know some of you are familiar
and some of you aren’t. Do you think that kind of -– because you
talked about efficiency and each health planning to have their own definition,
et cetera, and the problem of getting that into the EHRs and clinical decision
support. Do you think the idea of a QDS, a quality data set, that all your
organizations could draw from would be one of those breakthroughs to the extent
that you understand what is going on there.

MR. OPELKA: Let me just ask a question to help me answer that, Paul. If I
were to get a feed that gave me a list of data metrics, that feed could then be
mined locally even further.

DR. TANG: It is more that the -– somebody showed the supply chain and I
don’t know if that was Helen from before. Was it Helen? Okay. You start
out with creating the evidence, to creating the guidelines, to creating the
things that would affect behaviors like the rule sets, clinical decision
support to creating the quality and the performance measurements and then
feedback. Each one of those steps is done by different part of our sector and
they all invent their form of the data definitions and the measured
definitions. Wouldn’t it be great if we had one place where everyone could
go and draw out from the very start of the design of the protocol and the data
you collect all through and spit out the quality measures. In that sense that
would facilitate the data and hopefully would agree on 6 instead of 35 data
elements to feed into your registry, and case adjust these things and spit out
the quality measures that would go back to the EHR. Maybe that is the
bi-directional aspect. The outcomes that you get from your registered data
could sit in front of my face at the time I am making these decisions.

MR. OPELKA: I am thinking of two things with this right now. I have to
measure data sets that one is this LSU data set on, take my diabetics. Our
diabetic measures are pulled from the lab. The hemoglobin A1c is pulled from
the lab and we track less than 7, 7.5, 8, 8.5 all the way up to 10. When we
report we don’t say anything. We give an individual doc all their scores
and the system has all their scores, but the system then was allowed those
interested parties to mine the data further. When we mine the data further is
where we found the answer.

What we noted in our own population was — just as an example, if you were
over 65 in our system your hemoglobin A1c tended to be pretty well controlled.
You were more mature. You paid attention. You read other problems. Your doctors
were important to you but for the group that was under 35, which just happened
to be the musicians of New Orleans, the lifeblood of the city who drink and
smoke all night long but make beautiful music, none of them cared much about
their diabetes. Then we put in improvement programs focused. In this instance
performance measurement to physicians meant something because it came home to
treat and make a patient better.

Looking at that data if the data allows me to re-mine it and the NSQIP data
does give you a dump and you can do a lot of mining and re-looking at things.
If there was a system whether it had the parameters you could pick and choose
from allowing you to get the other analytics so you can do a deeper dive. As
you know that is how physicians think. Okay you give me this but now answer
this, this, and this.

DR. TANG: I think that is exactly right. The QDS would allow you to have
this is an A1c, this is what it means, this is how I calculated it and how you
interpret it so you can do that mining that you suggested. The breakthrough I
think I am thinking about is our quality reports – the quality measures we
currently report on actually take away that ability and that is the fallacy. I
think the breakthrough is to report differently. In our organization where we
are improving quarter by quarter because of the timely reporting back to the
docs we have a separate team to create the measures that we use and then a
measure that we have to report to all the various definitions that the health
plans have and that seems counterproductive. Maybe the breakthrough is pay
attention to what it is that will change the physician behavior. I think the
rest will follow versus the other way around. Report to health plans and then
expect physicians and patients to change.

MR. REYNOLDS: Frank, you commented earlier about the payors. The quality of
whether somebody has quality shouldn’t decide on whether they are a blue
patient or an – patient or something else. As I sat and listened to this
and I do a lot of standards work, I sit and listen to this and I hear we a lot
around the table, and every single presentation has been fascinating to me, but
each one of them has a different slant and even the most recent discussion in
Paul’s comment about what they do in their institution.

How do we take what all of you are working on and again as you look at the
chart from ONC and you look at the states that everybody in every state is
asking for money to go ahead and implement these things for all their docs and
everything else. What is the process that you all see that allows this to be
raised above whether it is payor provider or anything else, and you set the
quality standards and you have it in an open forum and as you say it is agreed
to like a lot of the other standards that go on and then once that is done then
–- so for example as a payor, I’m not speaking for our company, but
it would be awful nice if there was a set of those out there that everybody
agreed on that was the right thing and oh by the way it played for Medicare, it
played for Medicaid, it played for everybody else and you can use it and
improve North Carolina one patient at a time, regardless of who they are
covered with.

But in listening each one of you is doing something really good and even
some of our members on the panel are doing good things, but left as it is then
others are going to come and say, here is what we are going to do or here is
what a state is going to do and everything else. That is what worries about
getting to the meaningful use and getting to the things that are going on is
that really, really good work but no process to actually vetted in an
environment. Being NCVHS we get stuff all the time. Not recommending that there
would be it but it is vetted. It is vetted in one environment. It is vetted in
a way that is agreed to, and then once it is agreed to it could be adopted by
HHS or others, and that is how we start rallying around.

I chair a committee of all the payors on eligibility claim status, the same
kind of thing. You know you get the one. Any comments you could make because
that could be a way NCVHS could make some recommendations as to how to help you
get to that point, if that is a worthwhile point. If not then it is probably
going to be one state, one payor, one issue, one situation at a time, and
that’s not going to get us any of us I think where we all trying to be
when you put up any of these thoughts.

MS. SCHOLLE: I think that is the major challenge. We have had with HEDIS, is
that everybody wants to make a little bit of change specific to their state or
their population, and it is really hard to get people to agree to a standard
and working with NQF to get to endorse measures should really help that. But I
think this idea of having standards and then getting vendors to agree to put it
in because we have had – the vendors don’t want to put the resources
into any kind of measurement standard unless they think this is going to work
for everybody.

MR. REYNOLDS: Where do you see the process that says vendors you should do
this? I know some of you talked about meeting with vendors, but where is the
one body that says okay, this is it guys, go do it?

DR. TANG: We had a discussion of this at lunch. This moment or opportunity
where the vendors all said, well our customers aren’t asking. Well all of
a sudden because of the meaningful use dollars the customers are asking the
vendors, which the measure developers could never get the vendors to do. Now
the vendors have to listen to the measure developers because –

MR. REYNOLDS: If the measures are accepted across the board as that’s
what it is and let’s go.

DR. TANG: That is our moment of opportunity and I think maybe this hearing
is a plea to the measure developers and the people on that supply chain, hey,
this is time to hit the breakthrough measures and meaningful use is the excuse
to do it.

DR. ROSOF: This is not a new discussion, obviously. When I talk as a call
for national standards it was the discussion. At the beginning of the effort at
health reform this discussion began. Who is going to be the organizer of
national standards? Who is the recipient that is going to make the rules that
this is the national standard for X, Y, and Z or for the vendor situation?

That led to a lot of confusion, internal concerns, et cetera and still not
resolved. There are some answers that could be accomplished. Organizationally
this could be accomplished within the framework that is already established;
however, it would have to be agreed upon by so many stakeholders that unless
there is somebody who says you must do it. Unless there is somebody who says
national standards are the gold standard, it’s just not going to happen
meaning the discussion started perhaps with national quality forum being the
individual who would be the group that would handle that that created a lot of
discussion and didn’t go to where it was going to go. The same thing with
the national standards related to development of measures. PCPI could have that
role. There is no question about that. It has the background with the medical
specialty society, the clinical side, the methodology side, et cetera. It is a
group that should have that responsibility. It would have to be agreed upon by
so many stakeholders that it becomes – it is not a forum where you say yes
or no and 50 percent you win or 60 percent like Senate. We will see.

DR. CARR: A quick follow on and then Larry –

DR. MIDDLETON: Just two quick thoughts. Just a quick reaction is sufficient.
In a way I am struck with the assertion that the physicians of course are
interested in providing high-quality care and aim always to do so but of course
the result isn’t that. I am concerned about we are developing a new
measure of framework and all the stuff we have been talking about but absent a
sustainable business case for quality. I am just concerned it’s not going
to happen. Not to suggest that docs are mercenaries who are working only for
quality or the almighty dollar but could you each react to you what is the
sustainable business case for quality post stimulus?

MR. OPELKA: I think you have to stop paying for volume. If you are going to
have a quality system and you want this transparency in all the key pillars of
healthcare reform, there has to be a realignment of incentives across the
board. Until you do that I don’t think all of this stuff will go exactly
where it has to go and be as successful as it can. I also think you must
address the issues of defensive medicine. That if you are going to incentivize
me and my practice by $2000 of PQRI money and I put myself at risk for a
$20,000 increase year after year in my malpractice because I no longer practice
defensive medicine, and I was caught in a court system that isn’t educated
to quality metrics, it is easy for me to decide which risk I am going to take.
I am going to keep my malpractice insurance rather than $2000 on quality. That
isn’t necessarily something I bought into as the best quality.

You have to have meaningful quality and really something that can at the
local level create quality improvement, excitement about the practice of
medicine because of the state and this information. You have to have aligned
incentives that have to include payment realignment that fits that and you have
alignment incentives that correct the disparities that we see in the way we
practice defensive medicine.

MS. SCHOLLE: I think you also have to think about the accountability not
just being with an individual clinician for an individual patient of individual
point in time that care happens over time to patients and they are getting care
from different people. The accountability needs to recognize that it’s not
just the surgeon. It is the primary care physician, the hospital, the health
plan, that are all taking responsible for the care for this patient. That is
the other kind of financial alignment that needs to happen particularly since
some of the measurement that you might want to do at the clinician level it is
useful for quality improvement, but in terms of knowing – it is useful to
let that clinician know what is happening with the patients but to really get
something that is stable, you are going to need a lot more observations.

Like Frank was talking some things make more sense at the facility level.
Some things are going to make more sense at a community level to try to
understand how well the community is doing; the community of healthcare
providers and the healthcare system are doing for a population of patients.

DR. KMETIK: I will just add a little bit to Larry’s previous point as
well. I think we should use this moment of meaningful use. We should leverage
the heck out of it in a thoughtful way because it has enabled us at least to
begin to have a different dialogue with the vendors then before, as Paul said,
step one.

Another important step though is I think we can begin to have the different
conversation with the payors because as opposed to an old conversation of you
want to merge your big databases that you have. It is a new day. We are both
getting data we never had before. If done right, the physician has data they
have never seen before at the moment of the patient care and the plans, the
payors, they never had cancer staging data before. They never had prescription
data to marry with dispense to see is the patient picking up the drug or not.
Again, our vocabulary maybe needs to have a shift to say what do we all win
here if we do it right as opposed to just we are going to get backed and that
is what you have to do. What is the thing where we all win and it is
information we never had access to before and that is even important to the
plans.

DR. GREEN: I want to try something here. I want to ask all of you to respond
to questions that in my mind tie back to prior NCVHS work, the work of several
of our subcommittees over the last few years. All the questions I am going to
ask are capable of being answered with yes or no. The first question also I
built off of Sarah’s slide for she had the word dream and she also had
another slide that had the word stewardship on it. This question would be are
you experiencing in the current measured development endorsement and adoption
process difficulties that relate to privacy confidentiality or a research
community would call IRB issues. Yes or no? Or is this a nonexistent problem.
Privacy and confidentiality are not in play here.

DR. ROSOF: The answer to your first question is no.

MR. OPELKA: Yes.

MS. SCHOLLE: Yes.

DR. GREEN: Two yes’s, two no’s. Then we in the last hour and a
half heard Harry’s “we”. Does the “we” share a common
data model? The “we” of the vendors, data developers, the payors, the
purchasers, the government, providers, et cetera. As we massage this thing
anywhere lays a common data model. We are all working on that type of data
model.

MR. OPELKA: No.

DR. GREEN: No, that doesn’t exist. Then what was not said in this
section is there current active work on measured development endorsement and
adoption of measures of function or a patient feeling like they got some
relief? Four yes’s. Thank you.

DR. TANG: As a closing comment I just want to thank this panel so much
because I think there was such really good information and discussion. The
other part is maybe on an upbeat note. Bernie said that you need this power. I
think that you have more power than you realize because of the moment and it
really is leveraged through the meaningful use and really NQF because it is a
voluntary consensus standards organization, CMS has to use those when they
exist. The measure developers have a lot of power at the moment. Maybe that is
the upbeat way of saying this. I think the work is really good and to just to
accelerate it not because it is comfortable to but because I think the country
needs it.

DR. ROSOF: Some need the encouragement to utilize that power.

DR. CARR: With that I think we will just take a 10-minute break and then
reconvene for Floyd and Blackford and I add my thanks to this group. I know you
made this timing work and I appreciate it very much and thank you so much for
enlightening us.

(Break)

Agenda Item: Building Meaningful Measures –
Adoptability

DR. CARR: We are running a bit behind but really very enriched for all the
discussion. The next session is building meaningful measures adaptability and
Floyd and our own Blackford Middleton. Floyd Eisenberg and Blackford Middleton
are going to present. We will start with Floyd and I suspect we are going to
run over. Do we have Kathy McDonald on the line yet? Not yet. We can just let
her know that we are probably running 25 minutes behind but looking forward to
her testimony and if that is a problem for her just let us know.

MR. EISENBERG: Thank you very much. I thank you for the ability to speak
here today and I am Floyd Eisenberg, Senior Vice President of Health
Information Technology at National Quality Forum. I am very pleased to discuss
the topic of building meaningful measures and adoptability. Just in our last
discussion I heard a lot about vendor participation with using measures and
implementing them and also is there a model of information. What I am going to
be addressing in this presentation is pretty much a model of information for
quality. It was developed by the Health Information Technology Expert Panel,
chaired by your own Paul Tang, and one of our work group co-chairs was
Blackford Middleton. I feel like I am home here. It was funded by the Agency
for Healthcare Research and Quality, where iteration as the health IT panel.

Our original goal, the task we were given, was to develop a quality data set
and to identify workflow for quality in a clinical setting. I will be
discussing that with you as well as then how do we get this implemented, which
is the rest of the presentation.

To start with, in thinking about quality we call this a quality data set
thinking of the way – if you remember when Helen Burstin presented this
morning the quality tree, everything starts with guidelines and evidence and
the branch points are where guidelines recommend different components and the
leaves are the measures. It starts with the study designer, the guideline
developers that start with a concept. This is all about diabetes, as your
example. As we look at that we want to understand what do we mean by diabetes
and here the measure developers often provide a list of codes to say what is
diabetes. In the example provided it is ICD-9, could easily SNOMED, and will be
SNOMED at not too distant future. the list of codes that represents diabetes.

In order to provide additional meaning for any part of the measure I need to
know that there is an active condition of diabetes. I have now merged together
a concept, the codes, and the data type, the type of data that is active. It is
a problem or a condition and it is active. All of these together make up a
model of what we call a QDS element or a quality data set element. That element
then can be stored and properly identified, be reused in a database of other
quality data elements so any time we need to understand for a measure in a
numerator or a denominator the use of diabetes active diagnosis that
combination of code list, data type, concept together as a QDS element can be
identified and reused. Understand I am talking here about the quality side. I
have had a lot of interest from those in public health for public health
reporting and public health identification of illness, as well as a research
side either folks working with the same concept concern at National Cancer
Institute and elsewhere. The quality data set in these elements can be reused
for data out of electronic records and electronic data streams for multiple
uses not just quality, but we are going to be addressing quality mostly here.

Once I have done that I now have my quality data element and in the context
of a measure I want to know where is it coming and what is its source. What is
the workflow of it? When we talked about workflow in the HITEP we were talking
about the source of the data, that is, the originator. It could be a device. It
is a reading of a blood pressure off of a blood pressure monitor. The recorder
could also be the device electronically. It could be a clinician and it could
be the source as a patient and the recorder as the patient. We are talking
about those two elements.

We also want to identify in the flow what is the setting in which it
occurred. Is it in a hospital, in a home, ambulatory care setting, skilled
nursing setting? And in what health record field do I expect to find the
information especially if I am submitting data from one EHR to another, and in
this case we want to know that this is an active diagnosis of diabetes on a
problem list. The data flow would identify the location and also the source and
health record field. Once I have that multiple ones of these, multiple elements
make up a measure and those measures then each are stored in a measured
database.

In order to, again, looking at the model I have the quality data set
elements on your right, the measured database on the left, which is composing
those into measures and each of those measures then can be used to manage
measures in EHRs, registries, health information exchanges, et cetera. I give
credit to Danny Rosenthal, who is tremendous with graphics and he did a very
good job on this.

Once we have identified all that and we have our data types. If I were
thinking of something like medication the medication is represented by a set of
codes and it is either medication administered, which would be one data type
medication ordered or prescribed being another, each one of our concepts has
multiple data types. If I look at those as the center bar in this particular
drawing, that is the kind of the Rosetta Stone, the quality data set against
with measure developers and identifying the elements for the denominators and
their numerators for those providing clinical decision support to identify how
to define my population and what intervention should have occurred can reuse
that quality data set rather than an EHR vendor on the other side looking at a
text set of criteria in a measure reading every sentence in every line and
trying to figure out what that means in their EHR can identify I know this is a
QDS element. This is active problem of diabetes or just active problem. I know
where active problem is in my EHR. EHR 1 has it mapped once, and anytime I have
an active problem whether it is diabetes, congestive heart failure, whatever, I
know I expect to find it in that same location in EHR 1.

That might be slightly different in EHR 2 because of some innovation in a
way that is configured but they handle problems in a standard way also. Once
they mapped that to the active problem then they have mapped to the Rosetta
Stone on your right. The measure developers and users and the seekers of
information, map to it from your left. Using that same model without having to
individually look for the same element every time there is an element in a
measure or a guideline or somewhere else, to go find it individually in EHR as
long as you find it in the center on the quality data element and the quality
data type then you are able to do that mapping much more easily.

In order to do that we had to link. The next step to implement is how do I
link each of these data elements to a model of information that EHRs might be
using. Now HITSP has done a lot of work on this effort. The Quality
Interoperability Specification, known affectionately sort of by some of us as
ISO6, it has a table in the appendix, which we will be going to panel, I think
in the next month, that identifies all of the standard elements, the HITEP data
elements, a definition for the HITEP quality data types, and identifies where
in an interoperable way that information would be found in electronic setting
and it identifies the terminology that most effectively could be used to do
that.

HITSP’s terms for that would be C83, which is the location in a
structured document, C80, which is the vocabulary. There will be new numbers
for after comment. I won’t comment on them. I think it is C154 is going to
replace one of those but let’s not confuse the issue.

This was to take the HITEP elements and definitions, make sure there is a
setting in the record that represents each of those. A measure developer
doesn’t have to understand the entire model of electronic records but
understands the data type. The electronic record doesn’t have to
understand everything quality developers are thinking about. They know where
that information is mapped to and they can deal with that. That is the process.
The table is obviously longer than two rows, quite a bit longer, but I have
just showed that as an example.

What we did find was in the HITSP side there were some gaps. Some of it was
in thinking about transferring data and using data for individual patient care
the discussions and standards go so far but when we think about sometimes we
need more granularity where it can be free text and it doesn’t have to be
specifically defined for individual care as long as they know it’s in a
packet that deals with physical examination.

It does matter if which part of the examination should be identified I
SNOMED and which in LOINC or is there a difference because right now what HITSP
will tell it’s one or the other and what the EHR vendors told us is which
one and when. There is increasing granularity that needs to be built into the
standards work and that is where there is some more harmonization and
specification that is required that was one example.

There are also areas where there isn’t a good clear evidence of where
that should be or is in an EHR setting and some of that patient functional
status, a functional status survey exam. Where do I include that? It’s not
necessarily in the same place in any one electronic setting that needs work;
the patient’s care experience, the provider’s care experience both of
which are HITEP data types, communication to and from the patient. Devices.
There are standards for devices but sometimes measures are actually looking for
the use of devices where there are no standards. One of the examples was TED
hose or antithrombotic devices put on the leg. There wasn’t a good
standard for that. That is where there are some gaps.

To understand a patient has declined without asking a physician to enter a
code to say the patient refused this or declined it, how do we get that
information one from the patient and two how do we standardize how that is
represented so we can reuse it or treatment offered, which should happen before
it is declined we would hope.

Another area where there is a gap is we do have and Blackford shared for NQF
a team on structural measures so we have a number of those. Most of them are
basically indicating I have the capability in my office that I can do this
procedure that I can do ePrescribing and sometimes I can identify when each
patient left the office. Did I sent any prescription or did I not and the
reason?

What our next step and this is one of the panels that NQF will be calling
for nominees very shortly is looking for folks who can tell us what kinds of
information can I get out of routine EHR utilization to determine that the EHR
was used for this purpose. Rather than have a doctor have to put in a code to
say I just wrote an ePrescription, which sounds like double work, how do I just
get that stream of how many of those happen and how many prescriptions were
written directly out of the EHR? The reason for that is to identify if we think
of meaningful use, what are you actually doing with your EHR? Are you using it?
That was a gap that we are working toward filling.

In order to then now take the quality data types and allow them to be built
into measures – this is a very early picture of a prototype. This is not
what the tool will eventually look like but to try to give an example of what
would happen is a measure developer will be deciding on a new measure. They
will say I have a new measure, in this case hemoglobin A1c management. They
will look through a QDS dictionary where there already are quality data set
elements and they can pick one, see that it is there, the hemoglobin A1c. It is
a lab test. It is identified by LOINC codes. Here are the codes. If it
doesn’t exist they can then add it here. All labs in this tool should be
in LOINC and then the ability to add them in so that they can add elements that
don’t exist or reuse those that don’t. Once that is selected all of
the quality data elements for this particular measure will be down in this box
to show here is everything related and there will be a place for describing the
logic of what goes in numerator, what gets added, the mathematical operators to
fit those together.

The intent is once that is all defined is to put that into an electronic
format. In the background this measure authorizing tool will create an
electronic measure. There is now going through HL7. It has gotten through
ballot. It has not had a final vote to determine if it is accepted or goes to
second ballot but is an electronic measure. It has a representation of all the
elements within the measure mapped to the reference information model, the
model of information within EHRs and all of the measure elements come from the
data types. It then provides logic and with it provides a human readable style
sheet and an XML version from which EHRs can then extract the components they
need to implement the measure. That is now in progress. We will be testing that
in the next couple of months with some meaningful use measures that are being
retooled.

Let me just mention on that and I think I will talk about that more
tomorrow. There are 72 measures that we were asked to have retooled with
existing measure developers. The original stewards of those measures who
understand the logic, the evidence and what went into it to create them more
into a QDS format and electronic format meaning that hemoglobin A1c greater
than nine becomes hemoglobin A1c show me the value and not check a code that
says it is greater than nine but A1c give me the value.

Now there have been folks who asked us in a measure like that can you tell
me hemoglobin A1c six months ago and the one now and what is the delta. There
is a lot of value in that but that would change that measure. In order to
retool we have to keep the measure intent similar to what it was. To create new
measures we are very happy to look at those and send them through the
endorsement process but in order to retool we have to keep the meaning of the
original measure to keep it the same.

What are some new data types, new sources? Some of these were the gaps I
showed you but functional status, care experience, communication. How do we get
information from the patient? How do we get information transmitted to manage
the different new types of information we need? What we will see as we work on
the national priorities and look at the patient’s care experience, the
patient engagement and care, we will find new data types and new elements that
we need to add to this and address so we are expecting that.

I mentioned the HITEP data flow. The reason I included those here was new
data sources will be coming directly from devices. How can we identify if I see
a blood pressure what the source was? The more we can persist the origin of the
data with the data as it goes through the system it might be a health
information exchange having come from an EHR which came from a PHR which came
from a device in the patient’s home. If I know it is from a PHR I
don’t know if the patient entered it. I don’t know if it was the
device. How can we manage and I don’t know if it is clear yet how that
happens. Here is the new source. How do I keep persisting where it originated
so that I know the value of this data to use it for quality research or other
purposes?

Now this is purposely complicated because it is. One of the comments I was
asked to talk about a little was data collection. In the standards committee we
were asked what are our transactions and we tried to look at well we have to
look at what this flow. I have a measure source so the measure developer
provides this measure. Let’s assume it is electronic. It goes to an
electronic record. It might go to a personal health record, PHR. It might go to
a registry. Perhaps that EHR does everything. It assembles all the data in a
registry within it. It creates the report and sends it out, which is why you
are seeing all these different boxes that connect it. It may send the
information and work directly with feedback with the data assembly system, a
third-party warehouse, a health information exchange, a registry like American
College of Cardiology, and that might be collecting the data. There is a
transaction here that needs to be addressed. Sometimes the processing entity
that reviews the data to validate it is correct is separate from the assistant.
This is drawn to show there are various architectures that exist today. Perhaps
one architecture is best but I don’t know that we can get necessarily
easily to one. I think what we want to do is address the transaction from the
direct care delivery to whoever is assembling the data and feedback has to be
frequent so that the care site knows how to improve care and not just get an
end of year report. But this one is done to look at what are the different
levels of data collection. It can be complex but I think what is really
important is this collection of the data directly from its source.

In order to keep them current one of the issues as we talk about the quality
data set is maintaining currency of the very atomic particles for those
measures. That is the codes. If I am looking at RxNorm, for instance, as a
source for the codes for my medications, that changes frequently. Every time it
changes that code set or value set as some IT people would call that, is going
to change. Does that change the measure and at what point does it? We have a
team scheduled to meet in a couple of weeks to help us set criteria for when
does that change or make it a new measure. When does it make a new code list
and how do we version those and how do we manage them?

How do we maintain the QDS is our next job is to make sure we add to it and
modify it as we learn more through creation of measures. Measure maintenance
process, I think you heard some of from Helen Burstin this morning, and that
what we do have is a regular endorsement process. Every measure has to be
reevaluated every three years and every year, if there is a change is
reevaluated. All measures now being tooled electronically will go through a
modified process because they are not changing the measure. So the process we
will evaluate one did I change the measure and if it did change then that means
it is a new measure. Did it change the meaning of the measure that it was
retooled and two, was it retooled appropriately so address the QDS correctly?
That is an alternate process for retooled measures.

I am trying to keep to time but that is a broad overview of a new model for
managing measurement, which I think will work well for not only quality but
also public health use of data coming in and out of the EHR and also research.

On the code list issue I think what is important is there is currently in
public health something called PHIN VADS, Public Health Information Network
Vocabulary Access Directory, and they create value sets or code lists for all
reporting for public health. NCI has something similar. I don’t know the
meaning of the acronym as clearly, but I think it is CDAR and they do something
similar for research.

What we don’t want to do is a create another silo of creation of code
lists but our intent is to talk to the National Library of Medicine, CDC, and
NCI and get together with stakeholder groups to figure out how to do this best
nationwide and very shortly. Thank you.

DR. CARR: That is great. Thank you. It is so helpful the way you organize
the data.

MR. QUINN: Mike Fitzmaurice isn’t here but there is a thing called
USHIK, which I don’t know what it stands for but it is a big meta-data
repository.

DR. EISENBERG: US Health Information Knowledgebase. Let me just say I did
not mean to leave AHRQ and USHIK out of the picture. USHIK actually is doing a
lot of work with HITSP to store all HITSP documents and to create a kind of a
data repository for all of this and they have offered to do some of this for
the quality enterprise. What I think is important is that we all sit together
and figure out what is the way to do this. I don’t know if it would be
productive to have all the quality data sets or the quality information sitting
in USHIK and the public health sitting in PHIN VADS and the research sitting at
CDAR(?). I think it would be more helpful if there were a way that even if that
did happen how they are all coordinated.

MR. QUINN: To put them all in USHIK.

DR. EISENBERG: If that is the answer that is fine.

DR. CARR: Blackford, go ahead. Do we have Cathy McDonald on the phone now?

MS. MCDONALD: Yes, I am here. Thank you.

DR. CARR: It is Justine Carr. We ran a little bit over because of various
schedules. Are you okay to stay on the line?

MS. MCDONALD: Yes, I am.

DR. CARR: Great. Thank you.

DR. MIDDLETON: Good afternoon everybody. Blackford Middleton. I am from the
Brigham and Women’s Hospital, Partners Healthcare, a member of the quality
subcommittee and a member of the full committee. I was invited this morning to
share a few thoughts with you about these quality measures adoptability. I was
jotting on the spot and put together a few slides.

I guess when I think about this and I am very happy actually to have the
opportunity to share with you some thoughts really from the implementer’s
point of view. I really tried to take a cut at this as an EMR developer and
implementer having worn that hat more than a few times. It struck that may be
there are four core components to think about when assessing these measures and
their implementability or adoptability in HIT. I just thought I would consider
HIT broadly. It might be that these are implemented in a variety of HIT
systems, EMR and hospital-based records and other hospital systems as well as
ambulatory EMR, et cetera.

Anyway first and foremost the quality of the measure, secondarily
implementability or the adoptability, thirdly practicality of use in clinical
practice and then lastly the maintainability or as I think Justine said this
morning perhaps intentionally adaptability in addition to adoptability. Let me
talk about each of these very briefly.

I think many of these issues have already been discussed. This is a big of
an eye chart and I apologize, but clearly we have to start with a high-quality
measure, well specified, clear conceptual understanding, interpretability as we
heard before, measurable and discriminating among the candidate population
being assessed, precise enough but not too much perhaps, statistically sound
and valid, independent repeat measures and coherent and composite measure and
the like, free of any bias structurally or random bias error that might occur,
and what is the underlying distribution of the population? Is the measure
appropriate for the underlying distribution? Is the requisite instrumentation
from the HIT well understood? How is this data gathered? Is it gathered in the
process of care or as a byproduct of care or in some other automatic way
through an interface to an ancillary system or what have you? Are these
different instrumentation requirements considered by device or by interface
appropriately? Is the customer of the measure well understood? Is there use and
intent with the measure well described whether it is payor, medical management,
quality assessment, et cetera, and is it maintainable? Does it recognize
changing, evolving code standards in nomenclature work flow processes, roles
and responsibilities in the clinical care environment, and accountabilities,
again, as has been mentioned already and of course has it validated?

There are fewer words per slide going forward. To address the
implementability part though really getting at the task at hand in HIT, I think
as we have seen in the QDS work and the NQF work here that already as Floyd
described can we assess the numerator and denominator using standard data
elements? Is that even possible? Is it realistic? By the standard data elements
therefore then if they are to be used in the EMR typically and even if they are
in the EMR, are they well populated? There are several hurdles to scale,
mountains to climb. Does the measure rely on a particular HIT feature or
function for its acquisition? If it does is that being used? Does this require
use of a standard template or a form or a documentation widget? Or if a
physician dictating, is the data element achieved in another way? Are these
functional requirements considered actually at the time of the measure
specification? It could be clear to the HIT implementer how the designer
expected the data to be gathered.

Practicality of use in clinical practice with HIT. Are the standard data
elements well populated as I mentioned? Are they being captured automatically
or as a byproduct of care? Do the methods of data capture measure or are they
biased in any way as the measure is being assessed? And of course two critical
forms of error are biased both systematic, nonrandom, but systematic and then
random as well and they will of course bias the results. Are they captured as a
byproduct of care or is it outside the routine clinical workflow? If it is
outside the routine clinical workflow, who is going to get it into the system
so we actually have it as a source of measurement? Does the workflow in which
the measure is captured bias the measure in anyway? And were these workflow
considerations considered when the measure was being specified?

Second and last on the practicality of use of these measures in HIT. Does
the data source itself bias the measure in any way? One lab versus another may
have systematic, nonrandom, or even random error and they could be different.
When these data are pooled they may therefore bias measures. Are there
different coding schemes with partial concept consonants? I think that is a new
phrase, Paul. I have not heard that before. If the terms defining the elements
being measured are stored in the record aren’t exactly the same thing even
if they have a standard code and label applied to them. If they are not exactly
the same thing we will have this concept dissidents rather than consonants.

In the manual data sources does the quality of the data used in the measure
vary by which person and which role is gathering the relevant data? A classic
example here I think is the nurses are trained to gather the BP only measuring
at the 5’s. We are trained as physicians depending upon the level of
intensity of your professor of medicine to either measure on the even numbers
or even on the even and odd numbers. The nurse might take the BP and get 95
over 65 whereas I might take it myself and get 98 over 62 whatever.

Can a measure report be implemented? It is one thing to have the data even
if we have gotten this far in this system. Can the measure report be
implemented in a useful way for each use of the measure or each user of the
measure? Can the same measure scale for multiple uses? Can the measures be used
for different purposes, et cetera, at the point of care for the medical
director, for the payor, for public health reporting? I think this is one of
the visions in the NQF CDS work or QDS work rather that there will be a
standard toolkit, a set of measures which could be used and reused in multiple
different ways and have the same validity and integrity hopefully.

The last point thought is about this maintainability or adaptability and
this is something I think those who implement EMRs and try to derive data from
them wrestle with on a daily basis. Even if you have gotten this far and there
is some kind of data in the record and you are trying to get it out of the
record to make reports, does the measure support quality reporting at the point
of care? Is it biased in all the ways we have talked about or is it biased
actually in use in some interesting and usability type ways?

The key problem we face routinely is simple things like provider’s
assignment is the report about my patients only or is it about any number of
patients, which I might own or not own in my provider panel specification, et
cetera.

But equally important I think and from a sustainability point of view, can
the measure be updated easily and practically. Is there a way to change
numerator and denominator specifications easily and practically? Can we change
coding standards and evolve and maintain the history of evolution of definition
or coding standards being used and can we evolve the different nomenclatures or
vocabularies that might be used, SNOMED or other? And even within a terminology
can we actually handle the way they do the update process? In SNOMED this is
actually pretty difficult because SNOMED will do things like move an entire
branch of a tree from one place on the hierarchy to another place on the
hierarchy and any time you have any definition of anything using that branch
and if you use any of the hierarchical relationships, you have to go and look
back at all of your data and all of your specifications. For example, if you
define diabetes as a subset of a variety of trees in SNOMED and they move one
tree or one sub-tree from one place to another, that whole subset
classification, which could be a critical element of the measure, has to be
updated and that can be very tricky.

I guess lastly I just want to point to the issues therefore more broadly of
semantic and syntactic integrity. How do we maintain these concept definitions
and how do we evolve and migrate concept definitions as our needs change based
upon measure requirements? I think Floyd raised the issue or rather Larry asked
the question of is there a common information model for measures. Certainly the
HL7 QRDA reporting data architecture is kind of working towards that end and I
don’t know a great deal about this. It’s not yet been relevant to our
work but I think they are attempting to kind of define a meta-model if you will
or a common information model for quality reporting. I think that might
actually make sense in the long run but it may simplify things in ways that we
really need to have them simplified.

Lastly, is there messaging and reporting where standards methods for the
extract transform and load procedures that have to be done in the HIT with the
measure specification to get the data out and report on it? What is the
architecture for knowledge management and curation? One of my favorite topics
at Partners Healthcare. We have built a group there to worry about this kind of
problem and as definitions and terminology and specifications change on a
regular basis it’s not just define it once. It is define it and update it
and update it continuously. Thank you.

DR. CARR: Excellent. You didn’t need that time to prepare. It was
excellent. Let me open this up to questions. Maybe we could get the lights back
up as well. Could I start off with one question and maybe it is a naïve
question and maybe I missed something. As we are looking toward our future
state, certainly having the elements in the EHR will take a long way, but as I
think about registries and taking that data and turning it into specific
measures and/or even to do drill down, it seems that there is a need for an
intermediary data aggregator who can take and pull data from perhaps different
sources or can do manipulations and apply risk adjustment. Is that a
fundamental piece of the big picture?

MR. EISENBERG: Yes, I think a data aggregator definitely is a fundamental
piece of the picture. In enterprises, say hospital enterprises, it may be that
the hospital system acts as the aggregator as opposed to necessarily using an
outside registry. In other cases it would be a health information exchange or
registry. But I agree. There does have to be an aggregator capability.

DR. MIDDLETON: And I would agree too, Justine. From an architectural point
of view it is clear that the sources of data on which we wish to report will
come from a variety of different places even within one institution and even
potentially within one clinic. Depending upon the instrumentation of that
clinic, laboratory data sources, other types of physiologic assessment, BP
assessment tools, whatever, the data will be aggregated once for clinical
purposes in the EMR in some form or another but often times for reporting it
needs to be combined with administrative claims data or other sources of data
again. And further not only is it combined but also it really is cleaned up. I
haven’t seen a record yet and I have seen a few in details that really
have data in the clinical record in a way that is ready for this quality kind
of reporting.

DR. CARR: It seems that as we think about that for all the things that our
earlier panel talked about in terms of risk adjustment, are there 35 risk
adjustors or 6 and are we all using the same ones because even today I think
that is an issue across various roads. Do we have resources or committees or
panels focused on this?

DR. TANG: Focused on what?

DR. CARR: On the data aggregation function and risk adjustment. We had EHR
and meaningful measures.

DR. TANG: Somebody I’m sure know more about this than me but there is
PQRI registries, for example, and I believe some vendors will actually extract
the information needed and will act as the PQRI registry and extract the
information from your EHR for submission. That would be almost the best –
I mean that is an intermediary, but the nice part about that is it is coming
from your EHR. Now we need to have an API so that you can have some kind of
aggregator act for multiple EHRs but that is at least a model for how things
should be.

DR. CARR: In the administrative data world UHC is that kind of thing where
there is this standardized risk adjustment model and a cleaning up of the data,
but even hearing today about the variability and risk adjustment that seems
like a whole body of work that will be critical to meaningful measurement.

DR. TANG: But even without risk adjustment though –- I mean the whole
clean up that –- I guess my question back to the panelists are how
feasible is it to get to the F7 and the other is Blackford you had a long list
of considerations for how to consider which – the make up of the quality
measures. Who should be responsible for all that to consider all that?

DR. MIDDLETON: Well the NQF obviously. Right, Floyd? First of all there is a
question about the method if you will on a national scale. What we are figuring
out on a local scale is somewhat arcane and laborious procedures that you have
to go through to get the data from all the different systems in this
aggregation place. We call it quality data warehouse. It is parallel to and
corollary to the clinical data repository and even another one called the
research data repository. But whatever the architecture I think this is an
excellent line of thinking. What do we actually need to have by a way of an
architecture nationally for quality data reporting and management. Starting
locally with QDS and specification, implementation, and use, but then that is
all the afferent limb on the efferent limb. What do we do about data
aggregation, extract, transform, and load and et cetera? The problem I think
will be ameliorated to a degree when we have much more standardization of the
cardinal data types in the record. When the problem list has to be SNOMED if it
is going to be SNOMED then theoretically over time the problem list will clean
up itself in use; however, when the problem list is in evolution from ICD-9 to
ICD-10 or maybe partly to SNOMED and all of the rest of it, we are going to
live in this room of cleaning up the data all the time. I think it is going to
be extraordinarily valuable service if you will of the second question is who
pays for that if it is coming from small office environment, et cetera. How
does that get managed all the way to CMS and all the way back down?

DR. TANG: It almost seems like we have to in Floyd’s term the workflow.
We almost have to prescribe from the measure developer point of view or
upstream the clinical trials folks deciding who enters the blood pressure
therefore manifest in the EHR has to be a place for that kind of person to
enter the blood pressure so that information can be stored and passed on to
fill that measure definition.

MR. EISENBERG: Well I think when I was talking about workflow is what
information do you want. If I wanted a patient engagement I really might want
not blood pressure from – I’m not trying to create a measure here but
just hypothetically. If I want to make sure the patient is doing their own
blood pressure at home then I want it to come from the patient and I need to
find a source for that. If there is not a source obviously that is a problem. I
don’t know that it is prescribing what the EHR looks like as much as I
know the data I want and how the EHR captures clinician entered or patient
entered data if you will is in some ways up to EHR as long as what we are
looking for is identified as long as we are clear what we need because then
both can kind of build toward what would be the Rosetta Stone in the middle.

DR. GREEN: I want to thank Floyd for me was a pretty darn elegant model and
I appreciate it very much. As is often the case someone presents an elegant
model and you realize how sloppy yours is. I still practice medicine and as I
was looking at that model I kept trying to map it to a mental model why care is
obtained and rendered. Then your last comment just then about if you want a
blood pressure. The entire asynchrony of healthcare that is our future seems to
me has to be harmonized with this process and it sounds to me that NQF is
further along in its modeling than physicians and nurses and other folks are
with their models. I am ventilating now.

I really have a question for Blackford. Could you just say more about two
things on those slides? One is you asked the question about are repeated
measures independent and can you just talk a little more about what the nature
of that independence is to you and the other one is about your scaling.
I’m not sure I understood your point. I thought you were talking about
scaling them for multiple uses across settings and sectors and that sort of
thing, independence in scaling.

DR. MIDDLETON: These are relatively off the cuff remarks because I literally
did prepare them this morning, but here is my concern. In many ways the
statisticians will worry about kind of whether or not measures either repeat,
one time or composite are of course actually reflecting the underlining concept
being assessed. The issue with repeat measures is if they are not independent,
if they are dependent, there may be a false assurance that you are actually
measuring something – the measurement may be better than the state
actually is because the independent measures will confound one another.

Similarly, what did I say the first time? The other one. You can be over
confident or under confident based on the independence or the lack of
independence of measures. This is the same kind of thing that happens in
decision making actually. When you ask physicians of course about their a
priori assessments of the probability of disease and relationship to findings.
If you find the coexistence of elevated ALT, elevated AST, and maybe a total
bili elevated, it doesn’t give you much additional information then simply
seeing elevated total bili for hepatitis or some obstructive disease in the
liver. Similarly, I worry that if we have multiple measures of quality that are
highly dependent, we may not actually be measuring more than one central
concept. If you have independent measures and you can assess them as being
independent, you are actually get much more information.

The second thing about scaling sort of the simple thought there I think is
does the measure have the same applicability when measuring 20 or 100 or
100,000 or a million, does the same measure apply to populations in some kind
of way? I’m not sure I am going to be able to express well as well as can
it be reused in multiple context, multiple purposes. There is a scaling issue
about measuring more than just my clinic how I measure entire populations.

DR. CARR: Just one comment on that. Just to get back to one of the comments
from the earlier session again the importance of being able to drill down on
data because having this data source as well as other descriptors of the
population who is being measured becomes critically relevant. In other words,
if you have a bimodal cut population and your composite is average then you
have missed the opportunity to improve the low performers. Again, it goes back
to this intermediary aggregator. How do you configure the data and make it
available in such a way as to take it to the next level?

DR. MIDDLETON: That was the notion of understanding the underlying
population distribution for whatever the measures is trying to assess. If it is
not normal you may miss completely different subpopulations in a bimodal thing
or other types occurs and simply miss it.

DR. FITZMAURICE: I guess the second part of my question has been answered
about how do you get the distribution reported. The first part is the data
aggregator, is it the aggregator a part of the enterprise that is doing the
reporting that is part of the single hospital, the single physician group, or
the single chain of hospitals so that they have control or maybe they get to
see it before it is reported as opposed to some comes in from here, some comes
in from there and then it goes up and the person who is responsible for it at
the enterprise offices. I don’t know where these numbers came from. It
seems to me that it has to be aggregated at the enterprise level first then
when reported up, you have to report at least the denominator so that people
can multiply the numerator by the n and aggregate across, get the total
numerators then total denominators. If you just get fractions you are getting
averages of fractions rather than and aggregated the whole population.

And the second part I think Justine brought up and you answered is that how
do we get the distribution? You find out a lot of things from distributions
then just from one point.

DR. MIDDLETON: This is helpful actually to discuss these thoughts, which are
completely off the cuff. In a way, Paul, maybe a heuristic Justine that we can
take away is at a level we want to argue for a parsimony of measures. We want
to have the minimal set of required measures to assess quality because more
than that is just burdensome in all the different ways we have been discussing.
Is there one diabetes measure that will really cut the mustard?

DR. TANG: One of the challenges is that we have multiple siloed measure
developers and diabetes of course is cross cutting among specialties. And even
though there is NQF endorsement criteria that ask them to look at the
contribution of the next measure, we have a challenge when it comes to -–
who is making the decision? Is this one better but actually it would be better
yet if the such and such – and then you actually have to retire the other
measure otherwise the public reporting folks are going to ask you to do that
and just adds to the measure burden. The cycle time for that is every three
years. It is timing, et cetera. As you were talking I was thinking about how to
have some kind of coordinating body over the measure developers and that is
where it actually goes back to QDS. If the QDS and its authoring tools somehow
at least in form, the different silo’s creating measures perhaps there
would be less new marginally contributing measures developed.

DR. CARR: We have time at the end to come back to some of this discussion.

MR. EISENBERG: I was just going to agree with Paul, that I think the QDS
will help that because one is it will keep the elements and the sets of codes
you are looking for somewhat consistent because without that reuse of the
measure with different levels and versions of coding is going to change
reporting and is going to affect that. I also think that there are sometimes
and the additional data elements are often used for risk adjustment or for
adjusting performance. Those additional elements can be added to the measure
and reported even though they are not calculated in it. As long as that
availability is there, I think that may keep measures fairly consistent so that
you could do some analysis based on some variation with additional elements.
Does that make sense?

DR. TANG: Maybe NQF can offer that service as proposals come in using the
QDS and instead of measures to try to view its own sensitivity analysis in a
sense. Is NQF funded to convene the supply chain to start talking about the use
of and population of QDS?

MR. EISENBERG: We are. We are funded to continue and that is advice we are
looking for you as our HITEP chair, where we move forward for this.

DR. CARR: Did you have a comment? With that then, I would like to
reintroduce Sarah Scholle and then Kathy McDonald is on the phone. Kathy, why
don’t we start with you?

Agenda Item: Meaningful Measures for Care
Coordination

MS. MCDONALD: Would you like me to just go ahead and start with my remarks?

DR. CARR: Yes.

MS. MCDONALD: Do you have my slides there?

DR. CARR: We have handouts. Let me make sure everybody has them in hand. I
think they are in our blue books here. They are in these blue folders.
Meaningful measures for care coordination. Kathy, can you just say a little bit
about your background and then start right in. We have our slides.

MS. MCDONALD: Are they showing too, because there are a few spots there in
color? If they are not in color I can make sure to point out what I am talking
about.

DR. CARR: I think we are set. Yes, we have them on the screen. Just say when
you want us to change and we will take care of that.

MS. MCDONALD: Okay. Thank you. Can everybody hear me okay? Okay great. I
work at Stanford University. I am a Senior Scholar here and Executive Director
of two research centers that operate under the umbrella of Stanford Health
Policy. I have been involved in quality measurement development for about 12
years, and will sort of do some further introductions in my remarks here.

I appreciate the invitation to speak on this topic of meaningful measures of
care coordination. On the second slide I can introduce some of the work that I
will be drawing from as I make my remarks. If that slide were up that would be
great.

I have been the Associate Director and an Investigator at Stanford and the
UCSF Evidence-based Practice Center for the last decade and during this time we
have produced numerous evidence reports on healthcare quality and patient
safety. The three that are the most relevant building blocks for today’s
discussion are listed on this slide.

The most recent evidence report focused on care coordination as part of a
series called Closing the Quality Gap that reviewed quality improvement
strategies from the 20 topics identified by the Institute of Medicines National
Priorities Report as most promising for improvability. A care coordination was
listed as one of the cross cutting areas and our report included background on
the topic of care coordination especially stakeholders’ suggestions about
areas that required more research including measure development, a working
definition of care coordination, conceptual frameworks, and systematic reviews
pertaining to care coordination quality improvements.

This work will serve as a foundation for a new project recently started and
supported by the AHRQ Quality Indicator’s Program on care coordination
measure development for ambulatory care.

The other two reports listed in my background as building blocks for this
conversation constitute the initial evidence reviews for the first three AHRQ
quality indicator modules. The inpatient quality indicators, the prevention
quality indicators, which were both covered in the 2001 report and the
patient’s safety indicators, which were presented in the 2002 report the
evidence on those.

Next I would like to give you a little history of the AHRQ quality indicator
program since this is the home for the new care coordination measures project
that I am leading. AHRQ’s QI program and the evidence-based practice
center program each represents wonderful examples of producing research that is
tied to the actual needs of the fields. The EPC program routinely and publicly
asks for nominations of topics where the expectation is the stakeholder will
use the product of an EPC’s work, that is the evidence base synthesized
well and comparatively were applicable.

The AHRQ’s QI work started within the EPC world after AHRQ and HCUP
partners requested an evidence project to refine the original HCUP quality
indicators. The AHRQ quality indicator development has always been grounded in
evidence-based medicine methods applied to measurements.

In addition the measurement work has always been tightly coupled to
user’s needs since AHRQ already had its HCUP partners who wanted tools to
work with their data. The motivation for the original work was to satisfy the
needs of those who were collecting the data and who were working in their
states to supply hospitals, legislators, policy makers, and of course the
public at large, patients and their families with something meaningful based on
the routinely collected administrative data sets available at the time and
still in use.

As the program evolved AHRQ initiated the support contract to assure ongoing
refinements of the indicators, and this represents the guiding philosophy of
the program that of continuous quality improvement based on user experience and
changes in the medical evidence. In addition, the program includes expansions
within domains and data sets initially covered as well as expansions to new
domains without ties to any particular data set in reflecting new priorities in
the healthcare field, such as, the new project on care coordination measures.

All along AHRQ and the QI team have continued to innovate to expand
measurement methods always evaluating measures from initial assessments to
implementation of the measures and then feedback and support throughout the
life of the measures in the field.

I would like to suggest two analogies based on these observations of the
history of AHRQ quality indicator program. First there is an analogy between
continuous quality improvements for industries that delivery goods or services
and a similar need for measurement to involve continuous improvement as well.

Second, and more apropos for the current session there is an apt analogy
between coordinating care to deliver seamless and effect of delivery of
healthcare to patients, and coordinating measurement efforts to delivery a
non-burdensome and effective picture of the system’s ability to deliver
high quality coordinated care. In those cases it is crucial to minimize
important gaps, gaps in care or gaps in measurement. We should be most
interested in gaps in care that has the potential to contribute to bad outcomes
for patients, and similarly in the analogous case we need to pay special
attention to gaps in measurement that could result in missed opportunities to
improve the quality of care. It is often more difficult to pay attention to
what is missing as opposed to evaluating what is present.

I’m going to shift gears a bit and give my perspective on the four
questions posed as the goal for this hearing with the focus on care
coordination measures. The first question that was part of the agenda was how
do we approach building meaningful measures. For care coordination I think it
is important to think carefully about this domain and how measures might best
promote the best care. For care coordination I think there are four main
concerns. First in care coordination we have to have a working definition of
that is. In our previous work in evidence-based practice center we found over
40 unique definitions of care coordination, which we used to develop. The
working definition I would be glad to share that if asked.

Second, in considering care coordination measure development we need to draw
from one or more conceptual frameworks for being able to deconstruct activities
that lead to well coordinated care so that we can determine the desired
outcomes that might be related to that and establish a causal chain about where
in that process measurement would be most valuable. I know that in an earlier
session it looked like there was discussion of structure process and outcomes,
the classic Donabedian model. That would be one framework. With care
coordination we might also want to think more creatively given the conceptual
challenges of this particular domain, and I will share some of our thinking on
this in a moment.

Third, in terms of developing meaningful measures we require some research
evidence that shows that any measurement proposed actually maps to components
of any of the frameworks that allows us to say that there is some base validity
to the measure.

Finally, in building meaningful care coordination measures we need to make
sure that a measure set adequately covers the areas most likely to drive
quality improvement efforts, have transparency, and ultimately gain the
outcomes that we believe are sensitive to coordination or failures in
coordination.

It was suggested that we needed to have a conceptual framework or frameworks
of care coordination as the basis for measurements, development and evaluation
in this domain as in any domain. I offer two examples. Go to the next slide
with conceptualization option number one. These aren’t meant to be
definitive or exhaustive. They are simply to exemplify the importance of using
some logic regarding the connection between what might be measured and how that
measure could monitor the situation.

In this slide we see a diagram adapted for management sciences and
organizational design. The key point of this framework is the care coordination
of the product of a good fit between information processing requirements of a
particular care situation and information processing capacity of the system or
in many cases the non-system delivering care. We can see that an area prone to
failure’s coordination such as hand off to transition of care as we think
about that we can keep in mind this organizational design framework and are
reminded that we might want to monitor to the fits. That is what is in the
middle circle between information processing capacity and information
processing requirements. On the left side of the diagram processing
requirements depend on the situation and empirical work has shown that varying
levels of interdependence, uncertainty, and complexity can be addressed best by
varying the mechanisms on the right side and used to provide information
processing capacity.

Eric Coleman from Colorado has led the development of the Care Transitions
Measure or the CTM. There is a 15-item version and 3-item version and also
interventions to improve transition from hospitals to home. One of his studies
about an intervention to improve these transitions offers a good example of
taking stock of the care transition situation by getting input from patients
and caregivers about the areas of interdependence, uncertainty, and complexity
similar to that shown on the left side of the diagram. He refers to them as the
four pillars: medication management, personal health information, follow up
visits, and specification of red flags. This in turn allows for establishing
and testing the mechanisms to address the coordinating needs of the situation.
In the case of some of his work a transition coach is a key feature of one of
the interventions and Dr. Coleman has evaluated and could be thought of as an
example of structural linking between settings of care.

Another key feature is a personal health record, which is part of the
operational processes box on the right hand side as a supporting tool to
provide coordination and it is used along with specific casts for patients and
coach aimed at specific goals.

For measures we could target any of these steps in assessing the fit between
the needs of situation and the mechanisms to address those needs, or with
measures we could focus on understanding patient’s preparation for the
discharge transition as is the case for the care transition’s measure,
which is based on direct questions to the patient.

Another approach to conceptualization is shown in the next few slides that
again try to reduce care coordination as the components that might be
measurable. We built is the next slide up conceptualization option number two.
We built upon NQF’s work and mathematica’s evaluation of care
coordination demonstration projects and lists essential care tasks in bold and
associated coordination activities in italics. Measure development could be
organized around ways to observe or quantify the activities in their efficacy.

Then on the next slide we note common features of interventions to support
coordination activities in bold and list examples in italics. The extent that
there is evidence of the presence of the supporting feature improves care
coordination it would make sense to consider measuring the presence of the
feature. For example, in a 2008 article in Health Affairs by Diane Rittenhouse
and colleagues they used data from the Second National Study of Physician
Organizations and the Management of Chronic Illnesses or the NSPO2 to quantify
the extent of adoption of infrastructure components of the patient-centered
medical home. One of these measures is the care coordination integration
component consisting of several questions and resulting in an index from zero
to five, which asks questions about use of electronic medical records, exchange
of information across settings and presence of registries and nurse managers
for specific chronic diseases. All of these concepts and examples are shown in
our second framework.

These frameworks can be used by measure developers to consider the
relationship of potential measures to the ways the measures might be meaningful
to improving care coordination.

Now turning to the second question posed for this hearing, what is the
current process for developing measures? Does it adequately address measure
development for key national priorities and subpopulations? I would like to
give a quick thumbnail sketch of the AHRQ QI process and its application to
care coordination and measure developments. The remaining three slides provide
an overview of the process and evaluation criteria.

The first slide labeled indicator set development shows a standard AHRQ QI
development process, which starts at producing a list of candidate indicators
based on a variety of sources and leads to an evaluation of each individual
indicator, the development of a set of indicators after a selection process,
and then evaluation of the indicators and practice, which leads in turn to
further evidence and ultimately feedback to improved measures.

For care coordination given the dynamic nature of this area with lots of
work underway we have added the box in red call in development to highlight the
importance of looking for measures and development.

The next slide shows the steps, sort of a stair step, to evaluating each
promising indicator in our candidate list. We start at the bottom and work our
way up the steps. If an indicator doesn’t capture an aspect of quality
that is important and subject controlled by the healthcare community, it
isn’t going to be meaningful. For care coordination we have added the red
text emphasizing the patient in the first step. This is an area where failures
in coordination often only experienced by the patient. This is particularly
true in transitions across settings for example. Of course there are also times
where the patient does not know immediately or perhaps ever that a failure of
coordination has occurred as in a missed diagnosis based on test result that
the provider missed getting or seeing or reacting to.

The rest of the stair steps are no doubt were understood by this group and
already covered to some extent in remarks from earlier speakers. I can go
through these in more detail if desired later. The main point is that we apply
these evaluation criteria of all AHRQ QI development efforts but can emphasize
new aspects of the new domain called for such enhancements.

Similarly, in the next slide labeled evaluation stages for indicators we
undertake three main activities of background research, external input and
supplementary research. Again, items in red suggest enhancements that may be
necessary for the care coordination domain. Two of these coding consultation
and empirical analyses lead me to the next question posed for this hearing.
That question is how do we introduce new data sources, clinical data from EHR,
user generated data, et cetera into the measure development process.

It is important to start with that anchoring to any one data set, keeping
the options for candidate indicators wide open and looking for data sources
that support those indicators or concepts. Expertise from those who have worked
with identified data sources has been critical as empirical analyses to test
alternative definitions, assess rates, variation, and relationship among
potential indicators and also to test risk adjustment methods.

A given data source sometimes starts a measure development process as is the
case of the HCUP data and the initial AHRQ QIs, but in the case of care
coordination I believe it is vital to draw from a variety of data sources to
reflect a patient, clinical, and organizational knowledge and experience.

The last question for the hearing is how do we maintain and update measures
and what are the health IT system implications. Here I would suggest that we
draw from the experience AHRQ QI program, which follows a continuum for measure
development to translation of measures into practice and then support for users
and feedback to the developers about opportunities for improvement.

In conclusion I anticipate that we might expect care coordination
measurements to follow at similar paths to what our team experienced with the
development of the patient safety indicators, one of the AHRQ QI modules. Eight
years ago healthcare’s heightened attention on patient safety resulted in
numerous research efforts to understand practices for improving patient safety
and ways to measure progress. Our group worked on a patient safety practice
evidence report called making healthcare safer and at the same time develop
patient safety indicators, which started from a modest evidence base that built
over time with validation efforts by our team and others ultimately achieving
NQF endorsements.

Similarly, AHRQ QI care coordination measures project will draw on our
experience understanding the state of evidence and care coordination based on
our research for the EPC report on care coordination. Given the increasing
level of activity in this area we plan to form a stakeholder and
informant’s work group to assist in identifying all relevant current
activities. In addition to our usual practice of inventorying and assessing
measures from prior published measure development studies.

In addition we recognize the importance of the research community in this
developing area. Specific interventions to improve coordination are still under
development and testing either as part of research studies or demonstration
projects. As a result evaluators and care coordination improvement strategies
need guidance about what is measure in their evaluations and how to prioritize
potential measures. To address this need the AHRQ QI team’s care
coordination measures project will form a second work group of evaluators,
evaluatees, and research experts to assist and develop an evaluation tool to
guide choice of measures for research purposes. This part of the project
responds directly to one of the conclusions of our care coordination evidence
report and that is studies of care coordination quality improvement strategies
have immense heterogeneity and measures making comparative assessments of what
works best to improve care coordination challenging. It’s not impossible.

Any QI development process must adapt to the needs of a particular domain
understudy. We have a toolbox of AHRQ QI development methods from which we
choose approaches best suited to a new area. As needed we also develop new
research methods or tools to tackle unique aspects of a new domain such as care
coordination. In contrast the dynamic and adaptable nature of our growing
toolbox, the AHRQ QI development approach has kept the same standards and
criteria for achieving meaningful measurements and I think that this experience
is highly useful for the goals of this hearing.

Thank you for the opportunity. I am honored to convey my experience and
thoughts on this important topic and I would be glad to respond to any
questions at any point.

DR. CARR: Thank you very much. I think what we will do is go straight onto
Sarah Scholle and then come back and take questions for both speakers.

MS. SCHOLLE: I would like to talk with you about some of the work that we
have underway with colleagues at Johns Hopkins and Park Nicollet to look at
care coordination and I just wanted to start off with key points that I will
try to make in this discussion. First of all to suggest that care coordination
can be measured by thinking about the structures, the processes, and outcomes
and that processes are most actionable but that is where we have the fewest
measures. What is challenging about the measures is that when you are measuring
this process, you really need to have it embed in care. Your measurement needs
to be embedded in the care delivery and it needs to help with decision support
as well as be something that leads you to measures that can help you monitor
care and improve care. In particularly when we think about measuring care
coordination in an electronic environment we need HIT systems that can track
essential data elements and can support the care coordination process, but we
also need the workflows that use the systems. I will try to expand on that.

What is care coordination? It is the information sharing that happens across
providers, patients, different services, sites, and across time. What are we
trying to do when we coordinate care? We are trying to make sure those
patients’ needs and preferences are respected and that care is both
efficient and leads to good quality outcomes. We know that care coordination is
most important for people that have the more complex situations because they
are seeing multiple doctors, because they have more complex needs but it is
likely to change over time.

Care coordination. It can be structure, process or outcome measures. I think
structural measures are an important starting place. They lay out what
capabilities need to be in place, information systems, staffing. They
articulate expectations. Process measures try to get at what is happening. How
is the information being exchanged? Is it being exchanged? Outcomes of care
coordination are probably the things that are most relevant to families and
policy makers. These are measures like readmission rates and failures of care
coordination are probably easier or more relevant to families and policy
makers, but the problem is that you need risk adjustment. Some of these things
like risk or readmissions are rare and they may be difficult to attribute to
particular actions and players.

We have an ongoing grant right now where we are trying to think about this
whole measurement framework in the context of vulnerable children, children who
are at risk for developmental delay. This diagram tries to lay out. It is a
starting point for thinking about this structure, process, and outcomes. What
do we mean by structure? Well it is for care coordination. In a primary care
practice it could be a process for tracking referrals and it might be different
in different levels of the healthcare system. At a state it might be service
capacity. Within a community it might be whether you have someone to act as a
navigator to help families understand what are the service opportunities.

The process measures. Here are some examples of process measures and I will
talk about it some more but it is really the information gets sent from the
primary care provider to the specialist as the information may get back. Does
somebody act on it? Is there a care plan that identifies who is responsible for
what?

Then outcome measures could be clinical outcomes. Poor control of diabetes
could be a failure of adequate care coordination. Patient and family
perceptions are an important factor but I’m not sure. The challenge right
now is that we don’t really have very good measures of care coordination.
I think we are starting to have better measures on the structural side. I think
we have some sense of what are good or bad outcomes like readmission rates but
they are hard to measure. We have measures of care coordination from patient
surveys, but that is probably one of the weakest points of a CAHPS survey today
and part of it is that I’m not sure that people really know what it is
like to be in a system of care that is really well coordinated. It is hard to
measure that.

In our research work we are really going to try to hone in on this from the
perspective of looking at children with chronic conditions.

I said that we have been doing a lot of work on the structural side and I
think that physician practice connections, patient-centered medical home. These
standards for what practices should have in place to organize care well through
managing their patient population, try to get at some of these structural
aspects of care coordination by looking to see whether practices have tools and
processes for tracking tests and referrals.

To some extent the measures that are in PPC reflect – are limited by
what we think a practice should be responsible for, is accountable for. It
looks to see did the practice have a way to track whether information went to
the specialists but in some ways that primary care practice can’t really
be responsible for getting the information back from the specialist because if
the specialist doesn’t want to do it. That is one of the challenges that
we have with measures of care coordination is like where is the accountability
and how do you measure in that set?

I have been working with Jonathan Winer at Johns Hopkins University and
several other of his colleagues there as well as Jinnet Fowles from Park
Nicollet, to think about measuring care coordination specifically in the
ambulatory setting and to look at the process of referrals and consultations,
communication between a primary care practice and medical specialists. We are
just finishing up this work that was an initial phase of trying to identify
potential measures, develop preliminary technical specifications and then to
work with some practices and clinicians to try to understand where these usable
and acceptable measures.

This is the model for ambulatory care coordination that we developed in our
project through a number of iterations with clinicians and key stakeholders and
on the left side you see the logic model, the process flow of care coordination
from the physician discusses options for referral with the patient, identifies
relevant information that needs to go to the specialist, sends the information
with the request. The specialist receives the information, sends the results
back, and shares it with the family. The primary care physician sees the
information acts on it whether that is updating the care plan or talking with
the family.

That was the process flow that we developed and then we talked with our
expert panel and with clinicians and others in practices about what are the
meaningful points along this process where it would make sense to measure that
the measure could be useful for informing the care process. It could be useful
for monitoring care and measuring.

What you will see is that I have these in different colors. The measures
that have the dotted lines are measure concept that we thought about but
dropped and the reason we dropped those, actually several of them have to do
with communication with patients about shared decision making. Our panel said
focus on a narrow set, a small set of measures that we can really get
implemented and they said that is really important but it is not happening so
don’t start there. Updated care plan was the same way. They said really
important but there’s not one yet so don’t try to measure it yet.
Start with the pieces that are most reasonable today and that is where we get
the piece of critical – it is really a referral feedback. It is that the
PCP sends critical information to the specialist. The specialist gets the
information. The specialist sees the patient, takes care of it, sends the
information back, and the primary care physician gets it back, referral loop.

Now what we have also added in here is that some of that information is
communicated to patients both from a primary care physician this is why I want
you to go see the specialist for this reason and also that the specialist
communicates the results to the family. The green ones are the ones that we
thought were primarily on the accountability of the primary care provider and
the blue are on the accountability of the specialist. These are the ones we are
moving forward with.

I also want to mention the one that visits schedule within a requested
timeframe. One of the things that we are interested in from a community
accountability point of view was did this visit happen when a referral was
suggested. We had to drop that for a couple of reasons. One because it
wasn’t clear who would be accountable for it but the other because of this
idea about a timeframe or did the referral happen within a particular
timeframe. This is not something that people document and that is what we
found. I want to talk about some of the measurement issues that we have
identified. This applies both in a paper-based world and in an electronic
world.

First of all, really do any measurement here of completed referrals if that
is in your goal whether you get completed referrals. You have to kind of know
which ones you really expect to happen. When we talked with clinicians and they
said basically there is some referrals that are stats, some referrals where you
really want them to happen maybe in the next couple of weeks, and then other
referrals are just kind of when you think about it or they might be I want you
to get a colonoscopy in the next six months but they are not triggered to
timeframe. That makes measurement very hard and that information we found it
but if it was in there, it wasn’t defined and it wasn’t in a way that
would lead to measurement.

Effective communication with patients and families. We saw where practices
were using clinical summaries, visit summaries that they print out from the
EHR. When we talked with them about that being measure that like did the family
get the results, they said I could tell you that the report was printed. I
don’t know whether it actually got to the family or anybody talked to them
about it. That is a challenge for measurement.

Accountability is an issue that I talked about a little bit that there was
concern about are you going to hold me responsible for getting the completed
report if that specialist never sends reports. What should those be?

The other interesting thing is that we did our work in both integrated
settings and in non-integrated. Communities where we were just talking with
private practices that were interested in they were using an EHR and we saw
really different concerns about care coordination that affect measurement and
affect – one issue is are you worried about patient dumping or patient
stealing. In the integrated systems they were concerned that the primary care
physicians were sending them to many patients that really didn’t need to
be seen in a specialist setting. In the non-integrated settings they were more
concerned about well if I referred this patient to see a cardiologist just to
get a question answered, I will never see that patient again. The distinction
better referrals and consultation – Medicare pays different for a visit if
it is a consultation than if it is a referral. They pay more for a consultation
than for a referral. The integrative settings are really concerned about making
sure that they identify what is a consult versus a referral. A consult meaning
that you are just asking a question and you intend as a primary care provider
to provide ongoing care for that patient. Where a referral is you are
transferring here to the specialist. The integrated systems were really
concerned about this and in the non-integrated settings we really didn’t
see a lot of sensitivity to this. This could really change in an accountable
care organization model where those payments would differ.

The other piece is that actually in talking with integrated systems this
idea of the feedback loop and the information – we share a medical record.
They know that they can see in the medical record whether the patient saw the
specialist. They can look at the report. What does it mean? How do you know
whether somebody actually looked at the relevant pieces in the share medical
record to be able to say that the information was exchanged? This is so hard to
measure.

We actually have done site visits in three integrated settings and two
non-integrated settings to try to think about how to measure this and we have a
grant that we just submitted to AHRQ that we hope will be able to conduct some
research to try to understand this. What we learned is that even in all of
these settings where we had leadership wanting to improve care coordination and
HIT services to support it, we found that either there wasn’t electronic
functionality to report measures or the workflows didn’t really support it
or both. Even in integrated settings where they had this referral requesting,
all the data was free text. We didn’t have that timeframe. We didn’t
have the referral consult. It was very hard to construct a measure of completed
referrals and the same was true in the non-integrated settings where sometimes
the staff workflows didn’t support it.

As we go forward in thinking about how to use electronic health records to
measure this concept, we are really concerned about a couple of things with
EHR-based measurement. One is whether we might have underreporting of the
numerators. If you want to look at a completed referral in the EHR, often that
information is going to come back from a scanned document. It is the specialist
report coming back. It is scanned in, which means somebody has to take the scan
and document and report yes the doctor reviewed it and that is different from
the referral coming back. You might have parent quality failures that really
and truly the loop was closed but the EHR can’t identify it.

The other concern is whether the eligible population. Even when you have
this referral tool available, will people use that and can you track all the
referrals that happen? Those are some of the challenges that we have seen as we
have really tried to get into the details of measuring care coordination and
the ambulatory care setting.

We do think it is valuable to measure this process of care coordination and
to try to figure out how we can get some of these measures to come out of EHRs
but we think that having structural measures to support what the capability
should like and what the workflow should like and the staffing is going to be
important. Monitoring the outcomes will be essential as well.

We are hoping that these kinds of measures could really help to trigger
quality improvement, improve decision support, and help us to monitor the
quality of care.

DR. CARR: What about this? Could you just say did you see the last note from
the last provider and did you do what you were supposed to do? Just have it
sort of yes, no. I am being facetious but I mean we heard about that this
morning where we have so much complexity we become paralyzed and we lose sight
of the immediacy of what we are trying to say.

MS. SCHOLLE: I think part of what we are going to be trying to look at is
where would you put that question.

DR. CARR: Every time you log on to a new patient the first message that
comes up is did you see the note from their last provider. Who was their last
provider? Did you read the note? I am being simplistic.

DR. GREEN: Can I ask you to go back to one of your prior slides? It had
figure one on it, model for ambulatory care coordination. I feel a little
premature here in what I am about to do. What is setting up for me this
afternoon is I want to think about this overnight and come back to it tomorrow.
But what is setting up for me is I am learning something important I think,
which is how essential it is that the development of meaningful measures move
in lock step with explicitly articulated statements about the care process
about which the measure is to be pertinent. Justine, your question about that
and other comments that we make is teaching me how each of us at the table
carries around with us an understanding of the care process in which we
participate as a provider or a patient some way or another and that always is
coloring the way we think about measures and their development.

If you will indulge me I want to make two comments. Most of the
afternoon’s presentations have appealed to me as a health services
researcher type but Sarah’s and also Kathryn’s they appeal to me as a
physician trying to take care of folks. Let me use this figure to exemplify
this. As a physician who also served on the board of directors of a
multi-specialty group practice, this model for ambulatory care coordination for
a person with what we would take as part of our concept is that it was a
dermatological condition. It had to do with skin. It would be completely
different depending on one piece of knowledge. Is this person being in a payor
system in which dermatology providers are capitated or are they in a fee for
service model? Without knowing that piece of information this is nonsense. I
assure you. This is nonsense. You can work on a lot of measures that will have
no meaning if dermatology is capitated and you will be missing the measures
that will be crucial if they are in fee for service.

Jump from that to just what I saw happen a few days ago. Imagine that you
are seeing an 88-year-old mostly deeply demented Alzheimer’s patient who
has an unhealing lesion on the top of his left ear. He has already disrupted
the entire practice because he is confused about where he is. He is getting a
little noisy. He is getting hard to handle. Now we need to decide if we are
going to cut this guy’s – how much of this guy’s ear are we
going to cut off? That is the crucial question. Or are we just going let this
thing keep eating away at his ear until he dies of the complications of his
other morbidities? The way this problem actually gets done is how many of you
have a camera in your pocket right now? Debbie doesn’t seem to have a
camera. The way the ambulatory care coordination occurred was photograph, email
to the dermatologist, phone call saying look at it now, 90 seconds later
dermatologist says if that guy is going to live longer than 6 months he should
have more surgery on this and we can do it Monday and he goes home. Now look at
the things on here that don’t exist to measure what happened. I would
argue that that was optimized care.

I want to go back to my two examples to developing measures that are not
contextualize into the care process almost certainly compromises
meaningfulness. What a humbling presentation you have made.

MS. SCHOLLE: Does it work for many situations of those outliers that you
just described?

DR. GREEN: They work for many situations. Like I said I felt like I might be
premature and I think I am a little premature. I would like to think about this
probably the duration of my tour of duty on the NCVHS. This strikes me as a
pretty significant challenge in getting to the meaningful use of measures.

MS. MCDONALD: Can I just comment on that comment? That is why I actually
shared the somewhat complicated conceptual diagram of organizational design
frameworks because I think that does capture the statements that were just made
that you really do have to think about the exact setting and the exact patients
and what the interdependencies are. You mentioned the form of payments, the
types of contextual factors that would then relate to what is going on that
would then cause you to think about different coordinating mechanisms like
taking a picture and sending that to a dermatologist and there under those
circumstances you had a good fit, the best fit possible perhaps and what is
hard to do is to measure that, but we can intuitively tell if that was a better
option under that kind of context and sending the Alzheimer’s patient over
to the dermatologist. That is why I showed that diagram because I think
thinking in a linear Donabedian framework may not be as appropriate for care
coordination as it is for some other areas.

DR. CARR: Once we have the measures who is accountable?

MS. SCHOLLE: This was clearly a concern. We talked primarily with primary
care physicians and they were interested in having the denominator changed so
that the question about whether the critical information was referred that was
sent with the specialist, that was fine for everybody for whom they thought
needed a referral. But on the last one about the primary care physician reviews
as specialist report, they wanted that denominator to be for people who
actually saw the specialist but then what happened to the people who needed to
see a specialist but didn’t get one and that is really the concern about
accountability. At a community level it should be all referrals.

The other question on the specialist’s side was what about these self
referrals. When patients go directly to the specialists, is the specialist
responsible for trying to get critical information from the primary care
physician when they didn’t get a referral? It wasn’t really a
referral. It was a self referral. The patient just went to the specialist.

MS. MCDONALD: We talked about this in terms of starting up our projects on
inventory and care coordination measures and audiences and accountability and
such. We thought that maybe the first step is to look at those patients that
are in some sort of system of care in thinking about measurement within that
system since a lot of care coordination failures occur in the white space where
no one is accountable. That would be the next level of measurements that would
be needed but then there is this problem of but no one is accountable. If we
measure it what will really happen. Our thinking is that maybe what is most
useful is the start at least with patients for whom they are saying a patient
is under medical home, practice setting, or some sort of integrated system and
that does not provide a full picture of everyone and all care but it would be a
reasonable starting point.

DR. TANG: I am hung up on Blackford’s challenge about looking for
breakthroughs. It looks like the descriptions of decomposing these processes
into steps and then try to instrumental the steps. One question I have is has
the cost of complying with reporting been studied already? Yes. You are nodding
your head yes. You are nodding in agreement. My concern in today’s world
and that is the breakthrough I am talking about. You used to have to chase down
people with paper and pencil and then come in it. That seems so 20th
century and it seems like today care coordination is an outcome. I know it is
made up of processes but it is sort of like what Larry was saying. We all have
our own. They work locally. They work or they don’t work. But we want care
coordination. We don’t want a letter with the date and time stamp and a
– all that does is add to the cost more of trying to subtract cost.

It also does not leverage. In this office you might have the CNS, the
clinical nurse specialist who just drive your practice and thank goodness for
that person and that is how it is going to get done. But if I measured all
these other things not only would I pay for all these other things, I actually
am not helping pay for the person who is driving my care coordination.

Isn’t the time now to focus on the output? For example, readmission.
It’s not such a rare thing. It is 20 percent. Plenty of opportunity to
measure and to detect changes in whatever it takes for you to get care
coordination done. The reason I say the other style is 20th century
is I think we have new tools and whether it is – there is new tools to try
to figure out what our processes are and how to streamline or redesign those
processes but they are local. We are really trying to measure the final goal.
Is now the time focus a lot on the final goals and worry less and burden less
the precious resource we have? Primary care providers just aren’t a dime a
dozen and if we spend a nickel chasing after processes that may not be relevant
in a practice, I worry about that.

MS. MCDONALD: I think that is a really important point. There is a measure
out of a group in Australia. I think it is Clara McGinnis and Beverly
Sibthorpe, simply asked the question of a patient. In the past three months how
often did you feel the care you received was well coordinated? Of course the
issue was having not be the outcome as sometimes patient’s perceptions are
not going to pick up everything. But certainly prioritizing something related
to the perception of coordination on the patient’s part, the family’s
part and the health provider’s part would be important in this whole
picture.

DR. TANG: And I know Sarah wants to say something too but that is what I
thought about when Justine asked her question is in Amazon it pops up and that
is how I like that vendor or how I like the product. Well, when you open up
that consult result request we can ask, did it meet your needs? Just like,
Kathy, you were saying the patient did it meet your needs and to the specialist
did it meet your needs. That actually might be a whole lot closer and more
meaningful in feedback to each other then all these processes.

MS. SCHOLLE: I guess I have been making a point that these are process
measures but in reality they really are looking at depend on structure.
Essentially what you want is that you want this structure to be in place so
that somebody’s asking – when they are sending a patient over for a
referral for a specialist’s visit that you have given the patient enough
information, you have given the specialist enough information to know what
services requested so that the patient understands why they were supposed to go
there and that somehow the information gets back. I do think your point about
are we putting too much – are we investing too much into getting a measure
out is a big challenge to us. On the other hand, we are going to lose these. In
an integrative system this is where the difference between integrative and
non-integrative is huge. In the integrative system okay. Well, the medical
record is there. You are going to have to assume that somehow the primary care
physician is going to look at the data that came back and it is going to be
there and have a chance to do it. But what happens in a non-integrated setting
is that that information doesn’t come back or if it comes back it may not
come back in a timely fashion and you have already seen the patient and you
have already done the same set of tests over again or when the specialist gets
the patient they end up doing those things again. The other kinds of outcomes
you might want to look at would be duplicate testing or numbers of visits.
There may be other ways to try to get at that area of efficiency and they are
going to be different in different settings.

I’m not convinced that we should say no to tracking this. When I talked
with people in non-integrative settings, they think this is hugely important to
try to understand what is happening because they think that a way to understand
their community and their system. If you talk with John Blair, this is why he
set up all the work that he has done in the Mid-Hudson Valley of New York.

MS. MCDONALD: The other reason that the intermediate outcomes or process
types of measures can remain important as long as there is not too large a
burden or pressure on especially these primary care doctors as was pointed out
is that that is often where there is action ability. If you can measure it and
there is already been linkage to something like lowered rate admission rates
when some process has been in place that coordinates care better. If you can
measure that and pick that up as an issue then that is exactly where the
interventions and actions can occur and improve quality. That is what makes
that part of the measurement potentially meaningful and useful.

DR. CARR: I just want to jump in here before Floyd, and really amplify what
Paul was saying. Just as we were talking as I was looking up the commonwealth
fund has put together why not the best, and you can look at all the core
measures. I just looked to see who were the top performers on did you
understand your discharge instructions and 95 percent of them are specialty
hospitals, surgical hospitals, orthopedic hospitals. In that split second, I
learned a lot of things that just one thing, one question, asking the patient
and immediately seeing that the procedural areas have got it down. I can go
back and learn from that.

I am just concerned that the complexity of this is so overwhelming that we
might never get there. We might not get where we are going as we are trying to
figure out how to measure. I think a lot of the benefit of measures is to ask
the question. Again, this was a story that was in Europe or something that some
consultant from IBM told this story, that there is a mountain in Switzerland
and there is a tunnel through it, and they wanted to say if it is daylight,
lights are off, turn them on. If it is nighttime and your lights are on, turn
them off. They struggled with this for weeks and weeks and finally they said,
are your lights on and that was the sign that they put above. I feel like that
is the right question.

We talked about this at the OHIMA meeting. We did this very complex glucose
augment and finally STS and CMS said was your glucose less than 200 on the
morning after surgery. I think sometimes asking one question can catalyze a
universe of systems that will answer in a way that is right for them and for us
to try to take the elements of the answer before anyone has completely answered
it could put us in a cycle that would not see daylight.

MR. EISENBERG: I was just going to add to that because I see a lot of
complexity and I don’t want to go back to my QDS discussion, but a couple
of the elements we identified in there are care goals and experience. The
experience is not just the patient’s experience but also the
provider’s experience. One of our other panels actually through HIT we
have a couple of expert panels coming up. One is in care coordination to
identify what elements are required and can we help push standardization of an
EHR so that we can manage measurement and part of that will be care goals
because it is based on the plan of care and the care experience. I hate pop ups
so I understand, but when I see something did it meet your needs to be able to
answer that answers a couple of questions. One is it means I have read it
enough so you can know that I have looked at. It also means there can be
feedback to specialists. If that specialist isn’t meeting my needs on a
regular basis I may not use them again but maybe somebody else ought to know
they are not meeting my needs. A lot of this is not in EHRs today and it is
written somewhere on the side.

I remember on paper every time I got something I would initial it to say I
have read it. I know people initial it when they don’t read it but it
shows you have looked at it. I don’t want to create that attesting in the
EHR but I think there are ways to go about this that are kind of in progress
that may help us out.

DR. TANG: I just hope we don’t get to a point where we need measure
coordination on top of our care coordination but it really sounds a bit like
that. Another example is Geisinger got a lot of attention for their program
they called proven care. All that is is an outcome measure. I leave it to them
to figure out how to pay less output on that guaranteed price. I think that
would serve us well as a way of measuring care coordination rather than all
these process measures. I don’t know. Maybe it’s a bit much.

MS. SCHOLLE: I think these points are well taken. I can assure you these are
questions that have come up in our mind as we have done this work and tried to
think about what is worthwhile. Where should we be leading the field? It is
really going to be a different issue for integrative versus non-integrative
settings but in the short term for non-integrative settings where having a
pathway to what should you expect and to what information should be shared with
the patient with the specialist. Is the specialist getting the information they
need and is that information coming back? Having a simple way to know whether
on a community level you are accomplishing that is important and it is built in
these pieces of did the information get shared and come back or it is serving
patients and families in a way that is also another expensive approach. We will
certainly as you can tell from our broader framework it is something that is on
our plate to try to think about whether the benefits and drawbacks of all the
different approaches.

DR. CARR: I think we wanted to have a little bit of time to just recap and
prepare for tomorrow.

MR. QUINN: I was going to say another way to look at this is not in the
context of care delivery so much but in the context of social networks. Imagine
a social network with the patient in the middle and all of these other people
around them. There is a world of measurement around the activity of social
networks, how well they are used as well as their indicators of how well
connected a patient is and people in a social network is connected. To use the
paradigm friending someone. Well, maybe we call that a data sharing agreement
signed among a particular patient’s provider to show that they are able to
receive messages. And then we can also look at it and I’m no expert on how
to measure social network activity but how much traffic is on there. These are
sort of things that can be done in automated way versus this. I think what we
are going to find is that looking at those patterns of social networks for
patients some of them are going to be better trafficked. Others are going to be
less trafficked and they are going to be very different networks in an
integrated system versus not. I think that that is a different paradigm than
this for looking at it and it’s not necessarily tied to all the
dependencies here.

DR. CARR: Thanks very much to both of you. It really stimulated a lot of
thinking and a lot of appreciation for the excellent work that you both are
doing. Thank you very much.

Agenda Item: Re-Cap and Discussion

DR. CARR: I realized it has been a challenging day in terms of incoming
information but we had set aside a little time until five thirty anyway to
recap and discuss. One thought I had would be to just get major themes that we
heard today and think about them because our goal had been as we talk about the
national priorities, the care coordination as one, tomorrow disparities, value,
population and health, we want to think about the lessons learned about
measurement today and then take that and apply to those priorities to identify
again are these challenges present in these particular issues. I guess I will
open it up to just have folks throw out themes that we heard today.

DR. TANG: Well, it was a very informative day I thought and I think the
challenge that most fits where I am thinking is what Blackford said. Where can
we go from what we have been doing which is very process laden and based on the
paper world to where we would like to go recognizing that it will have a new
application in health reform and it seems like we want to if you start with the
morning discussion where we talked about the counterpart to CME where you sit
down in some location and try to promote knowledge to practicing with ongoing
feedback and that is where the quality measures can come on. Ongoing feedback
about how you are doing and how to change. That would be really different and
as a consequence that might free us up to instead of saying do you have what we
think should be process A and B and C and look more towards are you getting
outcome E again can be measured electronically through some of the systems that
we are now putting into place. That would be a very different way of looking at
both what to measure and how to measure it and at least it stimulated my
thinking that.

DR. MIDDLETON: Just a couple of quick thoughts building on Paul’s
summary. I just wrote down several themes listening through the day. One is how
do we enable this idea of the real time quality and management and feedback and
reporting and perhaps have an aspirational new approach to quality as opposed
to the sometimes pejorative approach, that is, I’m not doing good enough.
I would rather it tell me, how do I do better. And maybe even combine with that
some of the CME and other sorts of affinity type activities that might relate
very well to both data acquisition and management for quality.

We talked a lot about the payors becoming harmonized around a core set of
national standard quality measures, which certainly would make things a lot
easier on the payor side perhaps, the HIT side, and maybe the provider side, as
well.

There was a subtle point I think, Frank Opelka made about actionable
quality. Make the reports, make the data actionable from the provider’s
point of view to try to bring them along. I raised the issue of what is the
sustainable business case for quality post-stimulus. We are in a great
transitional stage now period, but what happens when all that goes away?

I guess the last thing would be can we through all of this think about a
parsimony quality measures and approach and what national architecture does
that imply to get quality all the way from the point of care, all the way up to
CMS and to get insights and payment reform all the way back to the provider.

DR. GREEN: I thought one of the most important consolidating statements was
there has to be an aggregator data cleaner. I thought that was pretty much
right on. I don’t feel like we have matured our thinking adequately about
this, but I do hear a solid conversation about what was at one point called,
concurrency of clinical decision making, local care arrangement, and
Paul’s point about can there just be a few measures that are independent
of all of these things including type of practice arrangements. That’s
where the sweet spot is. They have to come together. They have to operate with
this word concurrency is a notion.

When they are not concurrent one of the apparent consequences is that things
get very complicated, very fast and they get very expensive very fast and they
probably get undoable very fast. That struck me as an important thing.

Matt didn’t develop it and we didn’t really hear this except from
his comments, but what a great thing to take home tonight to think about is
instead of thinking about this around protocols of care for chronic diseases
for mostly older people that have expensive problems is maybe we should be
thinking about measuring social networks. That is a good question, in my
opinion.

MR. REYNOLDS: What was that again, Larry?

DR. GREEN: That is an observation that instead of us continuing to think
about this the way we are accustomed to thinking about it are these care
processes, care protocols, the usual suspects and players that are trying to
communicate with each other. That maybe we should jump and think about seeing
this as a social networking problem and think about how do you measure the way
a social network functions and does it accomplish what it is supposed to.
Something like that. I’m not giving it justice.

DR. MIDDLETON: I can see the NEJM article, Larry, with your name on it. Just
like your health behaviors are basically contagious. You are only as smart as
the people you practice with.

MR. QUINN: If you are a primary care doctor and you don’t have a lot of
friends, then you are probably not coordinating care.

(Laughter)

DR. GREEN: Let me invert that. I think it was D.F. Fox back in Lancet, about
getting dangerously close to 50 years ago, also pointed out that one of the
functions of one’s personal doctors to protect that person from the over
zealous specialist and we have a lot of evidence that this is true- that this
is something that is true. When you start measuring the social network you can
argue that a robust primary care system that has few the other referral to
visit ratio is low. That measure can either be neglect or it can really robust
care and we have to have a measure that is a way of approaching that
measurement that distinguishes that.

I really like Matt’s idea. I wish we could talk about it more and I
wish we had someone talking to us about it. Do we have someone testifying
tomorrow that knows a lot about this?

DR. CARR: No, we did have that NCVHS afternoon. I don’t know if you
were there.

DR. GREEN: (off mic)

DR. CARR: That is why we keep meeting.

MR. REYNOLDS: First, my comments are not as chair of NCVHS. They are visitor
to this committee. I actually was pretty disappointed. I heard an incredible
number of experts be experts on their piece and experts on why their piece was
good and what is happening with it and I think that is excellent. But I
didn’t hear any grouping up. I have actually been a huge fan of meaningful
use and I use it a lot at home. I use it a lot everywhere I go. Some of the
ability starts sliding further to the right as I listened today, as far as who
is agreed with what’s in what column. If the industry in essence is truly
committed to move this versus prove that individuals or individual groups are
the one. That worries me.

I think some of the things that I did like — I think Blackford we ought to
do a primer on how to think about this with some of the slides you did and that
Paul did. It really decide this idea because again, when you go home and you
start thinking of all the people that are going to have to catch up that
don’t even know this is going on today and haven’t seen the chart and
oh by the way they got to get somewhere between now and 2013 to do something.
There is a lot of help that needs to be done at that end of it and we can use a
social network and we use whatever we talk about to get that group to move. We
have a lot of educated players around this table but there are a lot of people
out there that don’t even know what the table would look like. I think
that is a key thing.

I think we have to have a light to go to and I think all of us complaining
about things like HIPAA but somebody forced it to happen and said this is it
and now we have come out with the next versions, which maybe it is going to
make a difference, but it grouped everybody up. I think rather than just
inviting the vendors, which some of the things that we are going to make sure
that we do something that they will play in because I think we all would say
there aren’t many doctors writing their own systems and there aren’t
many practices out there with a whole lot of time to figure out how to use it
so we better deliver that in the right way.

The standard data set, I will speak as a payor. I would love to know there
was a standard data set and I am telling you we would do everything possible to
push people there so that once you have the standard data set then you could
start actually having audible discussions. Right now it is everybody picking
their own side of the discussion and then you quickly start losing your thread
of what I really think about and what data is or isn’t actually there. I
always call them home base. As an implementer I love a home base. We can argue
away from home base but if we don’t have a home base we just argue on
lines and premises that don’t happen.

I just think it is a great time for NCVHS to be doing this. I think it is a
great time for us to be partnering with David and others, because maybe in some
ways we may be able to say things that may be wouldn’t be able to be said
in other ways.

PARTICIPANT: Say more about that.

MR. REYNOLDS: I would rather not right now. I am only speaking as a visitor
today.

PARTICIPANT: (comment off mic)

MR. REYNOLDS: In other words, all the meaningful use hearings we had
observations. Out of these hearings we could have observations and/or
recommendations in doing so we could help move a ball forward because one thing
is they have their shoulder to the ball. They are having to push a lot of balls
at the same time. We happened to have a clean space to step back for a moment
and we are looking at this – we don’t have columns, but yet we have
understanding of columns and we have understanding of timeframes. We have
understanding of things we have done in the past, and the industry has done in
the past.

Hopefully we could create a body of work that could be picked up as Paul and
I talked, as David and I talked and others have talked, could be picked up and
used in ways to make recommendations that didn’t necessarily come just
from them. That would be my summary.

DR. CARR: I wanted to add one thing too. As we are talking very facedly
today about SNOMED, and SNOMED and problem list. I would just like to point out
that I don’t know many physicians who are nimble with SNOMED. I think
there is a whole educational piece that obviously we will have to do some catch
up work, but I mean really building into curriculum not just physicians and
nursing and all health education. There needs to be a track within the
curriculum about measurement and documentation or taxonomy or something because
we do not have a workforce that understands us right now.

DR. FITZMAURICE: Maybe the physicians don’t have to understand SNOMED
but their vendors do and have to give them choices that will lead them to a
SNOMED concept so that they can be added across.

DR. MIDDLETON: Harry, thank you for opening the door to allow us to maybe
say a little bit more about what we really feel. I struggled a little bit
because I found myself, and it’s not particularly that I know that much
about quality, but I found myself kind of in and out. I found a lot of the same
sort of presentations about kind of the same stuff in their points of view and
what not. That is all well and good. People are expert in their individual
areas and what not, but when I got to the late morning presentation, whatever
that was, I wanted to kind of get us to think about the bigger picture here. We
really have an opportunity to think about the bigger picture because CMS may
not exactly be accountable for quality. AHRQ isn’t exactly accountable for
quality. Who is the quality czar in this country? Is there such a thing? Could
that person with an HIT czar help reform with healthcare reform, help reform
along? I think that is kind of the bigger set of issues that have to be put on
the table. As one of the folks back home likes to say, as Steve always says put
all the blood on the table and then let’s see what is going on. I
don’t think we have done that yet.

MR. REYNOLDS: I would like to see the excitement on the table, excitement to
go to the same place and to make a difference on that list – and try to do
some of this even though I don’t always have the ball.

DR. FITZMAURICE: Social networking. Sunday, my son Jim who is 39 years old,
went out playing touch football with my son Joey who is 41 who was a
quarterback and a bunch of other kids, kids of 41. Somebody caught a pass and
my son Jim put his hand in there to knock the ball out jammed the index finger
so that it wound up breaking two metacarpal bones here. This finger was
pointing in the wrong direction and he drove himself to the local hospital.
They couldn’t get it relocated. They brought in a surgeon who was able to
move it and finally get it in, splinted it. They had a before and after. They
didn’t pin it or anything. He came to our house and his wife came to our
house too, with the kids. He gave me a disc that had a DICOM reader on it and
his images. I put it in and we looked at it and it looked horrible. It really
did. You wonder is that stuff going to stick together or not with nothing
holding it.

We sent it around to the FTSE Google group. It is a group of nine kids and
their spouses, and maybe a couple of others. The suggestions started coming
back. One of them has a relative that just got out of the military. He is a top
hand surgeon in York, Pennsylvania. He was at the Ravens game but he is willing
to not drink, not have any beer, so that he could look at it tomorrow and then
operate on it. Somebody else said here is someone who did Johnny’s ACL.
You can try him. A lot of suggestions just from people who are not very
knowledgeable in medical care. My wife is a nurse. One of my daughters is a
nurse. But we got good information that at least gave him some paths to go. He
decided to go to the ACL guy and get a referral to maybe a hand specialist if
the guy thinks he needs it.

Can you imagine having that with medical professionals, that you send it
around and you start getting this feedback?

What are the themes that I saw? Like Harry, I couldn’t see how to fit
it all together and move forward? I am struck by there is a lot of investment
and demonstration of quality measure effectiveness in changing what we want
changed.

I saw quality measure architecture and workflows. I liked seeing that. I
wanted to see here is the way it should be but I don’t know if we are
there yet or not. But I think that is what Harry wants to see is what should we
be working for. How did we make this happen? How do we aggregate quality
measures for effective change? Provider information for clinical decisions and
for patient choices.

Another theme. Is there a financial reward system that promotes/rewards,
quality improvement and coordination of care? I don’t know. Should we
focus on is the mean greater than the bar or should we focus on is the
distribution changing? The percentage of patients who are below the bar,
percentage of patients who are above the bar. Are we moving toward more and
more above the bar? The bar is wherever we set it.

We have a transition coming up, a transition from quality measures to other
quality measures to performance measures. We need to keep track of the data
elements and their attributes. We need to get into different versions of code
sets and this is over time. This is not just in the next three or four years.
We are having changes in coding systems. How do we handle that and does that
impact – do our measures mean the same thing? Do we have the same validity
as we carry across these changes?

I have already covered the social networking. Maybe a social network is I am
a patient and instead of giving me a disc, can you email this to this Google
group? Can you email it to my primary care physician so that he can get on the
ball and find the best specialist and get me some answers very quickly? It
starts to pull everybody together as a team and with some accurate information
and then maybe physicians come back and say, you know I really need this
additional information. And the tingling at the end of the fingers, is it
grossly swollen? That is what the military doctor said. I need to know that
before I can do anything.

DR. CARR: Well, thank you everyone. Thank you for calling the question here
because I think it really helps us to focus on the job that we set out to do.
With that I think we will call it a day and we resume tomorrow morning at nine.

(Whereupon, the meeting adjourned at 5:35 p.m.)