[This Transcript is Unedited]
DEPARTMENT OF HEALTH AND HUMAN SERVICES
NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS
SUBCOMMITTEE ON QUALITY HEARING
OCTOBER 18, 2010
National Center for Health Statistics
3311 Toledo Road, Auditorium A
Hyattsville, MD 20782
Proceedings by:
CASET Associates, Ltd.
Fairfax, Virginia 22030
(703) 266 8402
CONTENTS
- Introductions – Paul Rang, MD, Co-Chair and Blackford Middleton, MD, MPH, Co-Chair
- Panel I
- Eva Powell, National Partnership for Women and Families
- Judy Hibbard, Institute for Policy Research and Innovation, U Oregon
- Dr. Eugene Nelson, Dartmouth
- Panel II
- Dr. Fred Rachman, Alliance of Chicago Community Health Services
- Yael Harris, HRSA
- Theresa Cullen, MD, MS, HIS
- Karen Kmetik, AMA
- Panel III
- Margaret VanAmringe, MHS, Joint Commission
- Kevin Weiss, American Board of Medical Specialties
- Rebecca S. Lipner, PhD, ABIM
P R O C E E D I N G S
DR. TANG: We have a rich set of testimonies before us. Before we get
started, we wanted to have everyone just go around the room perhaps and give
introductions including the speakers who are here.
DR. MIDDLETON: Blackford Middleton from Partners Healthcare, Brigham and
Women’s Hospital, co-chair of quality subcommittee and member of the NCVHS Full
Committee, and no conflicts of interest.
DR. GREEN: Larry Green, University of Colorado, member of the Full
Committee, member of the Quality Subcommittee, no conflicts.
DR. TANG: Paul Tang, member of the Full Committee and co-chair of the
Subcommittee on Quality, no conflicts. Any other members on the phone before we
go around the room please?
DR. CARR: Hi Paul. It is Justine Carr, Caritas Christi Healthcare, chair of
the Full Committee, member of the Subcommittee, no conflicts.
DR. TANG: Thanks Justine. Anybody else on the phone from the committee?
(No response)
(Introductions around the room.)
DR. TANG: Thank you very much. Okay. This is a very interesting hearing and
it is sort of a continuation of work that the Quality Subcommittee has been
doing over the years — talked about maybe a few years back — we talked about
quality measures in anticipation of the arrival of the electronic health record
system. Unfortunately over the past year and half or so, it looks like that day
is coming sooner than we — we were worried that sooner than we thought, and
that is a good thing.
We last talked about meaningful measures, piggy backing on the meaningful
use initiative as CMS that is providing additional incentive for adopting and
effectively using electronic health record systems. We are continuing on the
journey of improving measures that really matter. That is meaningful measures.
And today we are starting to look forward towards a roadmap of if really we
could have without being tethered to our path. In the past most of these
measures were developed and defined based on claims data or things that you
would get on from paper records.
With the incoming electronic medical record systems and the personal health
record systems that other electronic resources of data it would appear that you
could gleen other information that might be even more useful than the
information we have had in the past. That is sort of where we have our sights
set untethered by what technology might be available and what data sources
might become available in the future.
We would like to ask the question first, what information would be useful to
various stakeholders? Whether it is to consumers, to providers, the health care
professionals, the accreditors, the payers, the employers. What would be useful
in helping them make decisions related to health and not only health care? That
is sort of a setting and the context for this hearing. Anything further,
Blackford, you want to add?
DR. MIDDLETON: No. I will just say welcome again to everybody and echo
Paul’s comments. I think this is a very exciting time for measurement and
measurement from health care information technologies and a variety of sources.
But again considering this from a quality roadmap point of view, what are the
sources of data that might be relevant to future measures of healthcare both
quality and measurement of value, which is increasingly under intense scrutiny
and focus. And with a 3 to 5-year time frame, is our perspective getting these
multiple views from different stakeholders would be extremely helpful.
DR. TANG: Let me just review the panels we have for today, and thanks to all
the panelists for really participating on short notice. The first panel is
going to concern information – what information and measures would consumers
benefit from as they make decisions about their health and health care?
The second panel looks at the same kinds of things. What information and
measures will providers desire and make use of in order to measure and
continuously improve their quality and their accountability?
The third has to do with professional organizations and the accrediting
organizations and regulators. What information can they use to perform their
tasks in terms of overseeing professional accreditation?
And finally, the payers and purchasers, how do they assess the value of the
care and the services provided in the health industry?
That is what we have on tap for today, and we are going to begin with our
first panel which has to do with looking at from the consumer point of view.
And our first testifier is going to be Eva Powell, who is the Director of
Health Information Technology at the National Partnership for Women and
Families. Thank you, Eva.
Agenda Item: Panel I – What Information and
Measures Will Consumers Need to Execute Their Functions As Educated Purchasers
and Stewards of Their Own Health?
MS. POWELL: Thanks Paul and thanks for the opportunity to speak to you
today. I will start out a little bit by talking about the current environment,
which I won’t go into great detail about because we all know what that is, but
if you will advance to the first slide, the consumers and quality.
We have done a lot of work with consumers and consumer advocates to talk
about quality and to get from them what their concepts of quality is as well as
understand what things could go into leading them into more active involvement
and evaluation of quality and making decisions based on data.
And what we have learned is that there are a number of barriers in the way
of this and one is the fact that there are a couple of other things that are
much more prominent on consumer radar screens with regard to health care and
quality. And those two things are cost and access. There are more immediate
concerns for most consumers than quality as those of us in the quality world
think of it. And some of the reason for that is there are so many hurdles for
consumers to overcome before they ever encounter health care of any sort
whether it is quality or not. They have to find a provider. They have to
navigate the insurance with some. They have to make an appointment which often
times is not easily accomplished.
Then creating some sort of relationship with the providers is something once
they find that provider that most people want to achieve. Then also determining
how they are going to pay for care aside from navigating the insurance system
whatever out of pocket cost they have consumers have to figure out how they are
going to pay for this and what exactly am I going to end up paying?
Consumers really need data and information that is meaningful and useful to
them. Any data and information that can first of all help them clear these
first barriers would be useful, but I think my point in talking about this is
more to keep these things in mind as we move toward quality measurement from
the consumer perspective and what is important to them. I think if we come with
data and measures that are more meaningful, I know we can — overall experience
with health care system then they will come closer to using that.
In thinking about consumer needs I think it is useful to think obviously in
the broader context of health reform. On this slide I just kind of started with
the goals and objectives have already been laid out by the Secretary in her
draft of the national quality strategy and these are the three primary goals.
What I have done is pulled out some of the objectives where I think there is
the most opportunity and immediate need for consumer focused data.
First of all, person-centered care, which is one of the objectives under
better care will really require better consumer use of data not just at the
individual level which is I think what most of us think about in terms of
consumer data and how they might use that. It is on an individual level that
makes decisions about providers or insurance plans, and I think that is
certainly an important area to focus on.
However, there are two other areas that I think we need to bear in mind that
I think some measures and some data will serve all three purposes, but I think
there may be some differences that we need to look at for the other two
purposes and those are a governance purpose. If we are going to have a
person-centered health care system, that is not going to be achieved if all the
other stakeholders are making all the decisions and then we hand it down to
consumers. Consumers must be involved in governance and they will need data to
help them make decisions in their roles in that capacity.
And then also on the local implementation of that, and I think that has a
governance piece to it, but I am thinking mainly there about consumer
involvement and the actual quality improvement process which really isn’t a
prominent thing right now. But I think if we are talking about systems
improvement and really overall learning health care system, which we are
ultimately after, perhaps not in the 3 to 5-year timeframe, but ultimately we
need to be thinking about how can we learn from consumers’ experiences. And I
think at any point that a process touches a consumer that is a point where that
consumer can help shape and improve that process. There is a data need there as
well.
One of the things we have learned from consumers about their concerns and
their needs with regard to quality is that very consistently their problems
with the quality or their perception of quality issues revolved around
communication and coordination primarily. Whatever data and measures that are
developed ultimately really must address those problems as well as others, but
those are two of the key issues that consumers will be looking to evaluate.
And actually this brings up an issue of a data source because for much of
this, for the communication and coordination dimensions of quality there will
be times when consumers themselves are really the most valid force of the
information because they are the one consistent actor, if you will, in the
process.
And then that I think leads to another issue with regards to reform as we
start implementing some of the newer payment models as well as delivery models.
If we are going to engender consumer trust in those models then I think one way
to achieve that is to include consumer generated data in our accountability
structure for those models.
And that helps us get at the concept of value, value to whom. And obviously
every stakeholder has a stake in the value and has perhaps a slightly different
definition of what value is and all of them are relevant. But right now there
is really not a way of measuring and holding people accountable for the things
that are of value to consumers. Data as we move forward needs to be able to
enable that.
So how can these data need to be translated into actual measures? As I
thought about this, I think there are a number of criteria that can help direct
our efforts. First of all measures have to resonate with consumers in their
overall experiences with the health care system. Data also has to be available
on multiple different levels, those for individual providers and for overall
facilities or group practices. And then data also has to be available across
provider settings.
Another criterion is that the data must really show real differences between
providers. In other words, if we have data that has been developed using a
methodology that shows that 90 percent of the providers are in the average or
acceptable category and then there may be two or three outliers on the good end
and two or three on the bad end, that is really not very useful for consumers
or really for anyone in terms of making decisions based on quality. There are
the methodological issues probably that need to be worked on there, but also in
the presentation of that information needs to be done in such a way that they
are very clearly some differences in specific dimensions of quality.
Then flowing from that we also need to have data that are comparable so that
regardless of whether that person is making a decision between Dr. A in this
practice and Dr. B in another practice or with another health plan the data
needs to be comparable so that it is valid to be used for decision making both
in terms of helping consumers use that data, but also in helping those who may
not understand that there could be differences that makes the data
noncomparable to keep them from making bad decisions on data that they thought
was comparable, but really wasn’t.
And then finally comprehensive information is really important. I think
there are new pieces of information that we don’t yet have and that is an area
of development in terms of consumer-generated data. Those are perhaps some of
the newer concepts that we will need to work on. But some of the older
dimensions of quality, if you will, such as the clinical quality measures and
some outcomes measures and developing them more fully. I think it is not an
either or for consumers and for any other stakeholder really to be able to
judge the quality of a provider. There needs to be a comprehensive dashboard of
measures, if you will.
And the reason for that and I think this might go for other providers or
other stakeholders as well, but consumers vary and individual consumers have
different needs. What is really important to one consumer may not be so
important to another consumer in terms of decision making. We need to find what
those specific dimensions of quality are that are broadly important to all
consumers, but then present that information in such a way that people continue
to use those things that are most important to them.
And included in that full picture you have on the slide here. I won’t go
through each and every one of these, but outcomes data is critically important
and specifically patient-focused outcomes measures; things such as functional
status, health-related quality of life, safety. All of those things are
dimensions of quality that will really be very useful and very meaningful to
consumers.
Often times consumers when they are making a decision about a treatment or
providers for other people who have what I have and who have similar
circumstances and presentations, what can I expect if I choose A over B? Those
dimensions, those kinds of outcomes are the things that I think will really
resonate with consumers.
And then also in the area of specialty care there should be some priority
given to those specialties who have — where there is more opportunity for
consumers to make the choice such as OB/GYN, orthopedics where there is a long
process, a decision-making process and there is not an urgent need for care
that sometimes by necessity circumvents the use of quality data.
One more note, I put cost at the bottom of this slide. I wasn’t certain how
to handle this. This is perhaps something for the committee to discuss. In
terms of quality and decision making there needs to be transparency of cost
information. This is perhaps a separate issue, but at the same time if we are
talking about quality, efficiency, that is where this kind of comes in. If we
are talking about the data needs of consumers then cost data is a data need.
When we are talking about different potential treatments that may not have a
huge difference in potential outcomes, but if there is a huge difference in
costs then that is a really important thing for a consumer to know. I hesitate
to talk about that strictly in a measurement sense, but it is certainly a data
need.
Finally, the last slide here, I just wanted to lay out a few ideas for where
we might go in terms of next steps for meeting consumer data needs. First of
all, it is prioritized development, endorsement and use of patient-centered
outcome measures, and then secondly to identify some robust methodologies for
collecting and using patient-contributed data, and then finally coordinating an
integrated and consistent approach to measurement as well as reporting among
all public and private stakeholders.
And this is I think, a place where we can think about what Paul and
Blackford brought up early on is the context that we are discussing this
includes rapid advancement in the HIT world and there are certainly plenty of
opportunities to leverage what is going on there. And some ideas for that given
that that is where I have been 24/7 most of my time is to collect and use the
right information from appropriate team members.
And what I mean by this is and I guess this would relate to a data source is
that most of the work being done in HIT is very physician centric in terms of
we are talking about that from the sense of the physician will be entering the
data in the EHR, the physician is being held accountable and will receive
incentive money and there are some understandable reasons for that.
But what I worry about with regard to that is that there are many very
relevant and extraordinarily necessary pieces of information from a consumer’s
standpoint that will never come from a physician and one example — in my
former life I was a social worker for 10 years at a teaching hospital and there
were many decisions made about the care a person was going to receive based not
solely but in large part on that person’s social situation. If there was a very
complex treatment that was clearly the indicated treatment for what that person
needed but their special situation did not support successful implementation of
that treatment then I am not sure that we can say moving ahead with what is
clinically appropriate is appropriate and is good quality care.
Until we begin to incorporate not just the clinical information but also
social information as well as information from other disciplines that get at
functional status such as physical therapy and occupational therapy those are
all elements of data that are extremely important if we are really going to get
to quality outcomes that are patient centered.
Standardizing information particularly with shared care plans maybe also
care management tools. These are all things that are HIT pieces, but also could
contain information that ultimately could be fed into an HER, into clinical
decision support systems to support the collection of data and all of this is
being discussed in some of the efforts that NQF has going on with the quality
data set, but those are all direct lines to data sources and the data needs of
the consumers.
And then finally some of the basic pieces of the puzzle here I think are
just basic electronic communication methods and how can we start employing
those to communicate with patients about their individual care, but also to
receive feedback from them on their experience of care and get at quality that
way. And that is certainly an area for development, but one with the
opportunities in front of us in the HIT world we can begin now working towards.
Thank you for the opportunity.
DR. TANG: Excellent. Thank you very much Eva. Lots of good information here,
precisely answering the questions that we asked. Thank you.
Before I go on I wanted to, one, acknowledge Mike Fitzmaurice who joined us
and maybe you want to comment, introduce yourself and your conflicts if any.
DR. FITZMAURICE: Michael Fitzmaurice, Agency for Health Care Research and
Policy. I apologize for being late.
DR. TANG: Thank you. Any other folks on the phone, if you would introduce
yourself, including either public or speakers.
DR. NELSON: Gene Nelson from Dartmouth. I will be a speaker.
DR. TANG: Great. Thanks Gene.
MS. HIBBARD: Judy Hibbard from the University of Oregon. I am a speaker.
MS. BOWMAN: This is Danielle Bowman from Health and Human Services, Office
of Healthcare Quality.
DR. TANG: Anyone else? Wonderful.
DR. CARR: It is Justine. The presentations are great. Are there slides? I
know you have the slides there. Are they available electronically or on the
web?
DR. TANG: There will be shortly I am told.
DR. CARR: They will be posted on the website?
DR. TANG: Yes.
DR. CARR: Okay. Great.
DR. TANG: Why don’t we open for some questions for Eva and we will take
questions for each panelists and then at the end we will have some open
discussion with all the panelists.
DR. GREEN: Eva, can you say more about that bottom word that you said you
didn’t know quite how to present called cost? What do consumers want to see to
make comparisons about cost? What is the information they are looking for?
MS. POWELL: I believe the consumers can use just absolute cost information
which really isn’t available to many people if any right now. My line of
thought here is that again using the example I used before. If there is a
situation where there is a shared decision making process with the person’s
physician over potential treatment, there is not a clear, preferable, clinical
decision in terms of which one is better and there is a discussion about this
is what this treatment involves, this is what that treatments involves. What
are you particular circumstances? Which of these or your concerns about those?
The decision making process somehow seems to me to be incomplete if there is
not information available to the consumer such that they can use that
information in their own decision making about which one is better for them.
In other words, if this is a person who has limited means and there is
clearly significantly less expensive route and when I say less expensive I mean
less expensive from their own pockets.
The difficulty here is that this very rapidly goes down into the death panel
discussion and I think that is the reality in the environment we are working in
we have to be weary of. At the same time the complete lack of that information
is not helpful to anyone. I think this is a bigger issue than consumers. I
think it is an issue system wide.
But thinking from the consumer perspective in an environment where it is so
difficult when you are making a decision about a treatment to know exactly what
it is that I personally am going to have to pay. I just somehow think that that
is a huge gap in the availability of information. And I do view it differently
than quality data. At the same time it seems to me like the two pieces go
together. That is why I mentioned it and mentioned it last. Does that answer
your question?
DR. GREEN: Yes, it is helpful. Can we tease it down just a little bit more?
If I heard you right, you are saying consumers want to know what it is going to
cost them out of pocket for the options that appear to be equally satisfying,
another comparison. Is that correct?
MS. POWELL: I think in the instances when there is a comparison to be made,
in other words, if they are thinking treatment then yes they would use the
comparison between the two treatments. But if there is a treatment they decided
on and they are also in the midst of deciding about a provider or they are
overwhelmed by the thought of this amount of cost coming out of their pocket,
there may also be an opportunity to use that as a criterion for choosing a
provider. Someone from the payer group may speak more to this point about how
that might engender changes in behavior from the provider’s perspective, but it
seems to me like from a consumer perspective most consumers by the time they
get to the point of talking about different treatment options they probably
have already chosen a provider.
But there may be instances when people and again I am thinking out and how
patients might ultimately use data. If they are going to comparison shop then
they are going to comparison shop on what their clinical needs are and what is
going to meet their health care needs. But if they are paying a lot out of
pocket then that is something they are going to want to compare on as well.
DR. GREEN: I get that. I want to go to the word stewardship and your quality
dashboard. I would like to try to unite them and get your opinion about this.
If I heard you right, you are saying you have consumers need a dashboard that
gives them information that allows them to make comparisons. We are really
interested in quality. It is a quality hearing here. The dashboard has measures
of quality of some sort. I am hearing you say you are not quite sure how to
bring this up, but it needs to say something about cost.
And then in our questions that we are asking in this panel we are interested
in helping people be stewards of their health. And you are suggesting that
people might actually be — the consumer might be sensitive to the need to make
decisions about their care based on quality and cost and access. That is what I
heard.
MS. POWELL: Well and those definitely. The cost and the access are the
things when you talk to consumers initially and talk to them about the health
care system, those are the things top of mind usually. If you peel a little
further, you get to the —
DR. GREEN: Let me nail down just to almost a yes or no question. Do you
think consumers give a damn about the total cost of their care or do they just
care about their out of pocket cost?
MS. POWELL: I think probably just out of pocket cost right now. These are
questions that I think we will see when we have information available.
DR. MIDDLETON: I wanted to follow up Larry’s questions. Eva, again, thank
you for an excellent presentation and a great overview. I am stuck on this
patient centeredness issue a little bit. I certainly think I know what it means
in many ways and all of us have an image of what it is. As a clinician if you
are evaluating treatment A versus B for the patient before you it matters
whether or not there is social context or environmental or clinical or even
financial context allows you to think about that recommendation of a specific
treatment for a specific patient.
But when we turn to trying to measure this idea, it becomes very complicated
very quickly. Because of the confounding of patient preferences and costs, if
you will, externalities even at a level. When you think about patient
centeredness and I am just thinking at the top of my mind of miles per gallon.
If I am going out to shop for a car, I can look at miles per gallon. I have a
very clear indicator of some level of one assessment of value for two different
cars. What would be that thing if you were coming to see me or Paul for your
next clinical encounter, God help you? If you wanted to choose between two
doctors, what is the right thing? What is that essential measure or what are
the flavors of it that you think have to be incorporated in a single measure or
in a composite measure that a user can use, an end user?
MS. POWELL: I think bringing up the concept of a composite is a helpful
place to start because of the nature of the individual differences. When I
think about this, I think about the questions that I would ask if I were in the
situation of making these decisions. And, one, getting at what you mentioned
about the individual differences and patient preferences does this provider act
according to patient preferences. If you ask that question then that is what
relevant to me. I don’t really care what all his patient preferences are and
how many times he gave advice using an Arabic interpreter versus a Spanish
interpreter. That doesn’t matter to me.
What matters to me is this a provider that really acknowledges this as a
dimension of quality that he or she must work toward achieving. In terms of how
to form the measure or collect data on the measure I think that is where we get
to some of the ideas I talked about before incorporating social assessment
information. I think of it also as a clinical decision support role which can
then feed into measurement in the same way as clinical information feeds into
clinical decision support than ultimately why can’t social information feed
into that. In other words, if there is a — I will use my own experience.
I worked for six years on a transplant program and there were a number of
times when after all the assessment was done the psychologist and I recommended
not listing a person for transplant because of social factors. And sometimes
that fed into the overall decision not to list that person. Sometimes we got
trumped by other things and almost without fail that person did not do well
with transplant and it was a rough and rocky and just a difficult road from
start to finish.
It always made me to think if perhaps we can incorporate those social
factors particularly for those treatments or procedures that really require a
lot of a patient and family or a specific thing from a patient and family once
they are no longer in contact with the health care system or not in daily
contact or under the care of that provider institution then that would then
become a flag. It wouldn’t necessarily say okay always if this is the case then
this person doesn’t get this treatment obviously. It would be a flag just as
CDS is a flag for doing particular things, adding additional supports, doing
further assessment, those kinds of ideas. Does that get at your question?
DR. GREEN: Yes. Thank you. And a quick follow up. You also mentioned a
couple of times you alluded to the governing bodies or the need for governing
bodies or governance to help consumers or help processes enable consumers in
the ways you described. I am not sure what those governing bodies are. Can you
say a little bit more about that?
MS. POWELL: I think the federal governing bodies which I think we have done
a good job of having consumer presence on there. But there also and again
thinking in the HIT world, the state HIE effort is one example. How often are
consumers involved in the decision making on the local level? We at the
National Partnership are starting to get more and more calls from consumers who
actually have been very active in their state efforts to this point and because
of local politics whatever decisions they have already made that have been very
consumer friendly have been kind of disregarded in favor for the more
politically relevant of viable situation. There is just a lot to be considered
on the local level because there are differences there. There may be a
different decision made in one area than another because of the individual
local circumstances and that is neither good nor bad, but consumers must be
involved in that decision-making process if they are going to accept this.
And I think there are different questions. Some of the same questions but I
guess in my thinking that is where the population quality measures come into
play.
DR. CARR: Blackford.
DR. MIDDLETON: Yes.
DR. CARR: Blackford, it is Justine. I would like to make an observation
about the connection between what we are hearing this morning and what we heard
in July on the health plan ID testimony. At that testimony we heard the
importance of having electronic transactions permit both the physician and the
patient to know what is covered in their program either on that day or going
forward. I think I am hearing a lot of endorsement from a different audience
here from the patient perspective that how important that is in the moment.
MR. QUINN: Something that Eva said that really resonated with me is the idea
that consumers need to themselves reflected in measures of quality and other
aspects of care. I think we are going to hear this a little bit from the other
speakers in this panel, but there are some methodological issues that need to
be addressed especially with regard to clinicians accepting these measures if
there is consumer input involved.
This leads me to think about and this is the same for other quality measures
in other domains the difference between internal quality improvement efforts
and the measures that are collected for internal feedback, growth, improvement,
and then measures that are reported publicly or externally, for external
agencies. I think that if you were to talk to most clinicians they would say
the measures that are reported externally are ones that could be standardized
and other things, but aren’t necessarily the ones that they really internally
run their practice based on. It would be good to see where that intersection of
meaningful consumer measures that are methodological sound enough that
clinicians can accept them for internal use. And then I think that there is
going to be a higher level of methodological soundness that will be required
for them to be used externally.
MS. POWELL: I would agree. I think the place to start is the internal place
because that is I think where you are going to prove the value to the
individual clinician is to their practice environment. And then it is also
where you work out the methodological kinks that obviously need to be addressed
before we go to something like public reporting and accountability.
DR. TANG: That is something to build on this then Mike. Going back to — one
of the things that struck me about you made — first of all I really liked the
attributes you brought up in your presentation in terms of what is meaningful
for a consumer. One of the statements you made was in saying everybody is 90
percent of something isn’t that helpful. Let me try a couple of things and get
your reaction because I think this is something that we talk about a lot in the
various quality committees that are operating in this area.
One is let me build a case from Blackford’s miles per gallon. Miles per
gallon is done in a very standardized rigid way and you have a number and there
is no such thing as a standardized driver. One question would be would it make
more sense to you to stratify that. This is what a city driver would get. This
is what a person who does long commutes on a highway would get so that it can
be closer to what is meaningful for me because I know I do X.
The other approach I want to take is there are — I don’t think you used the
patient satisfaction which I think is good because there are companies out
there that do these patient satisfaction surveys. And it turns out for whatever
reason at least it seems to be that patients who return their surveys always
rate doctors highly or rate them high. Therefore, most everybody is in the 90s
percent in terms of satisfaction on these surveys. But then these companies
will turn them into percentile which means you could be 1 percent different in
the patients who rate you high and yet it makes a 10, 20, 30-point difference
in the percentile. And is that meaningful?
Two pieces. One, is it better to standardize, i.e., risk adjust on the
measures so that you get a number that is in theory comparable but doesn’t
actually answer your question and the other is what is the implications of
having these things by percentile when everything is bunched around let’s say
the 90 percent.
MS. POWELL: I hadn’t thought about it that way before in terms of
stratifying by consumer characteristics but I think ultimately if we could
figure out what those characteristics might be, I think, yes, that would be
helpful. I am not sure. Sorry, so your second part of your question again.
DR. TANG: The second one was turning these high percentages of favorable
opinion about patient satisfaction, so-called patient satisfaction, turning
them into percentile which ends up making small differences in percentages of
favorable opinion into large differences in percentile ranking because they are
all clustered around high numbers.
MS. POWELL: I think with that there are a number of ways to get at that.
First of all, yes, I didn’t say patient satisfaction because we prefer to talk
about patient experience because they are two very different things. Patient
satisfaction generally has more to do with was my food hot, was the nurse nice
and those are all important things, but they don’t have real bearing on the
actual quality of care. Patient experience, the things being evaluated saying
the HCAP surveys and the other CAP surveys get more at things that do come
closer or in some cases — to have a bearing on quality. I think that is really
— when we are talking about measurement and data, that is where we need to
focus.
And I think this is also where HIT comes into play and I am not a
statistician so the methodological issues I will leave to someone else, but I
think part of — what you say about those who return the surveys are the ones
that are enamored with their physicians. If we can make the completion of the
survey — if we can gather survey results from a higher number of people then
my guess is that we will reduce that effect somewhat and HIT is a perfect
opportunity to address that issue. I understand that there are methodological
issues there, but it is still an opportunity. And I suspect that those
methodological issues can be worked out. That would be one thing I would say
about that.
And that also addresses, I think, a concern on the provider end of the
expense of doing those kinds of survey. It also enables again back to the CDS
idea I had before; the electronic gathering of that kind of information. And if
there is a way of gathering that information at the point of care, you may be
more likely to get overall, yes, I am really satisfied with this. However,
there was this one time and this was not a very satisfying experience and this
is why. Those are the kinds of things that I think are lost in the current way
those surveys are done because it is after the fact, but that also — that
moves us towards this learning health care system.
It also is an opportunity for HHT and to use the electronic environment to
feed that information back to the provider in a way that informs their ongoing
practice, but it also then gets it — it kind of addresses Matt’s earlier
comment about the internal use versus the reporting use. I think perhaps that
might be a way to start addressing some of the things Matt said. I don’t know
if that really addresses your question. But I do think that the more we
actually gather that information and report that back certainly to the
clinician, but also make that information available to consumers so that they
begin to understand the various dimensions of quality and that is good.
DR. FITZMAURICE: I, too, enjoyed your presentation and I think in good part
because of your background having spent so many years in social service in a
hospital. You put your feet in the shoes of the patient and said what kinds of
decisions do they need to make and then how can I help them make the decisions.
As part of a research agency, we ask how do we get consumers to use federal,
state, health plan or association supplied quality measures since that is a
good part of our business at AHRQ. We ask are the data good, is the method good
so it gets at your robustness. Are the quality measures meaningful to patient
outcomes and were the providers patients adequately described? Were they in
fact sick or were they not?
I want to ask, one, is it a good thing for quality measures to — here is a
quality measure that we found out about the provider, but also to present a
provider feedback or a provider statement saying my patients were sicker or I
won’t say I had a bad day, but look at all that I did do during a day to put
into a balance or perspective.
Secondly, you talked about what I interpret as personalizing the quality
measures, giving them functional status, tell them about how this treatment
affects their quality of life. Is it safe? Would you go as far as to say some
of the quality measures given to patients should be disease specific and
tailored to the individual’s characteristics? Maybe they have — conditions.
Maybe there is something else about their family situation or their ability to
afford care that the quality measures ought to be tailored to that. I am asking
more the first question. How do we get consumers to use them? And I liked a lot
of your answers and I am probing to get more should we have physician feedback
also presented to the patients. Should we take into account more what is
meaningful for a particular patient who is going to be reading the quality
measures or looking at it?
MS. POWELL: I think on your specific question about should we include the
provider feedback in terms of kind of an explanation behind whatever score they
got. I hesitate to look favorably on that only because everyone says their
patients are different and yes there is a need for risk assessment and
incorporating some of those methodological issues. The patient seen at a small
community hospital are not the same as those seen at the tertiary medical
center.
But I think we can — again, not being a statistician it seems to me there
are methods to accommodate that via methodology rather than add that complexity
and confounding factor for consumers to have to figure out. And that also
sometimes ends up being an excuse. And I have spent the majority of my career
in the provider setting so I certainly identify with that. And there are. Every
patient is different and you are under the gun in the provider setting. I
understand those arguments, but I think ultimately there are ways to deal with
that and adding that dimension or that statement would only confuse consumers.
In terms of how to get consumers to use the information, again, the biggest
piece is making it relevant and we have talked about some of those ways. Still
consumers trust their providers over pretty much any other stakeholder. I think
a lot of that boils down to do the providers see value in this.
In another life I worked for the QIO in North Carolina. I know from that
experience and that experience was primarily with hospitals, but the more
advanced hospitals, the hospitals that really took quality seriously in the
early days beat CMS to the punch in putting their quality measures on their own
website. I look at that as being a really innovative thing to do and a lot of
those hospitals frankly were very small community hospitals.
I think a hospital that does that, a provider that does that that takes
quality seriously and understands the value of quality measurement to the point
that they are putting it in places where patients and consumers see it and then
that brings up the opportunity for it to be part of a conversation which then
becomes part of that provider/consumer relationship. I think that is how you
get consumers to use it once you have made the data relevant to them.
DR. TANG: So Larry because we are short on time, would you mind if we did it
at the end? Could we do that? Okay. Thanks very much, Eva.
Next on the panel is Judy Hibbard and Judy is a professor at the Institute
for Policy Research and Innovation at the School of Architecture and Allied
Arts at the University of Oregon. And Judy is going to be talking about patient
activation. Judy.
MS. HIBBARD: Good morning and I am sorry I can’t be there with you this
morning. Assuming you have my slides. I actually would like to start by
answering the question that someone asked about patient centered. What does it
mean to be patient centered? The way that I look at it is through the lens of
my own research. I look at it as the degree to which you as a provider
understand my job as a patient in managing my health and my condition and
actually help me do my job. To me that is what patient centered is.
Let’s go back to the quality measurements and what is going to be meaningful
to consumers and patients. One of the issues that has been a major barrier for
consumers to use quality information is they don’t understand it. We can go to
the first slide. What is meaningful in what we have seen and studied is that
people do understand the patient experience data and many find it meaningful
and useful. Based on that I think if we had more patient-generated data it
would also be useful and actionable to patients.
I want to talk about two major categories: the patient-generated data that
focuses on improvements and health status and functioning and the improvements
in people’s ability to self manage their condition. Let’s go to slide number
three and just think about the criteria for these types of measures. Of course
we want valid and reliable measures. We want measures that can be used to
improve performance that are actionable on the provider side. But we also can
find measures and use measures that can be used to actually inform the medical
encounter and to improve the care of the patient providing the information.
We want measures that are sensitive enough to reflect changes resulting from
medical intervention. And also I believe we want measures that are going to
shift the focus of performance to an area that consumers value and understand.
Let’s go to the next slide. I understand that Eugene Nelson will be talking
about the patient-generated data that focuses on health status and functioning.
I am going to talk about the ability to self manage. Now if we start with the
assumption that an important outcome of high-quality care is that the patient
should be gaining in their ability to self manage then we should be measuring
and tracking improvements in this construct which I will refer to as activation
as an intermediate outcome of care. In activation it means that the patient has
the knowledge, the scale, and the confidence to manage their health and their
health care and to also understand that they have a role to play that believes
this knowledge and skill.
This is actually something that can be measured. We can go to slide five. We
measured it. We developed a measure of patient activation that assesses these
characteristics. We used RASCH modeling analysis to do this and we have come up
with a very robust measure.
Now what I want to do is just tell you a little bit about the measure itself
and what we have learned and how this relates to these questions of the
criteria that I laid out in slide three. The patient activation measure is a
13-item measure. In slide five what you are seeing is the built-in difficulty
structure of the measure that is how hard it is for a person to say, yes, that
is true about me. And we have them measure, translated into 16 different
languages. We have evaluated the data on about six of those languages and that
difficulty structure is maintained across language and culture as far as we
have observed.
Slide six just shows the actual questions in the measure. We can go to slide
seven. This is where some of the key learnings have been. We have had an
opportunity just to learn so much once we have a valid measurement. One of the
things we have observed is that people appear to be — that this idea of
activation is developmental and people appear to go through these different
levels on their way to becoming effective managers of their health.
In in-depth interviews we have seen that people who are low on this
dimension tend to be overwhelmed with the job of managing their health. They
may not understand that they have to take a role. They have had a lot of
experience with failure. They have tried to do it. They couldn’t and they
become passive as a result. I think this is — and at the high end you can see
the mirror opposite.
This is important when we think about what happens to people in the medical
encounter where especially those with chronic illness are given a big long list
of all the things that they need to do to change about their life. For people
who are low on this dimension, they may try to do it, but when they can’t their
natural human reaction is to do nothing. They end up having another failure. In
essence by not really understanding who we are dealing with in a medical
encounter we are setting a lot of people for failure and kind of keeping them
in that passive position.
I am going to just go to the next slide. We have seen in multiple studies
actually all over the world with different investigators the activation measure
is predictive of preventive behaviors, healthy behaviors, disease specific
self-management behaviors, and information seeking behaviors.
If we go to slide nine, I will share with you another major aha we had in
this and this is looking at how someone manages their hypertension. These are
divided into the percentage of people who actually engage in the behavior by
level of activation. This is what we call a behavior map. We developed many of
these. And the big lesson from this was that almost every behavior is related
to level of activation, but as behaviors become more difficult and complex and
requires sustained action fewer people in all the levels do that behavior.
Go to the next slide. However, behaviors that focus on information seeking
many of them are only done by the highest activated group. I am sorry to say
that is also true about quality information using quality information that it
is the most activated group that will seek out, know about, or use quality
information and we have seen this in multiple studies. And I think this has
implications for how we promote and use quality information and also thinking
about the importance of people becoming more active and activated.
Go to slide number 11. One of the things that we sort of gleaned from all of
this is that we can actually use activation levels to think about what is the
realistic action step for a patient. Many of the behaviors we are asking people
to do right now are kind of way beyond them. It is kind of like throwing
non-swimmers into the deep end of the pool. We are asking people to do things
that they are not ready to do.
But if we start with behaviors that are more feasible for people to take on,
we actually increase their opportunity to experience success and that is how
people gain confidence. At the low end of activation it is really a matter of
not having confidence that you can affect your health. The point is that this
is actionable information in a medical encounter.
Slide number 12 we needed to see if this was a changeable construct and we
did observe over a 6-month period we observed chronic disease patients and we
saw that some people did increase significantly in activation and those people
had statistically significant improvements in 11 of 18 behaviors. When
activation improved behaviors, multiple behaviors changed. Those who did not
increase or stayed stable or declined in their behaviors had the same
trajectory.
Slide 13 shows and 14 we look at how activation is related to outcomes. This
is slide 13 is a study by AARP that looked at people who recently had
hospitalization who had multiple chronic illnesses. And what they saw was that
people who were low activated had more than twice likely chance of being
readmitted to the hospital after they were discharged. They were also more
likely to experience a medical error. They had more chance of having poor care
coordination and having harm as a result of that poor care coordination. And
not surprisingly they were more likely to lose confidence in the system.
Activation is related to many of the outcomes we are very concerned about.
Slide 14 shows a study that was done by the care management institute at
Kaiser Permanente. They measured activation in chronic disease patients in 10
states and then went back and looked at — 2 years later they looked at the
records of the diabetic patients in that sample and they saw that activation
score was predictive of whether or not patients were adherent to their diabetic
testing, whether or not they had good glycemic control 2 years later, and
whether or not they had had a hospitalization in this period. The measure is
robust enough to predict future outcomes.
Slide 15 is a summary of a study that was done to actually intervene and try
and increase activation in an intervention. And a study in disease management
there was a comparison group. What we observed there was that activation
increased more in the intervention group than the group that was getting
coached in the usual way. The intervention group was coached tailored to their
level of activation and they had better clinical outcomes and reduced
utilization.
Slide number 16 just says what we know cross sectionally that those patients
who say that their provider helped them to learn to monitor, set up an exercise
program that those patients are more activated although we don’t know the
direction of causality there.
We can go to slide 17 to summarize here. We do have valid measures, measures
that are changeable and are sensitive to intervention. The activation measure
actually focuses attention on important outcome of care that currently goes
unmeasured.
The other thing that this does is it changes assumptions about what good
care entails for both the provider and the patient.
I went through that rather quickly but I want to see if people have comments
or questions.
DR. TANG: Great. Thank you very much Judy. Larry.
DR. GREEN: Judy, thanks. Are you in Oregon?
MS. HIBBARD: I am.
DR. GREEN: Thank you for joining us so early. Judy, I have overlapping
questions here that are all connected. I think you are telling us to one of
your answers to our question about what information can consumers need is that
they need to know their activation level.
Then going back to Paul’s point about stratification are you suggesting that
one way to stratify Eva’s dashboard is by activation level of patients?
MS. HIBBARD: I am actually saying that I think that what patients need to
know is that providers do a good job or don’t do a good job in supporting their
patients’ ability to self manage. It is like a measure, yes, on a dashboard,
but what does the provider’s panel look like. Are they gaining in activation as
compared to another provider? So they can see how really supportive are people
getting the help they need to do their job.
DR. GREEN: Got it.
DR. MIDDLETON: Good morning Judy and thanks for joining us. A terrific
presentation – extremely interesting. I guess the first question I have is
about the activation measure and what we heard from Eva Powell in the first
panel was that consumers are very concerned perhaps most concerned about costs
of care and access even though they have differential out of payment and the
outer payment payment only pays for part of health care costs, et cetera.
MS. HIBBARD: Can you repeat what you said?
DR. MIDDLETON: Can you hear me now?
MS. HIBBARD: Yes, I can hear you.
DR. MIDDLETON: Okay. We are doing a Verizon commercial. The question, Judy,
is about measure. What we heard from Eva Powell was that patients or consumers
were most concerned about cost and access, yet, reviewing the patient
activation measure 13 questions not one of them deals with access or cost. Do
you think those are independent measures of patient activation since Eva also
said patients who don’t have access or can’t pay for their care sometimes
become passive, et cetera? Is that an important thing to consider beyond the
patient activation measure or how does the patient activation measure consider
cost and access?
MS. HIBBARD: Good because I wanted to get into the cost question too. What
we have seen in studies is that while every aspect of health behaviors related
to activation cost is not. It is true that people are concerned about cost, but
I think that there is a danger in the way we talk to consumers about cost. And
that is that what is being — what is seen in public reports is often there is
just a measure of cost or efficiency that is just added on to quality
information.
When consumers do not understand quality which is the case with much of the
measures that are out there right now, they will use cost as a signal for
quality. Higher cost for many people will signal quality. How we communicate
that our cost is really that we have to be sensitive to that and understand
that.
One way to get around that is to provide cost information within quality
strata so people can see that they can still get high quality and not have to
pay top dollar for it. But they are activation and costs are really separate
issues. The activation measure focuses on really mean managing by health and
how confident that I can do this.
DR. MIDDLETON: And how about the access dimension or issue or is it the same
thing in your mind?
MS. HIBBARD: We haven’t explored that as much, but I do want to say one
thing about this whole area of what consumers care about. I think it is really
important. And that is that we have seen this over and over again. When you ask
people about an area that they know is quite important but it is complex and it
is unfamiliar territory, they will give you an answer about what is important
to them, but it is a very changeable idea because people’s ideas are fluid
right now. Their preference is about or their ideas about what is important
with quality can be changed just by the way you ask a question, what you
presented just before because people don’t know. When we say oh people care
about cost, of course they care about cost, but they care about other things
too when they understand them. My point is we can’t just go with what people
say is important right now because that is just too fluid and too changeable.
When people really understand the quality arena, their ideas will become more
fixed, but right now they are not.
DR. MIDDLETON: Just one more quickie if I may. The data you presented, for
example, on slide 13 showing that the less activated patients might experience
— do experience more medical error. I wonder if you could just talk through
that a little bit in terms of what is the biological plausibility or mechanism
of that.
MS. HIBBARD: Yes. Okay. Good. So what we have observed with the less
activated is they really don’t understand that they have to play a role. If you
can imagine being in the hospital, the activated patient will ask about their
medications, will be more involved in what is happening if they are able. But
the less activated patient assumes this is not their job and that everyone is
going to take care of them. That is why I think that they lose confidence
because their experiences don’t match their expectations.
DR. MIDDLETON: Okay. That is helpful. And last one for me is just on this
critical question of the predictive nature of the patient activation measure.
Are the data you are showing on slide 14, for example, from a prospective
control trial in some way that actually assesses the independent prediction
capability of a PAM on utilization or I am concerned that there may be
extraordinary, confounding that is that the higher activated providers or
health systems, if you will, are influencing patient activation of course the
other way. How do you sort that out?
MS. HIBBARD: This is all within Kaiser. They are all Kaiser patients in this
particular study. It was the survey that measured activation with them in 2004
and then in 2000 they went back and looked at the data. It is prospective but
they went back and looked at the data and pulled all of the records for the
people, diabetics who were in the survey and looked prospectively at how well
it predicted what happened over the next 2 years.
DR. MIDDLETON: I get that sort of a historical prospective methodology, but
how well do you —
MS. HIBBARD: But there was no intervention except the usual care for
everyone.
DR. MIDDLETON: And do you have a sense of what factors they controlled for
or what interactions they might have controlled for?
MS. HIBBARD: They did control for multivariate — I am trying to read the
small print here. Age, gender, race, comorbidities, and number of
diabetic-related prescriptions.
DR. MIDDLETON: Okay. So if it is actually the case that the PAM is
predictive in a prospective way, should we make it — I can’t remember what
number of vital sign we are up to, but if we have —
MS. HIBBARD: I do think it is a vital sign. It tells you about the patient’s
ability to do the job you are asking them to do. That is to go home and change
the way that they eat or exercise or take their medications.
DR. MIDDLETON: Thank you. That is very helpful.
DR. TANG: This is Paul Tang. I wonder if on the 13-item PAM, how is that
scored? Is there just a point for yes? Is it a yes/no and one point assigned
for each of these?
MS. HIBBARD: No. It is actually based on the RASCH analysis. It is a simple
summative score divided by the number of questions. And then if you don’t have
the ability to run it through RASCH then we do have like a score table. The 0
to 100 scale is what it produces. And we have seen it just changes in like a
few points are meaningful in terms of behaviors and outcomes.
DR. TANG: Let me check an understanding I think you said. If you have a PAM
score at one point in your life, you know that people with these scores behave
in certain ways or don’t. If the same individual changes their PAM score at a
later point in life, do they then adopt the behaviors of the other folks who
were in that Pam score at baseline?
MS. HIBBARD: Yes. People do but you know like so in the study where we
measured 18 behaviors and people changed 11 of them, but we don’t which 11. The
idea is that and people start to feel more in control. They are going to do
multiple things different not just like one thing different.
DR. TANG: And then your final response to one of Blackford’s questions. You
talked about how the PAM score sort of assesses whether an individual would
“do the things they are asked to do by their health care
professional.” And I am wondering whether you could look at it the other
way around. Should we be actually taking into account both their PAM score and
what Eva was talking about the social determinants in health when you “ask
them to do something”? Do you see what I am saying? It is sort of the flip
side of that.
MS. HIBBARD: Yes. That is right. That is the whole idea of thinking about
sort of setting someone up for success instead of failure is ask them to do
something at least an initial step. What we have observed is that people —
their motivation kicks in when they start to experience a little bit of
success. They feel more motivated when they think they can actually do this.
Yes, asking them to do something that is realistic and then sort of thinking
about it more as a journey or teaching someone to swim. You don’t ask them to
swim laps right away if they don’t know how to float. You start out smaller and
kind of build to the skill base.
DR. TANG: As our lesson to you it sounds like one’s activation is more
malleable and there must be some kind of constraints on that than let’s say
contrast that to your cultural background. You have a certain culture. You have
a certain belief system. It is not as if we should be changing their belief
system. We, the health care professionals, should be working to develop a care
plan or health plan that takes into account their belief system, their cultural
background. You are saying the opposite in PAM, not quite the opposite, but in
addition to taking that into account you believe you can change one’s
activation over time.
MS. HIBBARD: Yes. We have seen it over and over again. Yes, you can. And
sometimes it is as simple as telling someone who has a big long list of things
that they need to do don’t do all right now. Just work on this one thing. And
the one thing should be what they want to work on because that is where there
is the most motivation. Now when they start to see they can actually do this
then their motivation increases. Their abilities increase to step up to the
next challenge.
We have seen people increase 8, 10 points in a month just because doing
something like giving them permission to do one thing. It is very freeing for
the individual. If you feel like you can’t do everything, often you just do
nothing.
DR. TANG: Great. Thank you.
MS. POWELL: Thanks. I apologize. Judy, thank you so much for your
presentation. I agree with everything you said. I just wanted to add something
quickly that I should have said particularly in reference to Larry’s question
about cost. The question was specific to out-of-pocket costs and I answered
that. Yes, consumers are concerned about their out-of-pocket cost. But what I
should have said at the time is that cost as we talk about that in reference to
consumers and to consumers perhaps is better termed burden and I think, Judy,
you were getting at this because cost to consumers — out-of-pocket cost is
just one of those costs. There are also costs in time. There is cost in work
hours and income and potential loss of income if there is an absence from work
too often. A cost of attention. There are multiple costs which really add up to
burden which perhaps is the better way to talk about this with patients.
And I wondered, Judy, if those things are linked to activation particularly
as it gets into some of the kind of other life demands kind of costs. Have you
all looked at that?
MS. HIBBARD: No, we haven’t. Only like actual money costs. But that is an
interesting question. I don’t know. One of the things that we have observed
that is relevant is that the more activated are actually — have better
experiences in care and are more satisfied. And I think that is because they
know how to get what they need and we have seen also in the study for health
systems change that use the PAM that even after you control for income and
insurance status, the low activated are much more likely to have unmet medical
needs. And I think that is because they come up against a problem, a barrier
and they give up. It is all about people’s sense of being able to get what they
need and their skills in doing so.
DR. TANG: A final question, Mike, if you could keep it short. We are running
out of time.
DR. FITZMAURICE: Sure. I have a good short one. Is PAM related to patient
outcome? You presented some evidence that it seemed so at least. Blackford
mentioned about should this be another vital sign which triggers in me a
thought. And by the way, thank you for a very thought-provoking presentation. I
wasn’t provoked but my mind certainly was provoked.
My question is if this is related patient outcomes then should the provider
be held accountable for his or her patients achieving a higher score. Is it
something like patient exercise where if I exercise more, I am going to be
healthier and be better, but yet it is my own motivation and maybe not my
doctors? Is the lack of exercise the provider’s fault? Is the lack of
progression in a patient’s activation somehow is the provider accountable for
that or is it something —
MS. HIBBARD: That is a very good question. And we are actually in a study
right now where we are linking patient PAM scores with a score with a continual
measure for clinicians about their beliefs in the — basically their support
for the patient role. What we are seeing is that they are linked. At least
statistically that those clinicians who are more supportive their patients have
higher scores. Now we need to look at that longitudinally and we need to see if
physicians do make a difference, but I believe that we are going to see that
physicians can make a difference and what we are going to see is that people —
that their panels look different in terms of their activation profile and in
which case I would say yes.
DR. TANG: Thank you, Judy. This has been very rich and provocative and
interesting. I think it feeds in very well with this discussion we are having.
Gene, I think you are up next and we will give you the full time.
DR. NELSON: Thank you. Am I coming through okay by phone?
DR. TANG: You sure are. Thank you very much.
DR. NELSON: Great. Are my slides ready to —
DR. TANG: Yes, they are too and I will just give a quick introduction saying
that Gene is the Director of Population Health Measurement Program at Dartmouth
Institute and Director of the Population Health and Measurement.
DR. NELSON: Thank you. The first question that I was asked to comment upon
is what data and measures will consumers need to be successful patients and
purchasers of care. The short answer is they will need a triple aim compass
with feed forward and feedback data. And I hope by the end of my brief remarks
that you will have a sense of what that means.
Amory Lovins, the chief scientist for the Rocky Mountain Institute, has a
question. And it is how the kilowatt-hour of electricity is like a day in the
hospital. And the answer is nobody wants either. What do consumers want? Hot
showers and cold beer. What do patients want? Better health, better care and
lower costs.
What I will do briefly is to focus in on a few questions that consumers need
data to answer, a few of the critical questions like, first, can I get the care
I want and need exactly when I want and need it. Second, am I getting the vest
value care for me given who I am and what health care can do? Third, how can I
get the data I need as care is provided, to make good decisions and to review
the success of the care that I have gotten?
What I will do is provide you with a couple of case studies to put these
questions into context and to show how this might work. First, can I get the
care I want and need exactly when I want and need it?
Go to slide five please. And this is Amy. We have Amy’s permission to tell
her story. This is a true story. Amy, at the time that I met her, was a
38-year-old single woman, new to the area teaching school for disabled
children. And she found a lump in her breast.
Amy at the invitation of Dr. Dale Collins who runs our comprehensive breast
cancer program was accompanied by Leslie Lamb, one of my quality fellows, to
just find out what Amy’s journey would be like for the first 6 months of her
care.
This slide six portrays the 21 different visits made in the first 6 months
to 14 different frontline clinical Microsystems for her first 6 months of care.
And Amy would tell you and she told the comprehensive breast cancer team as
well that at times her care was delightful. The wig lady. The first time she
met with her oncologist, Dr. Pam Ely. Sometimes her care was dreadful. For
example, having to have stat lab that weren’t stat and actually her first
meeting with the surgeon that did not go well.
If we step back and we look at Amy’s care from a distance, we have Amy who
is a person, a patient who is part of a larger population of patients: women
with newly diagnosed breast cancer. And they entered into our health care
system and they start a journey in health care in hopes of getting the best
results possible. There are tests. There are diagnoses. There are treatments.
There are decisions to be made. There are often shared decisions that are made
based on preferences and best available evidence.
And then we can pop out to the right side to say what were the outcomes at 1
month, at 6 months, at 1 year, at 2 years, at 5 years. What were the health
outcomes? What were the experiences that the person had and what were the costs
that evolved over time?
So the next slide eight, Amy and people like her, want data to answer key
questions as the care progresses over time. What are my chances of surviving?
What are the risks of different treatments? How will I feel physically and
mentally? Will I be able to get back to work to teach my children at school?
What can I do to help myself to get the best results that I can get? Will I be
treated the way I would like to be? Can I afford the best treatments that I
want and need?
Slide nine then puts these questions into the form of a compass that Amy was
in a sense, using in the early part of her care journeys. She was concerned
about mortality, the risk of death, and the opportunities for survival. She was
concerned about the risk of recurrence, the north point. She was concerned
about the morbidity and the complications associated with different treatment
options. Going north again, she was concerned about how she would feel
physically and mentally. Could she go back to being a teacher? Would she be
tired out or would she have vitality?
What were her care experiences? What was her level of activation as her care
evolved? What were her experiences with the delivery system? Amy had good
insurance. She wasn’t worried about her direct medical care costs. She was
concerned about being out of school, substitute teachers doing perhaps not as
well what she needed to do and understanding her community was paying for that.
Here are two more cases that I will use. I will use Brian and Betty and the
context being the Dartmouth Spine Center. Brian and Betty are fictitious cases
but absolutely typical. Brian, a 45-year-old man with a herniated disk and good
insurance. Betty, a 64-year-old woman with spinal stenosis, no insurance,
single woman, lost her job.
The context being the Dartmouth Spine Center where the motto is back to
work, back to play one person at a time and to achieve this in a way that is
patient centered. The idea is to bring together all the disciplines that are
needed to care for people with spine problems, to engage in shared decision
making, to use outcomes tracking at the individual patient and the clinical
population level and to use the measures that accumulate over time on outcomes
and costs and experience for collaborative learning and research.
Slide 12 shows that when the spine center was open, an idea that was novel
was to create a very rich information environment using feed forward patient
reported data to help inform what the initial plan of care would be and to
evaluate how that plan of care is working for the patient; and then feeding
back that data to the patient and clinical team over time to monitor progress,
to refine treatment regiments, as well as to aggregate the data for
collaborative learning in the spine network and for an NIH trial on back
surgery.
Slide 13 goes back to Brian and Betty. Brian with the herniated disk, and
Betty with the spinal stenosis, and people like them have key questions. What
are my chances of becoming pain free? What are the risks and benefits of
different treatments? How will I feel physically and mentally? How fast will I
be able to get back to work and back to my life? What can I do to help myself
to get the best results? Will I be treated the way I would like to be? Can I
afford the best treatments that I want and need?
In slide 14 you will see an image of how the patient-based feed forward data
works that the individual when they make a visit to the spine center can get a
touch pad or they can come in over the Internet and in a secure portal and they
complete a health questionnaire about how they are doing.
On the next slide, slide 15, creates a summary report that is produced each
time the individual makes a visit to the spine center. If you could see the
details of this summary report, you would see that you are putting critical
information about the patient all on one page for the patient and the clinician
or the clinical team to work on together: the health history, the health
habits, any red flag considerations.
And then to the right you will see that this particular person had six
visits over a 19-month period. On the right hand side you can actually see that
the physical function was improving from rather low levels to getting near the
national average. And perhaps surprisingly the mental health was drifting
downward causing a red flag to be concerned about loss in mental health.
The bottom panel shows the disease status as measured by the Oswestry
Disability Index, and lower numbers are better. This person’s disease status
was decreasing over time.
The bottom left has the patient’s expectations met or not, for example,
symptom relief. Definitely yes at 19 months. Sleep better. Probably yes at 19
months. Dr. Weinstein, the founder of the spine center, would say I can’t be a
good doctor without this data. I won’t have information I need to formulate the
treatment plan with the patient or to judge the goodness of the treatment plan
for that patient.
The next slide then, slide 16, gives you an image of a compass for Brian,
upper left, herniated disk and for Betty to learn about, upper right. And this
was data that is now given to patients in the form of a calculator customized
to them at the plan of care and also reinterpreted at the Center for Shared
Decision Making. What Brian would learn about would be that if we look at the
compass on the left herniated disk, people that got surgery the improvement in
physical function, north, was 44. People that did not get surgery, this was a
randomized controlled trial of surgery, not surgery, 13 centers, the gain was
very substantial, but less, 32 points on the SS36 physical component score.
If we go west, we see the disease burden dropped 38 points for surgery, 24
points for nonsurgery, both big drops, but again the surgery is better. If we
asked the person at 2 years, these are data at 2 years of follow up for asking
these patients at 2 years how much they were helped by their care 76 percent of
the surgical patients felt they helped a great deal, 58 percent of the
nonsurgical patients.
And we go south and we can see there was a cost involved that was greater
for surgery. These are direct and indirect costs, medical care as well as
indirect social cost on average, $25,000 versus $10,195. That is what Brian
would learn and would be able to make a decision based other people’s prior
results and his own preferences.
Betty, if you go to the far right would learn about this. This is our people
with stenosis. Betty is about 64 years old. The average person with stenosis is
about mid-60s. But you can see that the blue and yellow bars on clinical status
changes, functional status changes, and how much the person perceived they were
helped by treatment were pretty close and the cost were different, more for
surgery, $26,000 versus $13,000. Betty does not have insurance so she would be
able to take this information into account in making a decision.
If you will go to the next slide, then wrapping up with recommendations. Go
ahead to slide 18. The belief here is that people will need a compass to
navigate the health care system. People or consumers or patients are on a
journey and they are trying to regain or maintain their health. Their health
care needs evolve as their health status changes and as treatments work or do
not work and therefore patients need data to answer the key questions about
health and about their care experiences and about costs as their conditions
change. They need data on health status, their own, and likely outcomes given
the database on prior patients like them to match the best care plan to health
state and to monitor the impact of the care plan on outcomes, and to make
informed decisions about the best treatment options for me given my
preferences. They will need feed forward and feedback data on care experiences
of others like themselves and on costs.
Slide 19 goes back to this initial thought that my triple aim compass as an
individual in a point in time will in fact evolve over time as my health and
health needs change. The cardinal points on the compass needed to navigate the
care system, health outcomes, disease status, functional status, risk status,
care experiences, activation, the characteristics of care, safe, timely,
effective, efficient, equitable patient centered the care experience, perceived
health benefit, how much was I helped. And then the last major point being
costs of care, direct medical care costs especially out-of-pocket costs as was
noted earlier as well as indirect social costs related to not being able to
work or somebody else having to serve as the care giver, et cetera.
The next slide, slide 20, suggests that we will need a longitudinal
measurement framework that connects the patient’s experiences over time with
data and measures. And here you have an image of the NQF patient-focused
longitudinal measurement framework with some of the triple aim compass points
underneath. The logic is that we have people at risk and some of those people
will move to the right with an initial onset of a disease or a problem such as
an acute myocardial infarction or the need for total joint replacement or newly
diagnosed cancer, and then follow-up care over time, months, years often times
rehabilitation, self care, et cetera. And that we would like to be able to
connect the measures of risk function of disease of experiencing cost to people
as they move through the progression longitudinally.
The last slide, the conclusion then, is it goes with Wayne Gretzky who was
asked how he could score so many goals and he answered that he always tried to
skate to where the puck is going to be. If we think out 3 to 5 years and there
is a informal/formal Gretzky that has been working this with NQF and many
others involved that it would be helpful for us to adopt the longitudinal
measures framework, and to populate with triple aim measures, and to create the
mechanisms to feed forward the data especially patient-reported data or
consumer-reported data as care is given to patients, and to feedback the data
on the outcomes of populations of similar patients to support decision making
and care program improvement and patient-centered research.
DR. TANG: Excellent Gene. Thank you very much. This has been truly an
outstanding panel. Let me open it up for some questions.
DR. GREEN: I want to go back to the question I want to ask Eva and also Judy
and Gene ask you to chime in on this. After such a splendid presentation, this
is going to sound like such a mundane question. What can the three of you tell
us about the consumer’s quantitative skill set? Can consumers do percents?
MS. HIBBARD: About half of Americans are what we would say are innumerate;
that is, they have difficulty deriving meaning from numbers. Putting meaning on
numbers for people is kind of what we need to do.
DR. MIDDLETON: Gene, this is Blackford Middleton. A question for you.
Looking at the compasses on slide 16, I thought this was a terrific and very
readily accessible, graphical representation of a whole bunch of data, so sort
of you know visual representation tough T award for you. I guess what I don’t
understand from a clinical perspective is when you show the map like this to
Brian, this compass, what is the conversation and ultimately how are his
utilities or preferences incorporated into this decision-making process?
DR. NELSON: That is a great question and I will also loop back to what Larry
just asked. To help an individual make sense out of this kind of data it helps
to have several different approaches to customize them to the individual. There
is a Center for Shared Decision Making where a person can go to discuss these
kinds of information with a live person to see video tapes of people like them
who have had surgery or not surgery for herniated disks and graphic
representations that are more first patient lay-person friendly.
For example, a way of interpreting Brian’s compass might be that for people
that got surgery 24 out of 100 achieved substantial benefit in terms of their
pain or in terms of their physical function. Twenty-one out of 100 that didn’t
get surgery 2 years later had similar results.
We work with people who are experts at helping non-numerative people
interpret this kind of information. And one of those good approaches is people
like you X number out of 100 would benefit versus Y number out of 100.
DR. MIDDLETON: The tailoring for Brian has already occurred and sort of
taken the evidence base and customizing it to his specific condition or to what
degree has that occurred?
DR. NELSON: It occurs to the conversation with the clinician or clinical
team or at the Center for Shared Decision Making or with the use of the risk
calculator that I mentioned.
DR. MIDDLETON: Thank you.
DR. TANG: Any other questions? Mike.
DR. FITZMAURICE: Gene, this is Mike Fitzmaurice. Again, an excellent
presentation and thank you for your time. In looking at this I see sometimes
there is a sequence of a sequence of people who might want to do none surgery
for a while and then they turn to surgery particularly with back functions. Is
there another chart that says these patients put it off for 2 years and then
they went to surgery and they did — 35 of them had a good result as opposed to
44 who went to surgery right away and 21 who never went to surgery? Is there a
sequence like that that should be investigated?
DR. NELSON: There is not, but it would be very good if it were to be very
helpful.
DR. TANG: Thanks again, Gene. If there are any overall questions for all
three panelists, we could entertain a couple of those now.
DR. NELSON: If I might make a comment on the issue of value for the
individual. One way of thinking about it is the health outcomes that they
achieve and the care experiences that they have experienced in relationship to
their costs, the cost that they pay out of pocket as well as the cost paid on
their behalf. I think it is possible to develop a value equation that can work
well for an individual as well as for a defined population of people by taking
into account outcomes and experiences in relationship to cost.
DR. GREEN: Gene, this is Larry again. Let me ask you another question. When
I heard you talk about in your location and with your examples that the person
then goes to this center. They go to a center and get help and then
interpreting the information and measures that they needed to execute their
functions as purchasers and stewards of health care. Are you implying,
suggesting, recommending any of those verbs that we need to start planning on
having such centers scattered all across the United States for everyone? And
part two, are you assuming that consumers are on their own here that they are
not going to have a doctor? They just got to figure this out on their own.
DR. NELSON: Great questions. I think the first recommendation about shared
decision making is to enable shared decision making in multiple ways and
multiple venues so that it can take place as part of the clinical team,
clinician, patient interaction. You may have a real center like a Center for
Shared Decision Making at the Dartmouth-Hitchcock Center or the VA. You might
have a virtual center for shared decision making.
If you think, of course, about what the work that the Foundation for
Informed Medical Decision Making is doing and what others are doing to bring
together health information enabled answers to critical questions and data and
stories and videos and peer to peer as well as professional to peer support.
You can imagine having in 5 years some very excellent virtual centers for
shared decision making using social networking software using health
information, intelligence technology, using knowledge management as part of
that.
You can imagine if you look at Clay Christensen’s work and Innovator’s
Prescription and the model that I am speaking about is the distributed
model where professional knowledge and people that have a shared concern come
together virtually through a rich, Internet-enabled information environment.
MS. POWELL: This is Eva. To follow on that point about the centers I think
another element of this — I kind of see this as a two-pronged issue. One is
developing the knowledge base around shared decision making and the tools which
some of the entities already mentioned have done some great on and will
continue. But the other element of this is knowing how to use those tools and
that is a skill set that really needs to be incorporated in all of our clinical
training whether that be medical schools or other professional schools because
really this is an issue that each member of the care team needs to have some
knowledge of how to do. Of course the shared decision-making process is driven
by the physician, but then there are others who have input into those decisions
and into the decision-making process. It shouldn’t be physician exclusive I
guess I should say. I don’t think we can underestimate the value of that
particularly as we are talking about making these changes in the current
environment that really does not support this very well. That is something that
we just need to begin working on as well.
DR. TANG: Great. Thank you very much again to the panelists for an
outstanding panel this morning. And we will break for lunch and resume at 1
o’clock. Thank you.
A F T E R N O O N S E S S I O N
Agenda Item: Panel II – What Information and
Measures to Providers need to Improve Quality and Increase
Accountability?
DR. TANG: This is going to be the second part of our today’s hearings and
then we will have a final one tomorrow morning before we have some subcommittee
deliberations. We had a wonderful panel this morning on the consumer
perspective on information that they would find useful that would address the
— let me use the quality word it is really beyond that, really the whole
quality and meaningful measures that affect their health care decisions and
health decisions. This afternoon we will start off with a panel that has the
same question about what information and what measures would help the health
care professional address their needs to, one, understand the quality of the
care they provide as well as to increase their accountability.
We have Dr. Fred Rachman, Yael Harris, Theresa Cullen and Karen Kmetik. We
will start out with Dr. Fred Rachman from the Alliance of Chicago Community
Health Services. And he is on the phone I believe.
DR. RACHMAN: Hello?
DR. TANG: Why don’t you give us a few more words just so we can adjust the
volume?
DR. RACHMAN: Okay. Is this any better?
DR. TANG: Is there any way you could either get closer or situated
differently because it is a little faint?
DR. RACHMAN: Let me just try one thing — disconnected I will call right
back in. Hello?
DR. TANG: That is certainly better. It is better. It is still a bit faint.
DR. RACHMAN: I am not sure what I can do on this end. I am speaking directly
into a hand set on a land line.
DR. TANG: Is there anybody else on the phone? I am wondering whether anybody
on the phone can hear me better.
MS. KMETIK: Hi. This is Karen Kmetik. I can hear Fred very well.
DR. TANG: We can hear Karen better than Fred.
DR. RACHMAN: How about if I shout? Is that better?
DR. TANG: Let’s do it for now. We will give it a try.
DR. RACHMAN: I don’t mean to shout at you all but we will try that. Did you
receive the slides I sent fairly late?
DR. TANG: Yes.
DR. RACHMAN: Great. I will just try to walk through really briefly. I wasn’t
sure if folks needed any background on the community health centers because it
is largely from that frame that I am speaking. I am assuming that most of you
have some basic background, but just to highlight that there are currently more
than 1100 federally-funded community health centers and the health reform
package calls for tremendous growth in this sector. And it is certainly a major
linchpin in the provision of primary health care and not only that has a model
of health care that really encompasses much of what we are contemplating in
terms of concepts of medical home and a focus on quality, primary and
preventive health care. I just wanted to make sure that folks knew that. It is
really the claim from which I am speaking.
I am going to skip ahead to the third slide that really talks about the role
of health centers and health information technology. Not only is there this
background that the health centers have in primary care and medical homes, but
there is a tremendous focus and body of experience working with chronic disease
management. Informal programs that incorporate pushing out evidence-based
practice recommendations and then a set of reporting on quality and outcomes
measures related to those evidence-based practice guidelines.
In addition to that there has been now for almost a decade an investment by
HRSA in funding clinical information systems evolving to electronic health
record systems in this setting as well. I think these are interesting facts
about health centers from which to contemplate the questions you have put in
front of us today.
Should I press on or do you want me to try to call back in and perhaps have
another speaker go?
I would like you to move if you would to slide five. Just to put as
background that these health centers again to reinforce that the health centers
have been now reporting on multiple quality measures that are imposed upon
them. First of all, HRSA has a set of measures attached to community health
centers. There are measures attached to these. There are chronic disease
programs that I mentioned. Many of the centers receive Ryan White HIV funding
which have both core measures to that funding program, may also have state
measures, and also have quality measures for centers choosing to participate in
quality improvement programs attached to Ryan White.
Different third party payer measures that all of the health care world is
subject to now both private insurance and state Medicaid measures as well as
individual funding that centers may receive around specific grant programs or
other projects.
Just a little word about our organization, Alliance. Next slide. We are what
are called a HRSA health center controlled network. We are funded to provide a
common infrastructure bridging multiple health centers and our focus has been
on health information technology. Early on we were happy to partner with
Karen’s group at the AMA to look at layering incorporating clinical practice
guidelines and quality measures into a commercial EMR, and having had some
success at that we are now providing a centrally hosted solution with common
data elements, common clinical decision support and common performance
reporting that is spanning 28 health centers operating as 100 delivery sites
nationally. Again, that is the background from which I would like to share some
perspectives to you.
It is going to be hard perhaps for you to see this on paper, but if you look
at slide 10 this is an example of a clinical decision support screen that is in
our system that marries a practice guideline recommendation to actual
structured data entry and reporting that is then used if you skip ahead two
slides to be able to publish clinical outcome measures comparing practice
sites, health centers, providers could be sliced by health disparity groups
against those measures to aid in health disparities’ work and quality
improvement work.
What I would say from this work is it has given some insight to us about
some of the challenges, first of all, against multiple practices if you look at
the next slide. Underneath there is a huge change management process to get
providers to accept common practice guidelines and to adhere to standardized
ways of recording data. And then there must be strategies to make sure that in
fact the data of capture methods as designed are actually able to be carried
out.
Against that the next two slides really just talk and this is some
background you, just some of the challenges that we have as a nation in
implementing health information technology.
I would like to really focus on slide 15 which are really some
considerations as a jumping off point that I would like to share with you.
First of all, following from the multiple measures that our centers are
required to report on and I would say that is a proxy for what is happening to
all kinds of practice sites out there. There is definitely a need for alignment
of measures so that the complexity of data capture all the way through to
acting on date is simplified by not having so many multiple variations on the
same content.
The second is and this should be obvious is that we are only working with
specified measures that are set up to be collected in electronic format, that
are validated and have been tested so that we are sure we are putting this
effort to good use.
The third is acknowledging that there are complexities of data capture and I
will come to this in a minute, but some data can get in very simply but others
require very, very complicated work flow considerations in order to capture
accurately. And if we are going to try to use and interpret this data, it is
important to acknowledge that.
Certainly we all talk about health disparities, but if we want to use any of
these measures for health disparities, we are going to have to be very
thoughtful about the accuracy and consistency with which we are capturing
information about the disparities groups, who we are including, how we are
defining them.
From a health center model we are very aware that more than medical care is
important in global health of an individual and we really believe that this is
part of the reason we have had such disappointing outcomes in house. The pitch
that we broaden are consideration of what is included in health and quality
measures to include other parts of the health care system; things like
behavioral health, case management, other social, sociocultural data on
patients that may be necessary in order to make these measures meaningful.
Thinking about those levels of accountability. Each time we think of quality
measure thinking at what level that is actionable because the goal of this
should really be around driving improvement and reduction of disparities not
just measuring performance.
Just a couple of last slides and then I will finish up. I know I am probably
close on time. I would like you to look at the pyramid slide which I am
guessing if you are not looking at it as a presentation is much too small to
really look at. But it really is just meant to reflect the hierarchy of how
data gets in. At the bottom we have some relatively simple data. We are having
a direct electronic data element, something such as a lab result where if the
— this kind of a big if but if the interface is correct and the order is being
captured, that data element is seamlessly captured and available for quality
reporting just in the care process.
And all the way at the other end of the spectrum is some kind of quality
result where we need information coming in from outside of the particular
institution in which the practitioner is based and is now a very, very complex
health information exchange or continuity process. And measures like that which
may be among the most valuable for us from a system perspective pose tremendous
complexity for us to measure.
The next consideration if you go back a slide is looking at how we are
defining the measures. Really from a practice standpoint we really want to be
pushing to practices, evidence-based practice guidelines. And we want to be
using electronic systems to prompt decision making at point of care around
those guidelines. But in fact in many cases where the practice is being called
upon to report a quality measure, which may be less intense than what the
actual evidence-based guideline requires, we may actually be having in some
cases decision support that is around the grade or around the performance
measures rather than going a little bit deeper to the guideline itself.
And then finally just punching up that point if you look at the next to last
slide with the overlapping circles, where do the quality measures come from?
Are they system level measures that are coming through a health information
exchange? Are they coming from the EHRS which is provider or institution based
or should we be looking at some source of data that is coming actually around
the patient marrying multiple sources?
The last slide I want to leave us with is just this old thought on the
performance improvement world which says that the real aim in measuring quality
is not to tell us how we are doing, but to tell us what is important. I would
pitch that is we are structuring quality measures for ourselves that we want to
have measures that communicate to everyone where we should be focusing in
involving our health care system rather than judging performance. Thanks.
DR. TANG: (Inaudible)
DR. RACHMAN: Nothing changed on my end.
DR. TANG: (Inaudible)
DR. MIDDLETON: Dr. Rachman, Blackford Middleton. Really loved your
presentation even despite the challenges we had on our end listening to it. I
think it is terrific work you are doing and I was particularly impressed with
the dashboard ideas and the sort of shared decision support ideas I understand
you are doing. I think you skipped over the demonstration slide, but that looks
like a GE Office Centricity product, is that right?
DR. RACHMAN: That is correct.
DR. MIDDLETON: And what are they actually receiving from the Alliance. Is it
a decision support exchange or can you explain that in more detail?
DR. RACHMAN: Sure. I wish I could be there in person and show you
dynamically, but what I endeavored to show was a sample of a screen within the
EMR that a practitioner can pull up that summarizes for them all of the
practice recommendations, evidence-based recommendations around a particular
chronic disease. In this example it is diabetes. And can snapshot very easily
for the provider where the patient sits with regards to that recommendation. It
is pulling from other parts of that same visit and encounters any activities
that have currently been done or have done in the past and without going into
other order screens makes it easy for the practitioner to address those care
recommendations. It is an attempt to marry the work flow at point of care to an
opportunity to review in an organized fashion a particular performance
recommendations or practice recommendations and quality measures.
DR. MIDDLETON: That screen design and its construction based upon on the
guideline information, is that being —
DR. RACHMAN: I am hoping I was audible.
DR. MIDDLETON: We heard you loud and clear. Thank you. Can you hear me okay?
DR. RACHMAN: Hello?
DR. MIDDLETON: Can you hear us now?
DR. RACHMAN: I can’t hear you. Do you think it would be worth my trying to
call back in?
PARTICIPANT: Can you hear us now?
DR. RACHMAN: I can hear you perfectly.
PARTICIPANT: It may have been the mic.
DR. MIDDLETON: It is Blackford Middleton again. Can you hear me now?
DR. RACHMAN: I can hear you perfectly.
DR. MIDDLETON: Verizon is getting a nickel every time I say that. In terms
of this screen and the decision support you are providing, do I understand
correctly? Is that remotely provided by your community or is that something
that the user of this application in their clinic is doing?
DR. RACHMAN: I am not sure I understand the question. We have centrally
designed this screen. All of the health centers that are using the application
access the application remotely. It is housed in a level three data center. But
all are using the same clinical content and data capture methods. Does that
answer your question?
DR. MIDDLETON: Yes, it does. Thank you. In terms of governance around
arriving at those common data definitions and arriving at a common implemented
specification of that diabetes guideline, for example, what is the — can you
describe the process at which you arrive at that consensus? What is the
governance mechanism there?
DR. RACHMAN: We have a clinical committee that meets and reviews content and
guidelines. However, we made the decision using the Harvard Getting to Yes
principles that we would base our clinical decision support on nationally
established guidelines. Again, for the one challenge that we have is that there
is not necessarily consensus on all of the guidelines, but we very much
appreciate the work that the AMA has done to try to drive consistent set of
measures and drive consensus. I am sure Karen will be talking about that. And
in large part we have adopted those measures where they exist.
DR. MIDDLETON: Thank you. And my last question then is to make that
dashboard, the Alliance total reports with all the different individual
measures lined up and the stoplight summary. It shows in the prior slide that
data is returned to a data warehouse in a de-identified form for you to report
on. Is that right?
DR. RACHMAN: That is correct and this is because of our — this is a
technology that our commercial vendor happens to have. It is one of the reasons
that when we began this work 8 years ago, we chose that product. Our hope
certainly is through the certification standards that we are living into an age
where all EMR products will have that capability baked on, but currently most
EMR products are designed as transactional databases around single patients and
very tremendously in their ability to report at this population level native to
the EMR.
DR. MIDDLETON: Thank you.
DR. TANG: I have a question. Paul Tang. Follow up on Blackford’s question
about the use of standardized measures. You said that you followed the national
guidelines. The national clinical guidelines are often different or let’s put
it the other way around. The standard quality measures are often different from
the guidelines, the clinical guidelines. Let’s take diabetes, for example,
where the guidelines still say to manage a diabetic to an A1c of less than 7
percent. The measure is the percent of folks who are over 9 percent. When you
say you adopt the national guidelines in your measure, how would you reconcile
those two, the difference between the quality measure and the clinical
guideline?
Are you still there or can you hear us?
MS. KMETIK: Fred, if you can hear me, it is Karen. We cannot hear you at
all.
DR. RACHMAN: Was that a question for Paul Tang or for me?
DR. TANG: It was from Paul Tang to you.
DR. RACHMAN: I am sorry. Yes, that is what I tried to attempt on that slide
and I get got confusing with my comment about the AMA. Yes, you are absolutely
correct. What we prompt in the EMR is against the measure although that may be
different from the underlying guideline. It is really a question that I am
raising is what should we really be prompting again. Should we be prompt
against the guideline or a measure? And if we are trying to prompt against the
guideline, it becomes even more — that process of driving consensus becomes a
lot more complex I think as you were probably getting at in your question.
DR. TANG: Correct. I think Matt wants to follow up and Mike Fitzmaurice
wants to.
MR. QUINN: Hi Dr. Rachman. This is Matt Quinn from AHRQ. Thank you for your
presentation. I really enjoy looking at this diamond that you put together and
calling out what is necessary for these different types of measurement
activities and where the data is coming from and what needs to occur to support
that. Looking forward 3 to 5 years do you think that the approach for us to
measure what we need to measure what is most critical is sequentially starting
from the bottom going to the top or where would you prioritize in focusing our
efforts?
DR. RACHMAN: Again, my attempts of saying that we should be measuring
quality to tell people what is important and not how they are doing I guess
what I meant to convey by that is I think right now at this point in time we
should decide how do we want to focus this system in terms of getting us to
where we want to be 10 years from now. And I think it is totally fine to have
measures which are more complex to collect as long as we are certain that we
are interpreting that data with caution and we are realizing that what we might
be measuring is our inability to capture the information as much as we are
measuring performance. I think some of the nervousness about promulgating
quality measures comes from providers who are fearful that they are going to be
penalized for poor performance and that leads us to dumb down our measures.
I guess my hope is that that pyramid is something we keep in mind and we
caution our payers or our funders to be mindful about how we are putting these
measures forward.
DR. TANG: Thank you. Mike.
DR. FITZMAURICE: Yes, I also enjoyed your presentation very much. I have a
set of questions. Let me just give you the questions and you can choose which
ones you want to answer and which ones you don’t. Is it hard to get community
health centers to capture and report valid and accurate quality measures? Is
there much extra man power involved? What is the response of community health
centers seeing their own quality measure results compared with other community
health centers? And do patients get to see the dashboard? Is it posted
periodically where they can see it?
DR. RACHMAN: Great questions. I will try to do all of them. Is it hard? I
would say it is no harder in community health centers than in any practice
setting. One of my roles now is as co-director of the Chicago Health
Information Technology Regional Extension Center and as you know through
meaningful use we are trying to push this out all the way through to small,
private practices and I would say it is probably even more challenging in those
setting than in community health centers. But there certainly are a number of
challenges. I tried to capture some of those on the slides which maybe I didn’t
cover or my voice cut out. Yes, it is very intensive in terms of resources and
labor. I do not think individual centers or individual practices are equipped
to do this.
And I also really hoped I made the point that we can make it a little bit
easier by being more thoughtful about how we select the quality measures and
trying to drive some alignment so that we are at least simplifying the field
there.
What is the results — I hope I am not skipping questions here, but what are
the — how do health centers react about comparing data? Well, one of the
reasons I am thrilled to work in community health center world is that we have
a culture of collaboration and sharing and typically what happens is when
health centers see an area where another center is performing better, they seek
that center out and start asking questions relentlessly to try to figure out if
there are some lessons learned there that can be applied. That by in large has
been the response we have seen.
And then your question on whether we are making them available to patients.
Not yet. But we really want to. We do publish the dashboards to the health
centers. I actually shouldn’t say not yet. There may be some health centers
without my knowledge that are publishing them on their websites. I know that
several of them at least that tack them up in the hallways in their health
centers, but so far I have not seen actual proactive attempts to push this data
out to the patients.
However, our centers are embarking over the next year or so in developing
patient portal technology and it is definitely in our mindset to figure out
ways to have these measures sit out there to be shared between the practices
and the patients. I hope I hit all your questions.
DR. FITZMAURICE: Thank you very much.
DR. TANG: Any other questions? Okay. Thank you very much, Dr. Rachman, and
we will move on to Yael. Yael has taken up a new responsibility at HRSA as
Director of the Office of Health Information Technology and Quality. Welcome. I
got your title right.
MS. HARRIS: Thanks so much and thanks for having me here. I apologize that I
didn’t have much time to put this together so I will try to be creative, but
obviously not as insightful as Dr. Rachman has been. Thank you.
Basically I aptly titled 2010 a quality measurement odyssey because I think
we are really at a crossroads here between where quality measure has been in
the past which albeit was pretty advanced considering just about a decade ago
we didn’t have things that reported on the Internet. We didn’t have information
driving decision making. I think we have made quite of tremendous strides in
the last decade, but I really think we are now with the age of EHR really at a
crossroads in terms of what is available to us and also realizing what the
limitations are in the quality measures that we have to date. And I think that
some of those gaps have been really recently identified because of the move
towards meaning use.
For example, most of our measures are disease specific. There is really no
consideration or underlying factors or multiple chronic conditions. It assumes
that every diabetic is treated exactly the same not taking into account any
other conditions that they may have. If those are taken into account conditions
like mental health that may affect the effectiveness of some of these
interventions and I think we are very unidimensional in our approach.
I also think we have not been very person centric. We have been very disease
centric. Looking at it as a disease as opposed to as an individual with a
collection of diseases that we are trying to help improve.
Also measures have really been developed without specifying how they are
going to be used. And my favorite example is I was actually at what was HCFA at
the time when I was in a meeting and we were deciding on what was called the
Nursing Home Quality Indicators Project and I was in one room with a group of
stakeholders where we said these are quality indicators. They will not be used
to determine what is good quality. They will be used to help just identify
areas that need further investigation. Down the hall the administrator of HCFA
at the time was making an announcement that the quality indicators project
would become publicly reported and updated on nursing home compare. I then had
to walk out of one meeting into the next meeting and say I apologize. We made a
mistake. And these indicators that we just said are indicators are actually
measures and will be publicly reported by this time next year. I think we need
to be cognizant as we create measures what their purpose is for, what their
intent for, and think through how we structure them.
Another concern and this stems from — I was just at a meeting last week for
MedPAC and they were developing quality measures for long-term care hospitals.
And someone in the room said well, you know CMS always relies on those
averages. That shows they don’t know statistics. I was like I designed that —
compare websites. Thank you very much. But I fully acknowledge that.
The limitations of statistics using — relying on averages and seeking out
averages are not really telling us what is good quality. We really need to have
a better understanding of statistics. The issue of small denominators using
rolling averages to make sure we have a large enough denominator report means
we don’t really understand these measures and what should be reported and how
they can be used to improve quality. I think it is understanding outliers which
highlight unique circumstances when you need to use one or two details to
identify issues. When an average is appropriate and when an average is not
appropriate and when the denominator should be suppressed. Is it because we
don’t really see it as a quality problem or just that we feel that statistics
don’t point that direction.
I tried to find a nice picture of a landscape and I just thought this one
had more white space for me to type on so I picked this one. No appeal to any
specific nation or country. But I just really think as we think through quality
measurement, I think the critical thing is thinking about the landscape.
Everything has changed since the passage of ARRA stimulus bill and the passage
of health reform. Whether or not Congress repeals health reform I think the
landscape is completely different and will never go back to the way it was and
I think we need to be cognizant of how it has changed the landscape and make
sure that as we think forward through quality measurement, we think forward
with all these aspects in mind and think through how these are going to impact
what we are doing.
The first thing I had was patient-centered medical home and I don’t think
you can go anywhere now without hearing that word. I don’t know if we have a
standard definition. Obviously many people use it many different ways. CMS is
coming out with their demonstration projects around it but I hear a whole lot
of different entities. There is a patient-centered commission that I get emails
from literally every other minute. There is a whole bunch of people at the
table. I am sure everyone is in it for the best but I don’t know if we know
what we mean when we say patient-centered medical home or where we intend to be
except that we all have the best of intentions.
But I really think it has a lot to do with about measurement because when
you talk about care coordination and we don’t need to talk about it in the
context of one single disease or in context of a single provider. I think we
need to think of care coordination across all providers again at the person
centered interest. A person doesn’t care where they are getting their treatment
and a lot of times we don’t know where the best treatment can be received.
After acute care stay, do they go to a long-term hospital, a patient
rehabilitation facility, a skilled nursing facility, a home health agency? Some
of the same patients go to all these different settings and we don’t know where
they are getting the best quality. That is why we need measures that we can
standardize across all those settings so we can determine what are we getting.
What is the bang for out buck because these centers don’t — these places don’t
all provide — have the same costs. We don’t know if they all provide the care
and the same quality and so we really need measures that can help us determine
the best treatment for the patient dependent on the whole patient as a whole
not just one disease diagnosis.
Also we can’t just focus on disease as a corporeal phenomenon. I think we
fail to look at a holistic view which includes behavioral health and oral
health. Oral health is something that HRSA has been very involved with because
of the fact that we see how it affects health care in the long term. And mental
health I have heard championed for a long time but for some reason it just
keeps getting missed which is if you cut off the head sure you can treat the
body, but we all know how the mind affects everything. If you are treating an
illness and you have a mental illness underlying that, it is going to
exacerbate the conditions. You are not going to be able to treat the conditions
acceptably. So really when we look at measures, we have to look at
person-centered approach and consider disease prevention, health promotion,
looking at the whole person.
The next big thing is the ACOs. And I just saw a draft of the CMS rule and
it hasn’t come out for prime time yet. It is still in the department for
clearance. The ACOs will be a reality very soon. I think a lot of us are
optimistic and a lot of us are skeptical at the same time about what will
happen with ACOs. But I think we need to consider them as we go forward in
quality measurement because one of the critical features of an ACO will be
measuring quality and how they impact quality.
What is an ACO and what is its role in the patient-centered medical home? Is
it really just about controlling medical costs? Is it about being a partner to
achieve the patient medical home? Is it just about reducing redundant tests?
Who is responsible for the cost and who does the measure affect? Do they impact
how the ACO gets reimbursed, et cetera? I think these are things that we need
consider as we move forward. I don’t have the answers. You guys are the
experts, but I wanted to make sure you guys are thinking through these things
as you move forward because I think these will all come into play in the next
decade of quality measurement.
And then meaningful use which is the one I am most familiar with having
worked with Paul on the first draft of meaningful use. And I think there are a
lot to be said here and a lot that hasn’t been said yet. First of all, I think
this goes without saying is that we need to have the measures dictated by the
need not by what is available. For the longest time we have been going for the
low-hanging fruit and that makes sense in a lot of our quality efforts because
there is just so much to tackle that why don’t go forward with what we can
feasibly touch and feel and do right now. But as we move forward stage two and
stage three of meaning use only having measures that are easy to measure means
we are missing a whole spectrum of the quality and I often refer to quality
like a diamond. It has so many facets. And just because there are parts we
can’t measure doesn’t mean they are not important features. And if we don’t
have a measurement for them, they won’t be built into meaningful use and if
they are not built into meaningful use no one will ever work on them.
You guys are really in a drive seat about transforming quality in America
because if you just look at the facets of the diamond we can see, meaningful
use is going to miss the mark tremendously and I think that is a real tragedy.
Going beyond that I think there is unlimited potential in electronic data. I
think it opens the door to so many measures we have previously haven’t thought
about. Just thinking about patient errors everyone says you can only measure
near misses. No one reports on the real misses because the patient is dead.
That doesn’t get figured into the denominator.
I think with electronic health records we have a real opportunity to measure
things that were either too burdensome to collect information on or were too
difficult to identify such as safety measures, duplication of resources, et
cetera. There is really an opportunity to use electronic health records to
broaden what kind of information we can collect and measure.
And while it may initially be burdensome to collect it actually allows us to
measure things that previously we would say could not be measured because we
couldn’t collect the data. Now if it is somehow built into the EHR while
initially it is burdensome to collect, it is part of providing a quality focus
of care so it is something that physicians should be collecting and then
something that would automatically feature into the calculation of a measure.
I think it is also the value of real time use of quality data and this has
been worked on very closely by a number of colleagues being led by NIH and ONC,
but I know CDC and AHRQ and others are at the table about creating a learning
health care system so really using data in real time. So not just having a
quality measure that will be reported for last quarter or something that the
doctor could use say you know last week I had 20 diabetics and I only did a
retinopathy on 80 percent of them. It is more than — it is real-time data to
generate decisions.
Forget having someone read every single journal in the United States of
America which I have always joked would require every physician in America to
spend 2 months of their year just hold up reading through every issue of JAMA
and New England Journal of Medicine, et cetera which no one has time to do. But
beyond that having real-time education of information so you can generate
quality measures in real time looking at data because at that time you would
have access to the entire repository of patients to say all right, we don’t
know what the best treatment is for fibromyalgia. But we know that across
America these are the different treatments have been prescribed and this is
having the most common effect. It is a chance for actually real-time
measurements to be created as we move forward not just prospectively thinking
and that is something to take into mind.
Also, the flow of information. I think care transfers have been highly
emphasized. It is something that the QAI program focused on in their last scope
of work. They are now thinking through their next scope of work. But I think
the real critical thing is that having information is a necessary but not
sufficient condition to good quality care. I think we can all agree to that.
And I think information flow is the one thing that an EHR can give you or
information technology can give you even if it is just a CCD. Having
information in real time means that you have 24/7 access to information so
clinicians can make decisions from their bedside instead of saying just send
the patient to the ED because I don’t feel like coming in to look at my chart
today. It means having people make informed decisions, having clinical decision
support at their ready.
I was just in New York and heard a colleague talk about my favorite story
was he said it was a dark and stormy night. But basically he had been seeing
patients all day at his clinic in New York City which is a community health
center that treats a lot indigent people in New York City. And he saw a child
come in and he was close to the day, but the mother said I have been here
twice. I have seen two different MDs. Please see my daughter. I can’t get her
fever down. I don’t know what is wrong. It is 6 o’clock. It is raining. He just
wants to get home to his family. He looks at the patient and he says I see
nothing. Low-grade fever. Child is — there is nothing I can do — electronic
health record and out pops something that said have you checked for West Nile
Virus. And he said, you know the sad thing was I entered those 2 weeks ago when
the CDC notice came out. I took that CDC notice and I plugged it into our epic
system so that those alerts would pop up if someone saw a low-grade fever and
these symptoms. And he goes but here I was the one who entered it into the
system and it didn’t occur to me because I was so tired and it had been a long
day until the system popped up.
This is what I mean about the use of technology to really change how care is
provided and I think those are things that we don’t even realize until we will
start taking more advantage of them, but that is the real future and what
measurement can offer us.
And then finally innovations. I work on a mobile health work group across
all of HHS and someone gave me the example. They said when the cell phone first
came out and people carried a purse with a brick in it. They would bring out
the brick and bring it to their ear and people were like wow you are talking on
the phone. Could that person who was talking on this wonderful new novel device
that takes up this much space and is 10 pounds in their purse even imagine this
type of device and what this device could do? And it has only been a couple of
decades.
We can’t even begin to imagine what kind of innovation there is for us in
the next decade. I think we need to be cognizant of that as we move forward in
measurement because we need to realize what innovations can offer us in terms
of health management and health care coordination and treatment.
Then the big one is pay for performance. I don’t want this is anything new,
but I think it is growing in terms of its role as it expands to multiple care
settings and multiple providers and its role in terms of care coordination.
I think we need to think about measures in terms of how they are going to be
used. As I mentioned before the quality indicators for nursing homes were never
considered to be public reported quality measures and now some of them are
going into a pay for performance demonstration for nursing homes that CMS is
working on.
I think we need to consider the measures in the context of how they are
going to be used. Are they actionable? A measure that we can’t really do
anything about is nice to know, but it is nothing we can do. We want a measure
that we can actually do something with that we know how to improve and what the
right steps are. Evidence-based medicine.
Are they measures appropriate for pay for performance? It doesn’t mean that
they may not be good measures for us to indicate where further review is needed
where further risk management is needed, but we need to be cognizant of which
measures are good for pay for performance and which measures have a lot of
subjectivity and may not be the most qualified measures for pay for
performance.
What are realistic goals? In some cases some measures we say they are never
events. We know they should never happen and they really should be avoidable. I
would say 99 percent of the time they should be avoidable, maybe even 100
percent. It depends on who you talk to.
Other times we know they are going to happen. We know you are never going to
get to 100 percent of people having this or that or that. What is reasonable?
What should our goal be because that is what we should strive for and that is
what we should aim for?
And then we need to talk about what is average? Is average acceptable
because if you have outliers they are moving that average up or down? Do we
want to say that the average is what we want to achieve or do we want bimodal
distribution? Sometimes it depends on different populations. These are the
things I think as we think through measures, we need to take into account.
And then I also want people to consider what adverse behaviors might this
measure create? Putting this measure out there and having people report on it.
Is it going to cause people to act differently? And I will give you an example.
Falls. In nursing homes, now a fall is considered anything that is — where
anyone drops from one position to a lower position whether or not they are
safely lowered into the position it is a fall. Now you have thousands of people
reporting falls all over the universe. Do we think falls are a bad thing or a
good thing? If it is a controlled fall, if someone knows the person is at risk
for falls is following them and catches them, but doesn’t restrain them I would
say that is a good thing. If we don’t know if a person is at risk for falls and
we just tie them down to avoid the problem, we have 0 percent falls in that
facility but every single person is tied down to their chair. I wouldn’t say
that is very good quality.
Taking into account how different measures bounce one another and what
adverse effects may happen by reporting on a measure unless we think it through
in its context.
The next thing that I want to talk about was population health because this
is growing by leaps and bounds and it is really big focus. First of all, the
cultural context of population health in terms of how people think about
things. One example is reporting of pain. Different cultures announce their
pain differently. We have the standard face of scale that people use, but there
are a lot of other measures that may be affected by people’s culture and by
people’s ability to react to them. We just think about them in terms of the
different populations we treat both by finances, sociocultural phenomenon,
location, rural verses urban, et cetera.
Also the availability of resources. One of the things that always get cited
are facilities that are high on Medicaid always have worse quality. Is that a
factor of their resources or of the types of care they provide or of the types
of patients that seek care there? I don’t think anyone has really teased that
out. There have been papers blaming everything. But it is really a question of
how do we create a fair playing field to see what is achievable and what we
could achieve in that setting and try to figure out what are the factors that
contribute to poor quality so we can work to improve them. And then you need to
take into account the type of population that is coming there.
And finally, the implications for risk adjustment. I think a lot of times
people over risk adjust to the point where there really isn’t much difference,
but then at the same time if you don’t risk adjust everyone says my place is
unique. I have this or I have that or I treat more of this. Where do you find
the fine line? Risk adjustment is a necessary condition, but you don’t want to
use it to the detriment of actually identifying real problems going on.
And then impact on behaviors and attitudes. What is acceptable in one
population may be very different depending on the type of clinician you are,
depending on the type of care practice you are providing care in and depending
on the type of patient population you are serving. Taking those into context as
we develop measures is very important.
Just two more slides. I think patient preferences are a critical thing. I
know that there has been a lot of work that the NQF has been doing on, the
CAHPS, the Consumer Assessment of Health Plans. We really don’t have a good
measure of patient satisfaction. But I think there are measures of patient
experience are critical and it is no longer just black and white in terms of I
was satisfied or unsatisfied. There are varying gradations of that. I think we
need to take that into account. I think we need to take into account the
patient’s experiences, the provider’s experiences.
Levels of subjectivity. Obviously if mom died, they are going to have a bad
estimate of the care provided regardless of whether the doctor did everything
for them. How do you take that into account, I don’t know but I do think it is
something and I think when you are talking about a patient looking at quality
measures to choose where does he seek care, this is the information they find
most valuable and most relevant, not how many foot exams you have done for your
diabetes patients or how many of your patients had an obstetrical exam in the
last 12 months. Patients want something that is meaningful to them and so this
is something that we really need to develop.
Finally, I think we need to do an increased focus on people with multiple
comorbid conditions, not just looking at one situation, but looking at people
with multiple comorbid conditions and their experiences and what their
expectations are because this is a growing segment of our population especially
as we get more baby boomers and we really need to be cognizant of the fact that
they are not one disease. They are multiple diseases and their experience of
care is very different because they are managing and coordinating through
multiple provider types.
These didn’t really fit into any rubric, but I just wanted to put these down
there. One of the things that we have been thinking about in HRSA although we
are not really in a position to say anything, but we are on the board of
directors now for the NQF and on the NPP, National Priorities Partnership that
we have been putting our word in edgewise is the need to develop scales or
indices. So, rather than just one disease kind of a scale or a set of
conditions that we are measuring, looking at a person as a whole, not as an
individual diagnosis.
Also taking into consideration new technologies and how these affect the
care that we provide whether for better or for worse. I think the funniest
story I ever hear is those of you who haven’t heard it. When the new invention
came, I think it was the 1920s, the American Medical Association said this was
the worst technology in the world. It was going to separate the patients from
their providers, create this disconnect between physicians and patients. Does
anyone know what it is? Exactly, the stethoscope. It was in the 1800s. I
apologize. Thank you. I knew it was pretty early on. I didn’t realize it was
that early on.
When you think about it, people are resistant to technology but technology
can transform health care. Can anyone imagine practicing medicine now without a
stethoscope? I even have one at home just for my asthmatic daughter and I am
not a clinician just so I can hear how her lungs sound. Imagine clinicians not
using it. Just to think about what technology means in the future because it
definitely has a strong fit into where we go with quality measurement.
I also want to consider future changes to health reform whether health
reform is repealed. I certainly hope not, but it is on the table. Whether
another bill is passed to change some things. We need to take all these things
into consideration because it is going to change the landscape and what is
deemed important to clinicians and to patients.
And finally, I think the need for flexibility. As we move forward we need to
be flexible about what are the different thresholds we want for quality
improvement. Are there different thresholds? Are there two deviations above the
mean and that is what we are striving for? What are we going for? Is there room
for revisions? A lot of times measures have been developed and then they are
set in stone. And I know CMS has a quality measurement review process, but it
can only do so much and have so much time there. They are stretched very thin.
Once a measure is determined to be appropriate for a setting, it goes into that
quality measurement standpoint and there are several years before it is
relooked at to assess whether it is valuable.
And as we develop new assessment tools we can’t revise the measures per se
and I know I keep referring to nursing homes, but for example the quality
measures that were developed for nursing homes were developed on what is called
the MDS, the minimum data set 2.0. That system effectively stopped September
31. October 1 they started the MDS 3.0. It will quite a while before they can
figure out how to do measures using MDS 3.0 so we have no measures for nursing
homes going forward right now. I think it is being reported on very old data
that is irrelevant to the new system.
I think this is something that stands to reason for all of our health care
settings because as we move forward with new requirements for certification of
EHRs, et cetera, we need to consider are the measures flexible to accommodate
those changes.
And finally, the need for evidence-based science. Health reform has really
pushed for that. Directly AHRQ is leading the way on that but I think we need
to really be cognizant of clinical effectiveness research and relying on what
we know is effective before we put out measures that we want to improve on but
we don’t know how to improve on.
This is my contact information if you have questions. And I am happy to
answer questions at this time. Thank you.
DR. TANG: Thanks Yael. I think because of the time, we won’t have time to
ask questions right now. Perhaps we might have a little time at the end but we
have two more speakers. Thank you very much.
The next up is Dr. Cullen, and Dr. Cullen is the CIO for the Indian Health
Service.
DR. CULLEN: (mic off) I asked our sites about it. It turns out we are
evaluating that at one of our sites. It is interesting to note that is a
proprietary product so the site had to pay to use it which I didn’t know about.
So at the same time a two-question, non-proprietary evaluation. The subject
about consumers understanding percentages, we have done lots of focus groups
and we have real problems with that. Probably less than 15 percent of our
patients understand percentages. That even extends to the concept of BMI, which
seems like that should be pretty easy because it is a number like hemoglobin
A1c, but that ability to translate a BMI into whether I am overweight or obese
has been difficult for our patient population.
The dashboard approach. I just wanted to — Blackford, I am sure you know
this. We actually run our dashboard on a live database at the time of care. We
don’t export to — we do export to a data warehouse, but we run the ability to
do on the fly queries on a cache database. People have indicated they thought
we couldn’t do that. We have been running it on databases that have up to
300,000 patients. We actually asked 20 people to not do it in the middle of the
day because you can get some efficiency productivity impact, but overall it is
fine.
I ook somewhat of a different approach to these questions what you are going
to see here today. I do give you some specifics about measurement, but I think
that there are some major issues in architecture and design, and I am looking
probably not at 2013, perhaps not even at 2015, but really trying to do a
roadmap on what I think we should be doing as opposed to continuing on this
perpetual quest we have right now to come up with the measures. It is about
accountable measurable about health care organizations.
The other thing I should say I do work for Indian Health Service. I have to
use every opportunity to plug my little agency. There are slides at the end I
am not going to talk about them, but they tell you about us so if you don’t
know about it.
I went back to the quality of health care for 2001. And also at the end of
this is what we need to change because I think that we need to have some
heuristic approach to get us out of the chasm we seemed to be diving into. And
I just want to step back a little bit about what we are doing in health IT
because I don’t see this changing right now with what is going out there. It is
still driven by visit. It is still episodic. It is still based on
reimbursement. I, in fact, moonlight in a private ER. They love our private
system because my average EM code went from a three to four and they are
getting a lot more money. They are really intrigued by the fact that I am not
clear my patients are getting better health care even though this is what I do
for a living.
I do think that this traditional perspective insures that the provider sees
the data in the way they have always seen it in a SOAP-based format which there
are days and God bless, Larry, that perhaps we need to get beyond SOAP. I don’t
know where to go, but I do think that we are constraining ourselves by saying
that the way we have always done medical decision making and outcomes
evaluation is predicated on our recording data within that format.
And as we know right now we only measure the data that is contained in
obtained from this format. We have measures that reflect that appropriately.
IHS has embraced this for 12 years. We have been doing clinical quality
measures. We have used what has been available when there haven’t been
standards. We have extended standards. It has been highly successful. We have
great data to show that we have been to have significant change at the point of
care, but I think it is limited and I think we need to go beyond this. And I
think it is limited because it doesn’t incorporate a changing model of health
care delivery. And I am going to talk about a measure that I think can help us
get at that.
Also, the way we define the measures is really based on the evidence base
and available standards and it is not, for instance, reflective of an American
Indian, Alaskan native communities. It may not be reflective of other ethnic
groups. It may not be reflective of patients with disabilities when we try to
get into smaller denominators. What I think we need is malleable systems and
learning health care systems. Everybody is talking about that we need quality
measures that do that. I am going to go quick because of time here.
I think we need to step back a second and I realize it is not the purview of
this committee perhaps, but I think that if we don’t step back we are just
going to continue on the path where we are going. I think that we need to
ensure that the health IT — I feel like a broken record sometimes. That the
health IT systems are designed to facilitate a measure the business process
reengineering as well as the outcome. And right now that is difficult to do
because the health IT systems are designed to get what we have said we wanted
to do which is billing reimbursement and outcome measures.
And that in order to do that we need to make sure that the essential domains
that are captured there include that first question that I think Charlie made
at NCI has really taught me and I think he is right. Does this data need to be
interoperable? We, in fact, ran our pharmacy application. It has lots of data
fields, about 250. It turns in only about 15 of them need to be exposed. We can
do an API for the whole app but we really only need those 15. The local site
needs that other data for things they do, but I think that there are some
questions we need to ask and we need to relook at.
The data sources that we limit ourselves to and we all talked about this are
really based on what we have traditionally have done and I think we need to get
beyond that. The interesting data on narrative medicine, that whole concept if
we go back to history and physical really helping us decide 95 percent of the
time what is wrong with patients. I think we haven’t figured that out well. We
haven’t figured out how to record that well in the health care system
obviously. There is a subjective line in a note. But we aren’t getting it. We
are not teasing out these domains that I think could in the long run save —
ensure that the health care system became more efficient from resource
utilization.
The impact on communication. That was talked about this morning and it is
interesting. Once again these slides are really a compilation of my staff and
they are on to this communication. They think it is the critical thing. They
think it is what we are not doing well. And as you can see they believe that
that communication far extends beyond just the patient and the provider. It
really goes into the clinic into the community. It goes among the care teams.
And because we are predominantly a primary care setting, it extends to how we
share data with the specialists that ensures that we can get the appropriate
outcomes.
And this morning or maybe it was this afternoon. Somebody talked about Amy,
the patient Amy, and what she does when she goes through that health care
system. Our docs are that blunt. They are out here. They are sending that
patient through that and they can’t figure out how to navigate that.
Because of our cultural issues and that we are rural we have many patients
that choose to not do things. They are well informed. They are making
decisions. I don’t want to transplant. I am really okay with not having a
transplant. It is an okay decision. But we let them go adrift out into a health
care system that kind of pummels them and says no that must be the wrong
decision for you and actually it may be the right decision for that patient. We
need to somehow have the system do it.
Let me go back to quality measures which are what you guys really want. I
think they need to be dynamically designed. It goes to the question earlier
about what do you do when there is a difference between guidelines and there is
a difference between a measure. If you program that statically which we do for
our overall reports to Congress and they want one number, hemoglobin A1c is
less than seven, we give them that number. But we also facilitate and allow
there to be dynamic measurement at the site where a provider can say I am
looking at all this new data on diabetes and I am not going to get anybody
below seven and a half or eight. I know that there is a guideline. I will give
you that data, but at the same time this is what I am doing with my population.
It really begs the question of what happens with reimbursement and the scary
thing she does while it is a measure but now it is tied to reimbursement. But I
think that it is important that we do that.
We also need to include that there is different data sets. I have been on
some groups lately and we have been really driven by what are the major
mortality issues in the country. Well, perhaps it is really YPLL. I don’t know
what it is but the systems need to be able to facilitate that and this
framework of specific models of care and Yael talked about this. I actually
think that this is critical. This is old stuff to a large extent that
traditional patient-centered care and then the medical home which I know is
seemingly a novel concept, but I think all of us in primary care have really
focused on that for years. It goes back to Paul Nutting with work in
community-oriented primary care that is from the ‘70s that we just haven’t
figured out how to embed in the health IT because once again it is a visit
centric reimbursement centric system.
We need community-based quality measures and you can read here. I think it
is really important for us at least that while there be national measures that
the facility and/or the provider has the option of saying my patients are
special and I am running my special population and they are my denominator and
I am going to see what happens to them. Because I think if we start saying to
providers no, nobody gets to be special especially in primary care, we are
going to lose them. They all are special and we all relate to our patients
differently. To have us say at a national level that there needs to be national
measures I think people are fine with that, but I think that until we give the
providers the ability to be able to have some flexibility in how they define
their denominator, people will push back there.
I am going to go through those things really quickly. Incorporate safety.
These are some really specific ideas that are things that might be important.
The longitudinal data view. I am actually going to give you a slide of a new
application we just released to prove to you that this can be done. Tag
patients with appropriate risk assessment. Some of these are process. They
don’t all need to be outcome. I am going to talk about that because I think the
ability in fact to recognize process measures is really critical. Early
sentinel awareness. The ability to quickly identify patients at risk due to
blah and the blah is what the system needs to enable and facilitate. That is
really the measurement. It is did you recognize H1N1. But can you do that again
for something else that is coming down the pike really quickly and we are not
going to have time to do it.
This is where I want to spend just a little bit of time. The provider — I
have actually talked a lot about that. How flexible can the system be? Patients
confidence and self care. But really this improved care planning and care
coordination. We have an improving patient care initiative. It is now at 120
sites. Everybody in that initiative would say you know what you are — they
hate my old measures that I program 10 years ago. I was so proud of myself. But
they are like they are important but you know what is more important is I do
this intervention and 4 weeks from now I can assess whether it made a
difference. I don’t really care whether you want that hemoglobin A1c at seven.
Well, they do care.
But they want to say I changed patient registration. I changed the patient
experience. I want to be able to measure it in 4 weeks. In some ways the
measurement is does the system itself facilitate process improvement as opposed
to us trying to define what process improvement is. And that reflects that this
burden of measurement is high and that we need to make sure that it is passive
and that the investment in it is minimal.
Patient centered. These are things people have already talked about today.
The one thing that we have really been focusing on is this extending the views
to the family, to however the person defines their family, to the community and
the population there. mHealth has already been talked about and appropriate
patient knowledge.
I think timing gets really difficult and my whole team struggled with what
to do here. This is pretty traditional stuff.
Efficiency. I think there are ways to really do this. We should focus on
groups. We should be using more of them. A measure is how many patients got an
order through group order entry. How many patients got care through a shared
decision making process? How many patients got care through standing orders as
opposed to saying computerized provider or order entry requires me to hit a
button every time somebody gets a Pneumovax. Thank God CMS backed off on that a
little.
But I think that if we really want to be efficient we need to look beyond
that traditional doctor/patient relationship. And you can see some other
options here.
We are also really convinced that there are these regional differences in
utilization, that they are really difficult to tease out, that it is not clear.
It is just access to care. Obviously the patient and the provider themselves,
have a lot to do with that and how to measure that in a way.
Equitable. I think that this one is not as difficult as we have made it.
There are five performance measures and if you slice and dice those
denominators, do you come up with different access to care? Do you come up with
different outcomes? I think that the problem is that the health IT systems have
not really been designed to ensure that that can happen.
If we did all that, would it improve quality and accountability? I don’t
think so. I think there needs to be some other things. I think we need to —
this is probably why I am 5 to 8 years out, I really think we need to start
looking beyond and see what is true disruptive health IT technology. I think
just continuing to incorporate this formatting giving me access to everything I
have had may not make a difference.
I work in a system. I still provide clinical care where the providers are
expected to see 30 patients a day. No ifs, ands or buts. That is it. Sometimes
it is more than that. In that kind of setting which is primary care what you
want to do — and you are rural so you can’t send everybody to a specialist.
You have to figure out what is wrong to a large extent. This system is helping
and our docs love health IT. They have embraced it. But there is more that we
need to offer them.
And finally I just want to show you this. This is just a new application we
just released 2 weeks ago. And the reason why I want to show this to you is
that — you know that slide of Amy who went to 26 different places and it was
crazy and there probably wasn’t any longitudinal thing. I actually think it is
not that difficult. I think it is related to design. This is a big funnel. You
can throw in whatever event you want. It can be a lab. It can be a procedure.
It can be a result. And it allows you to have a longitudinal track. We have not
worked out all the glitches. We know it works. But what happens is on one
screen instead of having a problem list, it says abnormal PAP smear with 25
things after it because we see patients for their lives. I can go now to this
page that tracks everything and allows me to trigger patient-specific reminders
that override because they are more conservative than the reminders that are in
this system or in fact extend them.
With that I am going to stop. Thanks.
DR. TANG: Thank you very much. I think we will press on. We only have 10
minutes. We can go maybe 15. And we will see if we can catch any questions at
the end.
The next up is Karen Kmetik from AMA.
MS. KMETIK: Hi everyone. I am Karen. I am with the AMA and the PCPI and I am
sorry I am not with you in person. And I think given the time and I am sure you
want to have some questions. I am going to be brief and try to maybe comment on
something that I did not hear emphasized by the other speakers who were so
eloquent in their remarks – and I know I learned a lot.
One way to look at the question of what information measures do providers
need to improve quality and increase accountability, perhaps another item to
put on the table is they need confidence in the data and that is true really
for anyone who is going to use the data of course whether it is a physician or
another health care provider or an organization that is trying to determine
accountability or whether it is a patient who wants to be certain that their
preferences are recorded. I would just put forward that we need to be mindful
that for the data to really be useful in improving care and to be used for
accountability. We all need to have confidence in the data.
I will just give one very brief example of how we might want to look at
this. There are a set of measures for HIV/AIDS and to the point I think, Fred,
you made earlier about these measures need to be aligned. Here is a great
example where we did all sit around the table. Our group with NCQA, with HRSA,
with the Infectious Disease Society of America we said let’s all get on the
same page. What is important here for patients with HIV/AIDS? We have a set of
aligned measures and I have no doubt that over time those very thoughtful
groups will come back around to the table and add measures such as what is the
shared gold treatment. What is the patient-reported outcome? I have no doubt
that that will be added.
We take those measures and then groups are trying to implement them. We have
talked with Lisa Backus at the VA and she is actually integrated those measures
into the VA EHR system. And we talked with Michael Horberg from Kaiser and he
has actually implemented those national measures into the Kaiser EHR system.
And they are both sharing very valuable information about variation across the
measures and issues that they are now going to focus their QI efforts. That is
all terrific.
But what we started missing though is how confident are we in those data by
which we are going to make some very important decisions whether it is right
there when you are talking with the patient about what to do next or whether
you are sharing information externally for some decision making.
And to something to maybe think about is as we move forward in this
enterprise with these great opportunities, how can we also be relaying to
providers others confidence in the data that they are seeing, that they are
making decisions on? One step is in this case for the HIV/AIDS measure we
actually worked with Fred’s group who spoke earlier and said can you test these
for us to make sure that if somebody who wasn’t at Kaiser or wasn’t at the VA
implemented these measures in their EHR. What should they look out for? Are the
specifications written in a way that an automatic query to that system is going
to present similar results as to if an EHR in a different location was doing an
automatic query.
And it is from that kind of testing that we learn a tremendous amount about
what we need to do differently in terms of the data collected in the EHR, what
we need to tell providers as Fred said. We learn about what are some of those
glitches that we need to resolve as a nation in terms of sharing data or in
terms of using standard vocabularies. But I just offer for the conversation
that providers and others need confidence in the data and we need a way to
continually evaluate measures, test out new ideas for measures. We are putting
together a network to do this and I would be happy to share this with the
committee after the day here to try to make this happen in a more real-time
basis so we don’t have measures out there and even the next generation of
measures and we don’t have a lot beside that a way to help improve the
reliability of that data. I think I will stop there and happy to be part of the
conversation.
DR. TANG: Thank you, Karen. Open for questions. Maybe one crosscutting
question and I am not sure — we have heard some opinions. Let me refer back to
Dr. Rachman saying they use national consensus and I don’t know whether these
are guidelines or measures. But one of the things that I think that multiple
speakers talked about was the difference between what you use internally and
public reporting. That is a quandary that goes on in many circles including
National Quality Forum, but the whole notion of some of these publically —
these NQF endorsements publicly reported measures are designed to be just that,
publicly reported. And they seem to carry a different and someone comments on
the purpose, a different purpose and people designed the measures differently.
Is the question to improve the system or to ace the test I guess is one way to
ask the question? I wonder if the panelists have a perspective on that.
DR. RACHMAN: This is Fred again. I think that we should think nationally
about putting out a sort of standards that are going to be based on data
elements and content that again will send a message out to what we think is
important for people to focus on. And I think it is inevitable that the
consumers of these measures are going to include those that want to give
grades. But as much as possible I think we should focus on having a set of
measures that are in fact aiming at a vision for where we want to be as a
nation in 10 years and work hard advocating that those that are using the
measures for grades are resting somewhere short of the goal. They are measuring
someplace that is going to encourage people to participate and move forward and
encourage people not to pull away from the notion of establishing national
measures.
I guess also just commenting on what one does with an individual
institution, I think the opportunity is there to use the same data elements and
the same reporting and set higher bars. But again I would pitch that we make
sure that the underlying content the data elements that are populated in the
measures are ones that will be useful for those kinds of purposes at the
individual institution level. I hope that made sense.
MS. KMETIK: This is Karen. I would just add — if we work hard at getting
the data entered in a uniform way, the next question shouldn’t be so
problematic. It would then give people the freedom to do local variation as
they need to with their patient populations, but also be able to contribute to
the national measures which are very important as well. Maybe it is just
changing the focus a little bit to say let’s make sure we have the data in the
EHR that we need for all these different purposes knowing that some are going
to be interested in some measures and others and other measures and that is
probably okay.
DR. TANG: Let me ask you a different question. Let’s pick one disease let’s
say even though Yael talked about not being disease focused. If we pick one
domain, diabetes, would you propose assuming you use — the same data is
available to you. This data is in the EHR, let’s say. Would you define
different measures that you would use for improving your quality as a provider
from ones that you would report publicly or do they think they should be the
same measures and perhaps different thresholds as someone mentioned or not?
MS. CULLEN: I can give you our experience with that, but it is far beyond
just a specific diabetes measure. It is when we moved from specific diabetes
measures to a comprehensive diabetic measure. And what we saw was we went from
rates of 80 percent to 22 percent as soon as we combined six or seven things.
And what happened then, Paul, was the sites decided what is the most important
thing for me here based on the morbidity that I am seeing in my population. Do
I see amputations? Do I see dialysis? Do I see blindness? And then they would
set a higher goal for themselves in whatever that one part of the comprehensive
measure is.
We actually have seen in diabetes at least the whole issue of hemoglobin A1c
is concerning. People are backing off from it. We have given the sites the
option to not use seven at this point because of the data. Some of it is that
we report seven. Some of it is that — we are taking a hit on it but some of it
is that the providers want to be able to say you are not responding quickly
enough. The national measure groups are not responding quickly enough to the
new information that is coming out. I need the latitude.
DR. TANG: Let me just ask the concrete question though. Would you propose —
this is a roadmap for quality measures, are there different quality measures
for inside versus — you said that there could be different thresholds. You
said that you could have some local priority and where you want to focus your
efforts, but are you also saying that we can use the same measures, the same
quality of measures for quality improvement inside and public reporting
outside?
MS. CULLEN: I think you can use the same measure. The issue becomes what are
they doing with the measure result. It is not that you can’t measure. It is
whether it will be putative or negative or you will lose CMS funding if you
don’t have 50 percent of population of hemoglobin A1c of seven.
DR. TANG: Do the other panelists want to comment?
DR. RACHMAN: I am not sure I understand the question. It seems to me if we
are promulgating a set of national measures, those would become internal
measures just by default and I don’t think we could — at the same time I don’t
think we could limit an individual institution from building onto those
measures. But I do think that we should be thoughtful that the measures were
promulgating at the national level are ones that we are sure will be useful to
drive improvement efforts at the individual institution and practice level.
DR. TANG: That is I think another vote for saying use the same definition of
the quality measures with respect to some domain process.
DR. GREEN: Paul, I want to shift to another area and notice like Karen did
something that has not been talked about before we leave this provider segment
of the hearing. It is interesting to me — I missed it if someone brought this
up, but part of the answer to the question is what information of measures
providers need to improve quality and increase accountability. I want to make
it very, very explicit and call that — providers need to know for whom they
are responsible.
Say it another way. They need an answer to the question who is my patient.
And this connects back to what Yael was talking about avoiding measures that
are so disease specific. It also goes back to Judy’s comments earlier this
morning about all of that interesting stuff about the PAM and about how it
characterizes the patient, the type of patient that the disease metric is being
applied to.
I just want to beat this to death for a moment. Providers who do not have an
unduplicated and accurate account of who their patient is are going to have a
very, very difficult time dealing with quality measures that are patient
centered, person specific, or summed up to population specific. To go to this
conversation about the seven and for the A1c. Well, it really helps if you
don’t get the same patient counted six times in the dataset.
When I said I was going to beat it to death, I was really serious, Paul.
This subcommittee is part of the whole committee. Revisited at our last meeting
the fact that in the HIPAA law there were some identifiers that were identified
as necessary and as we all know we have had all we want about the provider
identifier. Thank goodness we don’t have to do those hearings again. But we
still don’t have the individual identifier for reasons we understand. If you
don’t have the identifier, the use of all these quality measures from a
provider’s perspective can lack traction. It can lack confidence. It can lack
all sorts of characteristics.
Made worse by Judy’s comments about how so much of what we need to actually
measure quality depends upon who the person is that the measurement is about
because for somebody a nine is the greatest success story of their life and for
other people a seven is an absolute disaster. We have to know more about the
individuals not just the diseases like Yael was saying. You can’t do that if
you don’t know who they are.
MS. CULLEN: I would offer one comment though because we have been in a
dialogue with CMS over meaningful use. The other caveat here is depending upon
your practice. If you are in a group practice, group primary care practice and
you share patients because of your lay out or something like that, the issue
becomes how do you report then on those shared patients. Right now meaningful
use in 2011 will probably include many patients that are duplicative in the
denominator depending upon who is reporting on them.
DR. GREEN: I would accept that comment readily, but in our question we have
the word accountability and that begs the question about is it the group that
is accountable or is it someone in the group that is accountable. Back to NCVHS
work, it helps if you know who the provider is. It helps if you know who the
patient is and helps if you know what the problem is. This is so foundational
and I am disturbed just a little bit, not a lot, just disturbed a bit that as
we ask this question about what providers need that we are sort of hurdling
into more details and a level of granularity about this that I am concerned
about because it begs this prior question for accountability.
DR. RACHMAN: I guess I attempted to deal with this with that slide saying,
where does accountability lie. Is it with the individual provider or practice?
Is it with the system? Is it shared with the patient? I would pitch that the
real fear here is about how the measure is going to be applied or how it is
going to be used, but that if we back away from providing the data and
measuring it we are losing an opportunity to identify where the problems exist.
I guess that is what I would say.
Health centers operate from Wagner’s chronic disease model that says that
the patient/provider interaction rests within a whole larger health system. To
that extent I think providing this information at practitioner level as long as
we are not then beating the provider over the head with what we are learning
provides the opportunity to uncover where we need to do better even if it is
just doing better with having access to total information around the patient
becomes a measure of how we are doing with health information exchange.
DR. MIDDLETON: I have a big question for all members of the panel. I am
concerned about — Karen, your comment I thought was very apropos about the
reliability of the data and I am concerned about the reliability of the data
because from the get go it is extraordinarily variable in how it is acquired,
what we record when we see similar patients, how we say what we see and how it
is documented in the record. On the in-bound side it is extraordinarily
variable. And then even when we try to support decision making in health care
with HIT, the decision support is variable both in its representation and then
in its localization with whatever threshold is relevant to particular
hemoglobin A1c or what have you.
And the same logic is variably expressed in how we measure the performance
delivered. That is a hemoglobin A1c rule has a corollary hemoglobin A1c report
that tries to assess based upon who the patients are for the appropriate
denominator, et cetera whether or not that population is under good control. We
also use different definitions of disease. The way you define disease in the
Indian Health Service might be different than how we define it in Boston. The
value sets are different, et cetera.
This variability is just rampant. The question is how we begin to chip away
at that. What are the relevant standards or what are the high value initial
steps to take to try to reduce the unwarranted variability either in recording
documentation and observation or decision support or reporting?
MS. KMETIK: This is Karen. I will just take the first crack at that. I would
say to shine a really bright light on it just one way to begin to go about it
and you are right in that is expediential as we add all these opportunities for
error. But shine a big, big light on it. For example, again, if we take a
particular measurement set notwithstanding the question about patients have
more than one issue, but if you start there and when we talk about it at the
national measurement set let’s side by side say and today here are a dozen
places that are entering data that would be required for these measures and
here are the different ways they are doing it. And here are the different
vocabularies they are using even though we know the nation is to be moving
toward the standard vocabulary. And here are half a dozen other issues. I mean,
Blackford, I just don’t know how we will chip at it until we put a really
bright light on it and step by step try to resolve those issues that we can.
MS. CULLEN: I would say our experience and it is probably why I don’t talk
about reliability of the data because we have almost gotten beyond that because
in 2004 when we said the only way you will be evaluated is based on these
definitions of the data. It has to be entered in these data fields. It needs to
be consistent. We did 18,000 chart reviews that year and we continue to do
chart reviews to document the data. And even though the incentive for us was
never financial, but it was improving health status.
It kind of goes along with what Karen did. We made it all public and said do
you think your PAP smear rate is greater than 30 percent then put your PAP
smears in this way in a standard fashion and lock down the datasets. It is easy
for us to do because we are an integrated health system. But it is easy for the
software vendors to do also and say this is the way the queries are done. This
is the education for the doctor when the data comes in from an immunization
registry the first time you do a QA check on it. The problem is I don’t think
you can — I think you can accelerate that a little, but that is a couple year
processes where the patient and the providers will finally trust the data.
DR. TANG: Thank you both. Any other questions? I am going to turn over it to
Blackford with a 10-minute deficit. We are going to have a little break here.
What 10 or 15 — 10-minute break. We will resume back at 50 after 2. Thank you.
(Break)
Agenda Item: Panel III – What Do Professional
Organizations, Accreditation Organizations, and Regulators Need to Assess
Clinical Performance Across the Continuum?
DR. MIDDLETON: Why don’t we reconvene? I apologize for the short break but I
think if we can get started up again, we will be able to finish on time, which
will be an equally important goal.
Our third panel is all about the professional organizations, the
accreditation organizations, and regulators addressing the question what do we
need to assess clinical performance across the continuum. And we have Kevin
Weiss from the American Board of Medical Specialties, Margaret VanAmringe, I
hope I am doing that right, from the Joint Commission, and Rebecca Lipner from
the ABIM. And Kevin is coming late. So Margaret, why don’t you take it away
please?
MS. VANAMRINGE: Thank you very much for the opportunity to meet with you. My
presentation is going to be a little bit different but hopefully get to the
points also of what we need for measurement purposes.
Just a quick background. The Joint Commission is very well known for its
accreditation programs which you can see in the dark blue colored, but I first
want to mention we have a lot of certification program because this really
bears I think on the topic today. We have about 19,000 health care programs
that we are accrediting or certifying today and they do really cut across a
continuum of health care.
When you look at certification programs, they are usually unique to a
specific medical condition or a particular topic in health care that is very
timely like for example, it could be on health information technology, use safe
adoption, something of that nature. But the one that we are most well known for
is the primary strokes certification that we do and we do all of the hospitals
that currently have primary stroke designation.
We also programs for the federal government in lung volume reduction surgery
and in ventricular assist device. Those are the kinds of nature of the
certification programs, a lot of disease specific or procedural specific type
of care.
On the accreditation programs while clearly we have the majority of
hospitals who you see down in the bottom left. We have about 95 percent of the
hospital beds are accredited under the Joint Commission. We have laboratories.
Under home care we have a large number of about 4800 home care organizations
that we are accrediting. Under long-term care that also includes hospice and
other types of post-acute, office-based surgery. And ambulatory is an
interesting program because ambulatory runs the gamut from something like
ambulatory surgery programs to school health clinics, prison health, retail
clinics like minute clinics that you see in CVS and almost everything else in
between. These are all programs that we are seeking to have some kind of
measurement for which makes it very interesting you see.
You can see the programs going down the middle of this particular slide. And
across the top are the kinds of things that we are looking to have measures in,
everything from care coordination transitions of care, patient safety, moving
across to comorbidities and other types of areas. These are really the typical
areas including efficiency that you are going to probably hear from most of the
organizations that are presenting to you. I don’t want to go into a lot of
depth on any of them.
I will say under patient safety having measures for harm is very, very
important because when you have measures of harm and you can look at why harm
occurred, you actually can delve much deeper into care processes and get an
understanding of many of the causal effects which can inform the requirement
standards and other types of measures. It is a very interesting group that I
would say needs some additional attention.
We also on the right you can see it is very important to start harmonizing
measures for facilities and health care organizations such as what the Joint
Commission is accrediting and certifying with physician measures. And there is
probably a lot more work that needs to be done in that area.
Some other important areas that I would like to mention is we are now
beginning to look at what does it take to measure the new delivery models which
are coming out of health care reform. In some cases not out of health care
reform, but they are just more ways to try do clinical integration or other
types of integration. We intend to have programs in the accountable care,
primary care home settings. We have already launched the beginning of the
primary care home.
But it is a real challenge given the broad array of participants that we
think ought to be in these new delivery models. As I said before, if you want
to bring in retail clinics you really need to figure out in accountable care
organizations or primary care home, how those are going to fit in.
And we would like to see more attention to patient experiences across the
delivery sites in these new delivery models and the management of chronic
conditions because so often the physicians aren’t talking to each other. If
someone has multiple comorbid conditions, it is difficult to get that kind of
measure that looks across how well everything is being coordinated and the cost
experience.
And I would also say that a very important challenge that we are going to
have especially I think in the ACO area is that we are often looking at merging
information from different payers across different patient populations and we
need to have data consistency so we can really benchmark and be able to look at
what we are seeing and I think we are going to have to start facing that
challenge right now.
And the last thing on this slide is one of our most important areas we would
like to see is measures that are more sensitive to the failures and care
transitions. And I say this because as we look at our sentinel event database
which has collected a lot of information over the years on serious adverse
events what you see is the number one safety failure underneath these events is
communication, hands-off transitions. That is the number one. Now, it is not
the only one because often if you have failure in communication, you also have
five other things. But if you really plotted out what is the one single thing
that you could pull out of the majority of these and that is probably about 85
percent of them, it is failure in transitions of care.
The Joint Commission is moving in a new measurement direction that I really
wanted to take the opportunity to talk to you about and this is under Dr.
Chassin, our new CEO and president. For the Joint Commission we are trying to
see if we can’t move really as much as the world as possible to a higher bar
for programs that do look at accountability in the programs and I will define
that in a minute. And we are really hoping that the Federal Government states
other stakeholders that are involved in accountability programs will join us.
And we are not looking to have less measurement. We are actually looking to
improve the measurement so we have better cost benefit for measurement, better
results, higher quality of care, and we really have to start now. And that was
one of the things I was asked to speak about is what do we need to start now if
we are going to have some fruition in 3 to 5 years from now.
We believe that there should be first of all a national goal that can focus
on measures that maximize health benefits to patients and examine the current
measures. And we have thousands of measures today that have gone through
various processes including the NQF processes that are included in national
quality programs and that we should use some established criteria which I will
offer what we think they should be and we should make sure that we are
replacing any poor measures that don’t result in good quality improvement with
better measures. And we suggest that all quality measures used in national
transparency and payment programs be vetted against some specific criteria for
accountability programs.
Accountability programs for us are ones that are used for accreditation,
certification, public reporting, and value-based purchasing. That kind where
there is a clear significant consequence in the public to the metric. Clearly
accreditation if you are not achieving the metrics, your reputation may be at
stake, ability to participate in Medicare and Medicaid. Sometimes for the Joint
Commission hospital programs it is whether your residency program will be able
to continue for value-based purchasing updates, obviously financial incentives,
medical use incentives, penalties. And public reporting can certainly drive
market share and other investments.
It is really important to have a high bar. And the reason we think we can do
this now is to just go back a little bit in history. In 1998 the Joint
Commission launched for hospitals it ORYX program which was the first national
reporting program for hospitals. And in 2002 we said all right. We are going to
take ORYX and make sure that all the measures that are used in a core program
are standardized so we can benchmark and we started that. In 2004, we began
putting that standardized information out on the web and CMS was following
close behind in its reporting program. And in 2005 using these measures went
live with the reporting program which we were very pleased about.
You can see if you look at this graph in 2005 to 2006 when CMS decided to
make public reporting part of its requirements, you can see the large increase
we had in measures being complied with. The reason for this I think is that
between 2002 and 2005 while we were collecting data at the Joint Commission and
we were publishing it to some degree, the consequences we not so great for
people. And it wasn’t until they went on a CMS website the people said I better
look at those data and see if they were really accurate. You saw a big charge
in the reporting and the compliance.
And in fact if you look at 2002 the overall performance that we had with
measures that we believed meet these criteria which I will get to in a second
was about 82 percent. By 2009 it was over 95 percent. This has allowed us
because of the amount of experience we have had to look at which measures are
working and which ones don’t. And to decide that we really need a more formal
process than just the Joint Commission’s activities to assess that experience,
learn from it, and act on it. And we need to really find an approach to do
that.
We put out recently this year — Dr. Chassin in the New England Journal
wrote up his thinking on this and published it. You can see here there are four
real attributes to the measures that we think we really need in accreditation
and other oversight programs.
Here I am talking about process measures. I am going to get outcome measures
in a minute, but we are talking here just about process because that is
primarily what we have been working with. We believe that these are the four
attributes that oversight programs, value-based purchasing programs and other
types of public reporting programs should be adhering to. We need to really
evaluate the evidence to make sure that the research behind the measures is one
that shows that the specific processes that are being measures actually relate
to improved outcomes. That there is some accuracy here that it captures the
evidence-based care process and I am going to describe that later a little bit
more when I show you some examples of ones that don’t. That there is proximity.
That is that the process is more approximately close to the outcome and not a
very distally related to the actual patient outcome. And that that there aren’t
any adverse effects or little or few adverse effects to actually implementing
the measure.
The measures that we are looking for need to have clinical integrity. Well,
we have seen is that when clinicians believe that the particular process
measures they are using actually will lead to better performances. You get
adherence to the measures. The collection of data is better. It is more
accurate. Physicians believe in it. And if they don’t they really turn away
from the improvement efforts that need to follow the measurement because
obviously we are not measuring just to get data. We are measuring to try to
drive improvement from the measurements. And it often leads to workarounds and
a lot of wasted effort as organizations try to pull resources on a measure that
turns out not to be that clearly important.
Some examples of accountability measures, again, process measures that we
have seen in our programs that have clinical integrity that meet the four
criteria that I mentioned earlier are the aspirin at arrival, beta blockers,
ACE inhibitors, the antibiotic prophylaxis, and the new perinatal measures that
we are putting in place. And all of these meet these criteria. These are all
Joint Commission measures. They are also used by except for the new perinatal
measures by CMS. And we think that these are the ones to focus on.
The non-accountability measures that have been in our system that we are
removing from our system are things such as smoking cessation counseling and
heart failure discharge instructions because they do not measure the actual
process. They just measure whether some box was checked often on a form. There
were workarounds for those. And you are not really looking at the quality of
the discharge instructions or the quality of smoking cessation counseling.
Oxygenation and left ventricular function assessment are very distal to the
actual outcome of the patient in surgery for care for left ventricular. And the
last two — they actually can have some untoward consequences and we really saw
that with first dose of antibiotics with people getting antibiotics who really
did not need them.
What we are doing now with the Joint Commission is and I think there are
four of these at least — is we are going to be including the accountability
measures. We are trying to work them into our accreditation standards so that
now that we have seen that people can actually achieve a very high compliance
with these and as you saw from the slide earlier there are a large number of
people who are achieving over 90 percent now. We are trying to look at ways to
bake those into our accreditation decisions. We want to work with other
stakeholders like CMS to eliminate any measures that don’t meet the
accountability criteria, again, process measures. And include only
accountability measures in our program going forward and we will also try to
provide evidence for organizations and help organizations understand the
evidence behind the accountability measures.
And then lastly because we don’t believe that measures are just ending at
the measurement stage. Our Center for Transforming Health Care that we created
a year and a half ago will help organizations improve on accountability
measures and will offer solutions exchange which is a web-based tool with
solutions for achieving higher reliability on the measures.
We are going to also try to create a recognition program to further reward
those who have top performances on the accountability measures.
Our next step is to go to outcome measures which we have not done yet, but
that will be what we are working — that is what we are working on now. Clearly
clinical outcome measures are going to require risk adjustment and I think we
sometimes have a long way to go in risk adjustment on many measures. That is a
challenge that we have moving forward. The measure obviously must capture the
outcome and have minimal or no adverse effects. Some excellent sources of
outcome measures with clinical integrity that we have seen, or the patient
experience measures that we have been using so far in this country for
hospitals and others, functional outcomes for congestive heart failure and
joint replacement.
The cardiac surgery programs listed that some states have, STS’ registry and
American College or Surgery NSQIP. These are all good outcome measures that we
think would meet good clinical integrity standards, but we really need to also
move into this outcome area.
There are some problematic mortality measures that are out there that are
outcome measures that we need to take care of. And what happens is we often see
that the risk factors are not using the factors that are really influencing a
patient’s risk of dying. You can see, for example, that sometimes there is
misclassification of hospitals using mortality measures when you come back and
study it. And very often the crucial clinical factors that are known by
clinicians to have a major impact are missing and I think this is a very
important point because if we can’t get the clinical factors right in the
measures then we are really not going to have the buy in of the people who need
to do the improvement and this is a very critical issue.
We have seen, for example, in Massachusetts recently, I am sure you saw this
summer, when they tried to look at a mortality measure for hospitals and they
used different types and evaluated different types of vendor systems. They
finally somewhat threw up their hands and said there really isn’t one that we
can use at this time because all four were giving different results.
This is very important. Even Medicare’s Pneumonia Model and we certainly
applaud Medicare for going down this route. But if you look at a Fine article
that was printed in the New England Journal a number of years ago and this is
not — this has pretty much stayed constant. These are very much the important
risk factors for treating pneumonia patients. And yet if you look in the
Medicare model only two of those risk factors are actually included. We need to
figure out ways to include more of the risk factors that we need to do that.
That is on clinical issues. I just wanted to end on this particular note
because I can’t leave without saying this. We are a little bit concerned at the
Joint Commission and that sometimes the move to putting measures into the
electronic health record are threatening the integrity of some of our measures
and we have seen that with the VTE and the stroke measures. When they have been
simplified to go into the electronic health record, some important clinical
information gets left out. And we really think that we need to do a better job
nationally in conducting some type of evaluation when we go to eSpecifications
of whether or not we have changed significantly what the derived measure is.
This is something that I know maybe too much in the weeds although I am sure I
am getting a lot of people’s — you have been dealing with this probably form
other people.
We are looking at ways that we can also create mapping to some of the
language that is currently in the eHealth records which uses SNOMED, LOINC and
RxNorm, but some of the information that we may need to get in for the measure
may not be captured in those languages. Can we collect data in another way and
map it into those languages and get some type of consistency and reliability
there? We would like to make sure that at the national level that as we move to
meaningful use phases two and three, we look at the measures that we want to
use and make sure that any EHR specifications have not damaged the measure
themselves and what kind of work can we do to make sure that that hasn’t
happened if we want to use these measures going forward because I am sure there
are fixes. It is a question of getting the right fix.
We also need to resolve a couple of questions. Who is the measure steward
when the eSpecifications are not created by the original steward and that was
the case with the Joint Commission measures? We were the measure steward but
somebody else did the eSpecifications. And then now who owns those because we
didn’t do the e-specs but we are responsible for the actual measure itself that
is derived from the medical record abstraction process. And where is the locus
of accountability nationally to fix these problems and to make sure that they
are getting the attention?
We will be raising these things to ONC and others, but I wanted to put them
on the plate because it doesn’t make sense to have really good clinical
measures and then find out when we get to the measure collection process that
we are not getting the information. Some of the clinical nuances that we really
need to make sure that the measure is accurate, maybe lost.
I am sorry I really raced through that.
DR. MIDDLETON: That was terrific. Thank you very much, Margaret. And we will
take a few minutes for questions now.
DR. TANG: Thanks, Margaret. Can we drill down on this question because that
is actually the question we want to ask is how do we use new sources? We are
technology agnostic, but how — assuming that these data exist what would the
desirable measures look like? Your question about how you lost some of the
integrity when it got converted certainly concerns me. I think there is a
little caveat though, for the short term because of the meaningful use criteria
that have to be put into place yesterday and then now tomorrow. They did
retooling, so-called retooling of existing measures and I can see how there is
some fidelity that could have been lost. Can you go ahead and go through the
VTE example so we know what was lost and can imagine or see how we would deal
with in the future how we would not lose that. Presumably some of the issue is
not wanting to continue with the manual review because that is high cost and
you can’t scale that and the other piece is the number of exclusions that are
in these definitions that contribute to the high cost.
MS. VANAMRINGE: There are so many exclusions but I think also there is what
we see often being lost is contraindication information of that kind of
clinical nuance. I am not the expert at the Joint Commission I will tell right
now on the weeds and the measures. When people talk to me at the Joint
Commission, I think they are talking a different language than I speak. What I
would love to do is get back to you. We do have a list and I think some slides
on the particular things that got lost in VTE and stroke and if I could submit
that to the committee, I would love to do that. But I may not be able to go
down too much in the weeds on the specifics.
DR. TANG: That would be very helpful. Thanks.
DR. MIDDLETON: If you were supplying that information in another deck or if
there is another way to get this is a related question. I guess if you think
about the measurement processes and the evolution towards standards-based
measures and then eSpecifications which has a very rigorous specification, if
you will, with current terminology standards and the like. What would be your
preferred standards for those measures? And then when you think about that, how
do we trade off what is available in the EMR again in a technology agnostic
way, but some things will be there and some things won’t be. How do we get the
information that is not from an IT source into an IT source?
MS. VANAMRINGE: I think there are probably ways to map some of the
information. If some of it is in administrative data but we don’t have a way to
actually get it so that it can be linked into the value sets that are in LOINC
and SNOMED. We could probably do some kind of a translation to get that done. I
think it is an eminently solvable problem at least for some. I don’t believe
there will be perfect congruence, but we may not ever be able to live with
perfect congruence in the near term. That is something that — there are
tradeoffs, but our think our concern is that we don’t know right now unless we
start evaluating this how much did we lose and where are those bias direction.
That is our concern and we believe there needs to be some investment in doing
this testing and then seeing if different mapping strategies will get us to a
higher level of fidelity.
DR. FITZMAURICE: Mike Fitzmaurice. When you get down into say the weeds,
does the Joint Commission specify the vocabulary for the quality measures, that
is, for the variables reported the name, definition, how it is represented? You
might say here is a set of ICD-9 codes that we want included, but these ICD-9
codes should be excluded and maybe it is the same thing with some procedure
codes. These count. But if somebody has had this procedure could, we don’t want
to count them in the denominator there in exclusion.
MS. VANAMRINGE: Yes. That is absolutely true and that is one of the things
we could map actually is the ICD-9 codes and make sure that what we need in the
measure gets mapped and see how close it could be to one of the values that is
in there already and most of the EHR system. Absolutely. We get down to the
specifications. We have data dictionary. We spend a lot of time on that and I
think that is one of the areas that we can be very helpful in doing —
DR. FITZMAURICE: A second question. Are you preparing for ICD-10, that is,
here is what we have for ICD-9. Now if we were to go to ICD-10, here are the
codes that we would include and exclude.
MS. VANAMRINGE: Absolutely. We have been doing that for a couple of years
now.
DR. FITZMAURICE: Very good.
DR. GREEN: Margaret, thank you for really clear presentation. A quantitative
question. What would you estimate to be the total number of measures that the
Joint Commission uses to assess clinical performance across the continuum?
MS. VANAMRINGE: Across the continuum that is — we don’t have measures right
now that go across the continuum. That is part of our concern. Our measure work
has largely been in hospitals and then the other areas where we have measures
are, for example, primary care stroke certification where there is a
measurement requirement specific measure. But many of the other programs were
using measures that were already other people’s measures such as the
MDS-derived measures for nursing homes, oasis, for home care. We have not
really created and this is what we are looking at right now for accountable
care organizations. What measures do we have to create that really go across
this continuum and we are also looking in the post-acute area not necessarily
for ACOs, but just what can we look at in that area. We don’t really have good
measures now.
DR. GREEN: I think the answer to that was zero.
MS. VANAMRINGE: I would have to say zero if it were me. Yes.
DR. GREEN: So let’s back up a little bit. Do you have any opinion or insight
into of all the measures that the Joint Commission uses that are site specific,
location centric? Do you have a guess about which ones are applicable in other
settings or cross settings?
MS. VANAMRINGE: Cross settings?
DR. GREEN: Or is this going to have to be a slug fest place by place, site
by site?
MS. VANAMRINGE: Yes, I think it would have. We are just beginning to pull
that kind of thing together. It is very difficult because now that we have a
certain inventory of measures we are beginning to look at where do we apply
them and to what type of organization. But I would be hard pressed to give an
answer to say how many in that category.
DR. MIDDLETON: Two quick questions. Matt.
MR. QUINN: Something that as we look ahead to things that — I have used you
guys as an example several times, as someone who has done a really good job in
specifying measures so that they are applied consistently. Back in 2001 and
2000 I was a product manager of GE’s core measures in ORYX stuff and I went on
to Quantros who does a whole passel of that stuff today. And one of the things
that was really refreshing as a product manager was that we were given specific
ways of implementing those measures in the systems down to the very detailed
level and there was a testing system that accredited and certified that we were
measuring things consistently on behalf of our clients.
Two thoughts. One is my product development folks said we currently measure
some of the core measures slightly differently in our system that is used for
quality improvement internally. Should we change it to reflect the core
measures? And I said yes, absolutely, even though that meant it was different
than what it had been in the past and we had a lot of benchmark data on it.
The second was the question is all of this going to be electronic data that
is supporting these core measures. And at the time even though they were
administrative data that we were sourcing, we had to create a system that
allowed people to go collect data from paper charts and actually enter it into
the system. There it was electronic in some form, but it looked both at that
and I would just propose that as a model that could get us over the hump on
electronic health record data in addition to this.
MS. VANAMRINGE: I agree. And I think you raised the point of data collection
standardization. That is really critical. When we originally started out with
our perinatal measures, we saw that you could specify numerator and
denominator. You could have a good data dictionary, but you actually could
collect the same data about 20 different ways and that does not make for I
think reliable information which is one of the reasons we are concerned about
when you start getting into accountable care organizations and other delivery
reforms. How are you going to make sure that those data are collected in a
standardized way?
We would also like to be able to take many of the measures that are coming
out of other sources whether they are state-based reporting resources and HSN,
other types of data collection sort of tracks registries and so forth. But the
issue still comes up what about validation. What about data accuracy and how do
you really assure that if you are going to have a program accountability where
somebody’s finances and reputation are held to a high standard on those
particular things?
MR. QUINN: Just as a super quick follow up. One of the things in the model
that I described with core measures is that there was vendor intermediary in
that and as somebody who saw what was coming from hospitals towards this and it
wasn’t consistent and it wasn’t — many times there were many iterations in
going back. Do you think that that is necessary if we are going to have quality
reporting?
MS. VANAMRINGE: We have talked about that a lot within the Joint Commission.
The vendor system has evolved over time. There are fewer vendors than there
used to be. But right now we feel for data accuracy that is the best thing we
have. Eventually obviously everyone wants to move to a different system, but to
me I see very little attention paid often to the data infrastructure in this
country and how are we going to get information especially if we are going to
start aggregating from different sources in a way that is consistent and
reliable. I don’t think we do nearly enough data validation. At least the
vendors are held to data collection methodologies and they have to be audited.
I feel that right now information that is coming into perhaps value-based
purchasing and meaningful use needs to potentially have a greater audited
capacity.
DR. MIDDLETON: Just one last question. I don’t feel we can leave your
presentation without the following. In a way we think about accreditation and
all the usual process outcomes manners that you have been discussing. And I
guess when we think about where meaningful use is going with increased
attention on clinical decision support, do you feel the Joint Commission will
ever be looking at the quality of clinical decision making itself or perhaps
secondarily the quality of the knowledge bases underlying clinical decision
support systems perhaps warding off the FDA investigation around regulatable
devices and what not? What would be your —
MS. VANAMRINGE: I would say more the latter than the former. I don’t really
see us evaluating individual clinical decisions, but more sort of the
infrastructure below that and what goes into that and I think clinical decision
support is just critical and ways to really improve that. I see us perhaps
getting more and more into that area as we look not only solutions for health
care organizations, but we are looking at where the failures are and often
there is an information failure.
DR. MIDDLETON: I am sure we could go on. But thank you very much for a
terrific presentation. Welcome Dr. Kevin Weiss. Blackford Middleton. Nice to
meet you. And why don’t you go ahead and take it away if you are ready.
DR. WEISS: Also I see that you have Dr. Lipner joining you today. Has she
presented — I don’t want to preempt what she might want to talk about with —
certification and board certification. And I have my presentation ready to load
up. However, it may be more helpful to maybe just give you the presentation for
the record and actually just enter in a conversation with you, but I can do
either way if you would like because you may have some very specific questions
about this whole credentialing environment as seen by the physician community
that you may get a little bit of when Dr. Lipner presents to you particularly
— I am assuming you are going to be presenting from the ABIM sort of
experience of that maintenance certification. Great. Would you be willing to —
you want me go with the fall presentation and leave that here or do you want me
to actually maybe just give you a more —
DR. MIDDLETON: Why don’t you go ahead and step through the presentation and
be use as much of it as you would like, but I also think the conversations that
we have had all day long have been extraordinarily valuable. We have time for
that as well.
DR. WEISS: I will step quickly through the presentation and then I will give
you some time just to maybe cue up some thoughts. I just want to start by
saying that it has been — for me this is kind of nice. It has been about 20
years since I last spoke in front of this committee that you are now members
on. I did last time when I was an NIH fellow here at the National Center for
Health Statistics. It has probably been about 20 years since I have been back
in this part of time. It is kind of a treat to be back here — big round
circle.
I come to you in my role as President and CEO of the American Board of
Medical Specialties, ABMS. I assume that what we can do is — if I say next
slide or something like that. Just to give you a sense of that framework which
I think most of you know but at least you would see it from our perspective is
we have the regulatory side, the legislative regulatory sense of physician
licensure which are the state medical boards and then we have this national —
and those are state based. And we have this national program of credentialing
that was started over 80 years and ABMS is now 75 years old, 76 years old where
the profession said it needed to create a cross the board standard that it
could use to define for the public what defines specialty medicine,
specialty-based medicine. And over the years accreted to the American Board of
what used to be called the Advisory Board of Medical Specialties eventually
became the ABMS, American Board of Medical Specialty. From a looser affiliation
to a more formal affiliation with standard setting obligations the ABMS was
created and these are the boards that exist in the ABMS.
ABMS now through its 24 boards oversees the certification and the continual
recertification of 750,000 physicians in the United States which is about
somewhere about 80 percent of the doctors who are practicing right now carry
one or more certificates from ABMS. We are now the only certification branch
that has a national program. The osteopathic association, the AOA has their
version as well as there are scattered — I think we have lost count at 50 to
100 other individual places where you can get a certificate that would give you
a sense of being board certified, but it is not an ABMS process for sure.
I like to use this framework when I describe the world that we live in
versus what else is going on in terms of the accountability and transparency in
health care quality. It really shows what I call the three tectonic plates of
accountability that are going on. One your left side is this physician
accountability both driven by both the regulatory which is the licensing
boards, but also the self regulatory, the physicians. Now mind you the
licensing boards are predominant physicians often have public members. The
certifying boards are predominantly in some cases near exclusively all
physicians.
And then we have these two other accountability frameworks which we are
going to talk about both the purchaser and the consumer and I would like to
talk a minute about both of those because that is what is colliding here and
the need for data underneath all this is colliding as well.
For the new world with these tectonic plates is kind of evolving. It is
being driven by purchaser and plan needs driven by the big purchasers including
the Federal Government. If you tape once, you have this whole consumer
reporting health plan reporting and part of what you have been talking
throughout the day on these. And then what you have on the next is maintenance
certification. Maintenance certification started behind that firewall that I
showed you in the last slide, but really is now beginning to reach out through
interest and transparency to serve these needs of the market place in better
ways.
And right behind us is as of spring of this year you will see maintenance of
licensure which now is official. Have you had a discussion from anyone from the
Federation of State Medical Boards? MOL, I will talk with you a little bit
briefly. Kind of view it as a program that is very reminiscent of what I will
show you for MOC, but a little bit lighter in terms of its obligations and
there are obvious reasons for that.
On the purchaser accountability you probably know this better than I do so I
am just going to blow through this next slide. I think there are like four or
five bullets and if you can stop before you tap again. It is all about this
report card. It is the idea that you can actually aggregate data and create a
performance measurement, numerator, and denominator. It can meet all that
performance requirement for the science of performance measurement and you can
report that back. And the purchaser has been doing this now driving the
insurers and other people to do it. It has been administrative. It is what
Arnie Epstein likes to call munching on administrative data, but really nothing
more. We need to clinically enrich it in some fashion. They have put on
hemoglobin A1c and blood pressure is attached to it. But it is a very fragile,
rudimentary sort of system to actually look at medicine.
The other problem is the lack of uniformity which you are all familiar with.
I imagine is part of the interest of the committee. And the fact that the
robustness of the data to look in granular fashion you just can’t get there
because when you start drilling down individual physicians, you really start to
get into numbers problems which drive any statistician crazy.
On the Federal Government you probably know this better than I do, but we
have all the versions. But for the physician it is really resting around PQRI
which started out as a truly voluntary program, the physicians voluntary
reporting issue of PVRI. It became PQRI. Incentives are positive. They are
going to negative. Eventually you are not going to make as much money if you
don’t follow them. But the really limited data and CMS is struggling in a very
positive way to try to make this more worthwhile. The physician community has
not bought into it because they don’t believe in the data. The entire surgical
community doesn’t hold ownership to it because it has these skip data which are
really evidence based but don’t really lead to individual surgical
accountability. There are lots of problems with it. They are trying to fix it
by putting in registries. But ultimately it is this carrot that is becoming a
stick that is going to drive people to it even though if it is an albeit
imperfect system. We are beginning to align with it.
I think there is one more bullet here. And this is what happened in the past
4 or 5 weeks which you may or may not closely aligned with and that is CMS led
a series of contracts, five of them, which are to develop efficiency measures
and that was a big release into the world of performance measurement. ABMS is
working on one of those contracts to develop those measures working with
Brandeis University and the AMA’s PCPI.
Consumer accountability is what it is all about because I think so much of
what we are doing in performance measurement rests on that. The boards are
beginning to try and figure out what does it mean to be more transparent
because historically we have said board certified, yes or no. Now we are
talking about board certified and participating in MOC, yes or no. Is that
enough for the public or does the public need more information to make good
decisions? Do other people need more information? That is a key point I will
come back to at the very end of this presentation.
What we find from consumer surveys done by Gallup poll and such is that the
public actually think that board certification is really important. They tend
to want to have a doctor who is certified. They say that they would leave their
doctor if they weren’t certified, but they don’t always know if the doctor is
certified. It is kind of a confused moment. And if you ask them and we haven’t
done this yet, but we will on new surveys ask them. Do you think that your
doctor is recertifying? They are going to assume yes which is a big assumption
because only about half the doctors in the country are actively going through
recertification activity.
We are left with the consumer movement that I like to think of as the wild,
wild west which is any information is good when you haven’t got good
information. You just suck up anything and you end up with things like ZAGATS
which is being supported by one of the health plans which will go unnamed as a
way to support consumer information. It is rating your doctor kind of thing. A,
B, C, and D. Now the boards have been struggling with this issue and should we
as part of a transparency talk about rating doctors. I think we keep on coming
back to a clear no. And the reason is there aren’t enough doctors in the United
States and if you start taking doctors who are just below average and telling
people they are just below averaging, you are going to be driving more and more
patients to a smaller and smaller workforce that is going to drive everyone
crazy. What the answer is the way to solve the problem of quality is the old
adage to raise boats, raise the tide. I didn’t do that right, but I think you
kind of know it. Rising tide raises all boats I think is how it goes better.
This doesn’t work for us.
We began to form — we, being the boards, began to formally explore what it
means to take that physician accountability plate and start talking with these
other two plates because we had a very large meeting with NQF co-sponsorship
about a year ago and the results from that will — the proceedings from that
are going to be released in the next month or so. I would be happy to forward
those to you. But basically they said and this is the quotes from — not formal
quotes from Janet, but I will paraphrase it. That there was real clear
understanding that the credentialing environment created by the boards is
essential to health care quality in this country, but it is insufficient
particularly if we don’t align to the other activities that are going on. If we
have a misaligned plate in those three plates, that it not helpful to the
public. On the other hand, the other two plates really can’t work without us.
It is really kind of a new dialogue, new day.
And then of course health care reform really does drive it home. You
probably know these again better than I do. These are the five sections of the
ACA that talk about quality in one way or another. This is the first federal
health legislation that really does tie quality into performance in the health
care system and they do it in a number of different places in here, one of
which is related to us. I will get back to that shortly.
What do we do on the credentialing side and what do we do on the licensing
side? Well, the Federation for State Medical Boards is a similar organization
to the ABMS except it doesn’t set standards. The FSMB brings their state
licensing boards together and suggest best practices and they have suggested
one now call Maintenance of Licensure. I will get back to that.
ABMS, on the other hand, brings our boards together and has the authority
from those boards to actually set standards. For example, the biggest thing
that ABMS does is it states whether another board can come to be or not,
whether specialty can come to be. It also set standard for the recertification
program which is very important as we go forward.
Maintenance of Licensure after what Hank Chaudhry who heads the FSBM, my
counterpart of the FSBM — so to speak states that after a 7-year labor and
delivery they have delivered MOL, Maintenance of Licensure. And that happens
spring of this year at the FSBM’s annual meeting. They voted it in. I believe
it was unanimous or unanimous minus one territory. I can’t remember which one
it was. Overwhelmingly support.
It has three parts and you will hear me talk later about MOC having four
parts. The three parts have to do with education, some sort of objective
assessment, and then some sort of practice improvement activity. Education,
which they are translating into CME, some sort of objective assessment, which
they haven’t figured out yet and then the third has to do with performance
activity, improvement activity, which probably is more like a PI CME, practice
improvement CME activity.
The next one is what is MOC in that light? Well, MOC is four parts. The
first is making sure you have a valid license and now emerging, developing
standards of a patient in peer survey that will be appearing as part of a
professionalism piece. And then it has education. An objective assessment,
which is a secure proctored exam. And then it has a practice improvement
activity that is more formally defined. The three parts of MOL and the parts
two, three, and four of MOC are the near equivalence except the standards for
MOC are set much higher than they currently are for MOL. It is a higher bar
that we have currently.
Here is what MOC is. It is maintenance certification program. It was set up
to promote lifelong learning for the specific point of improving patient care.
And then we have these six competencies. It is not just knowledge. It is not
just practice. It is also these other element system-based practice and
professionalism. Elements of care that really comprehensively define what you
want to see in a good evaluation of an individual physician.
The four components I went through. And if you look at components two, three
and four which are listed there, that is the same if you would renumber them as
two becomes one, three becomes two and four becomes three of MOL near
equivalency. Again, the bar higher right now in MOC than it is being set
currently in MOL for lots of obvious reasons.
The question is why four parts, and why not just do performance measurement.
Why not make it easy and do what NQF is suggesting we just take the performance
measures and bring them into practice? And the answer is you can’t evaluate a
doctor and performance measurements alone. I can do as a primary care doc you
can measure my hypertension and you can measure my diabetes care, but to get to
thyroid nodules on me in terms of do I know what to do with thyroid nodules,
you don’t have the numbers. You don’t have the denominator to get your
numerator on that. If you want to know how I manage stiff neck and fever, it
isn’t going to happen through a performance measurement. The frequency is too
low. You have to find other modalities for assessment of a doctor and to get
performance in it. In addition, you are never going to measure professionalism
through a performance measure that looks like an NQF measure. You have to look
at different ways.
The other piece is you need to know that the doctor has a certain amount of
core knowledge and knowledge we know decays with time at least my knowledge
decays with time, unless you keep on refreshing it and refreshing it and
refreshing it. This comprehensive evaluation of physician is where we are
after. One part of it has to do with the performance measurement work that you
spend a lot of time I would think considering.
I wanted to show this slide because it says we are only 10 years young. It
was 10 years ago that we decided — we, the board, decided to do this complex
thing. It has only been about 4 years since all 24 boards have engaged in their
first cohort of doing it in terms of actually implementing it. It is a very
young science on our part and we are learning and one of the criticisms people
have of us is it all looks very different from board to board. The answer is
yes. We are in a very rapid experimental cycle which you will see from Rebecca
very soon for Dr. Lipner is going to be a really nice view of what ABIM has
done. I think they have really shown the way, but there are other great
examples as well.
And then I want to just for the last couple of slides to get us right into
the question of data which is — where the committee might be focused is this
part four. I wanted to wrap this in the knowledge that just doing performance
measure data and getting clinical data out is not going to be enough to certify
a physician. It takes more. But you still need it to certify a physician
because you need to know how a physician is doing and the common things they do
and are they doing them well as best we can measure them with flow of data.
We now have in this thing called part four which is practice improvement
four basic pathways that are emerging. The first is the practice audit which
means in simple terms that physicians go into their practice, they pull charts,
they abstract data, clinical data most commonly, usually 20, 30, 40 records and
then they enter that in. They do a practice improvement activity where they try
to improve the care based upon feedback and then they do that a second time.
And the PIMs, the practice performance improvement modules or is it practice or
performance. Practice improvement modules that you will hear from maybe ABMS.
It is just a quick essentially wonderfully good example, a high good
science-based way of doing it.
Next is the outcomes assessment through data bases and one of our boards,
the thoracic surgery board, uses the participation in the STS registry which
you I am sure know. If not it is very easy to sort of — this is the
cardiovascular database that is maintained by the Society of Thoracic Surgeons.
There are a number of other boards who are building their own little registries
or working their societies to build a registry-based data to get clinically —
direct clinical data in so that they can evaluate on an ongoing basis in a
series of patients.
The third which is a brand new area that we are working on and it probably
has a lot of meaning to even the Joint Commission is that we are working to say
can a group of — that there is a lot of improvement activity going on where
physicians participating in a group activity. You can identify the physician’s
participation, but it is being driven by a group activity. I think the nice one
nationally is the Cystic Fibrosis Foundation has for years had a cystic
fibrosis national registry and the centers all participate in it and they send
the data in and that data gets reflected publicly. And their centers are all
judged by lifespan and quality of life of kids growing older with cystic
fibrosis. And the physicians participating in that even if their practice per
se is not but they are part of a center and they are doing this actively. They
should get credit for that.
Now we are doing this more specifically at the institution level now as
opposed to just national. And the example that is being ruled out and Dr. Green
knows about it because it is American Board of Family Medicine is doing this
with the American Board of Medicine and Pediatrics and we are now scaling it up
is at Mayo Clinic where we are actually saying if Mayo is doing quality
improvement activities can physicians who are doing those who have recognizable
activity in those use the data from those improvement activities to be part of
part four activity. And that institutional recognition of what I am calling
institutional accreditation for part four activity is when we are rapidly
trying to explore and evolve.
And the final one which shouldn’t go unnoticed but it is one that only
represents a small number of boards is where you don’t have patient care but
you have direct patient responsibilities so pathology, diagnostic radiology.
Those are where you actually want to know that someone is making sure that
those past slides are being read right and there is a peer-to-peer activity
that goes on. Actually with pathology it prescribed by I think it is CLIA. And
that is recognized by our world.
With all that in context what are the emerging data needs? It is all around
part four. We really have a good sense of what the other data needs are. And it
all rests on this very simple construct of patient-level data with
accountability to individual physicians. The patient-level data you spend a lot
of time talking about in terms of how do you get patient data, a good data out
of EHR. It is the second piece to this statement that makes it so important to
the boards in order for us to be able to tell the public what we need to tell
them and that is linking that patient data to a doctor and that isn’t easy. And
I think if there is one thing I can come out of you in terms of your thinking
here is no matter what you are doing in terms of trying to understand the data
and how to make a hemoglobin A1c look the same in all electronic health
records, how do you attach the ordering of that to a doctor or to another a
licensed physician or to another licensed health professional because that is
going to be key as you go forward as you make recommendations?
And it is not always easy to manage that accountability because of obvious
reasons. The downside is that is what the boards have to do because we certify
individual physicians. That is going to be key.
The other question in data is whether the member boards are finding their
data. Most of our data is not coming out of records through data entry as I
have mentioned. A little bit of it is coming through these registries and we
have almost none coming through electronic health records. The question to us
is as we start to get more open in transferring and moving data how do the
credentialing and the accrediting bodies get access to that data to do the need
that the public needs us to do and that is another key issue in terms of I
would encourage you to think about.
That is particularly true of the electronic health record data, but we are
going directly to patients now for the first time. Some of our boards have
already started this where they are actually asking for outcomes data. We are
starting with experience with care surveys, but it is not that far of a leap to
do what they are doing in the UK where these four conditions — I think I is
hip and — well, I know it is hip fracture where they are actually saying 6
months after a hip operation how are you doing on your hip repair. And we
should be doing that in the US and the boards will start to consider that and I
am hoping we will start to adopt that. It is not just experience with care we
will be getting. How do we get that on a more national basis is a question.
I think this is my pet ultimate which is what can we do to help. Well, I
think it directly relates to the last two slides so I won’t read this. And just
close by saying that it is easy to think that we can build a system, a
performance measurement and satisfy the need for public improvement and care
that somehow we can measure everything in a performance measure and get there
and that if we move the electronic data out of the electronic health record,
that is the answer. What I come to say is no. That gives you part of the answer
and it gives you part of the answer on the big, broad brush stroke thing. It is
those smaller brush strokes, those more subtle brush strokes, those low
frequency, high risk, intermediate frequency activities that have to be part of
the everyday diagnostics when you start talking about appropriateness and
misadventures in care that are just as important as those high frequency, high
priority items that we are all working to define for the national priorities.
And if we don’t make sure that the data stream is not just about those high
priority items, the hemoglobin A1c sort of speak, then we are going to miss the
boat. I will stop there. I hope this has been helpful.
DR. MIDDLETON: That was really terrific, Kevin. Thank you very much. Rich
slides and excellent presentation and lots of issues highlighted. Given the
time I just want to ask if there are two burning questions and then we will
move on to our next speaker. Paul.
DR. TANG: One question. On part four you mentioned the Mayo activities and I
think let me just if I got this interpreted correctly you were saying in a
sense participation in the Mayo activity could deem them in fulfillment of the
part four of your MOC, correct?
DR. WEISS: That is the direction we are trying to move. Yes.
DR. TANG: Likewise you might say it could relate to meaningful use in one
way or another in its later forms where we are measuring outcomes, et cetera
than if you do this then it shows that you have one and you can use it to
improve your practice, that kind of thing.
DR. WEISS: Absolutely. Now take one more step further and say that under —
as of about 4 weeks ago, and you may know this, we are under a contract from
ONC to develop MOC components both for part two, part three and part four. And
the intent of that is to build components that will relate to meaningful use
assessment. We will be over the next year or two developing formal projects in
part four that will look at the question of meaningful use. We will be looking
at part two in terms of what do people understand in their own self assessment
of meaningful use and then also developing items that will end up on the high
stakes’ exam that will question whether physicians actually know what
meaningful use is. We are directly going to be approaching that.
DR. TANG: I get that for part two. How does part four — I thought part four
would go the other way around like the Mayo serving as qualification,
satisfaction for part four of your MOC.
DR. WEISS: What we can do very simply in that is part of what I am calling
the accreditation standards for what an institution can participate. The
institution would have to meet certain obligations for all of its physicians if
we are going to participate in improvement activity. That is where that can be
inserted.
DR. FITZMAURICE: You had mentioned that low-frequency events are just as
important as the high-frequency events. Are you saying that the development of
quality measure that would look at rare bad things that happen and so you might
keep account of them and put that over some denominator that if it could be
reported could be accurate data could be an important quality measure.
DR. WEISS: Absolutely. And again I think the example that comes quickly to
mind which is the thyroid nodule. In an everyday general medicine practice you
may see one or two a month maybe, but yet when you do it you can really take a
misadventure in care if you don’t know what you are doing well. You can miss a
thyroid cancer. It would be a horrible thing. That is a feature of something if
you have that low frequency. But that persistent over practice you probably
start to have accumulative risk for seeing a thyroid cancer in your practice
probably every 5 to 10 years as opposed to diabetes which you are doing it 20
times a week.
DR. MIDDLETON: Thanks very much. Why don’t we go ahead and switch gears. Dr.
Lipner from the American Board will be presenting. I am going to ask the
committee members at least and consultants here indulgence to go over a little
bit. We are just running about 10 minutes behind schedule so if we could have
10 more minutes that would be perfect. Dr. Lipner, please take it away.
DR. LIPNER: Thank you. I am very glad that Kevin went first because I was
hoping that he would give you the big overview of the ABMS. What I am going to
be talking about today is really from the perspective of one of those 24 boards
that is working under ABMS. We happened to be the biggest board so we have
300,000 of those diplomats. But we have sort of entered this into what do we
need to assess clinical performance across the continuum in a different way
than all the other boards. We are sort of leading some of the research that is
going into part four particularly but also part three.
I just sort of put this out there. We all know that clinical performance
assessment is complex and as Kevin mentioned we look at it from the ABMS as
clinical performance is really a function of multiple competencies. It is not
just about those measures that you are measuring in practice, but it is about
your diagnostic reasoning skills. Can you come up with the right diagnosis? If
you look at a chart, you don’t know that they come up with the right diagnosis.
You just see the measures there. The clinical care is important: the outcomes
measures, the process measures, the mortality, morbidity.
Communication with patients and peers. How do you get along with your
patients? How do you get along with your peers? And that includes things like
team work, coordination of care. It is the big picture.
Are you able to work within a system and if you change systems, can you
adapt to the different systems? It is not just about the system that you are
currently working in but can you move to another system.
And finally, about professionalism. Do you come to the job with the right
characteristics and the right approach? Are you — simple things like do you
have disciplinary actions against you. Some basic things like that all the way
to sort of your ethical values and how do you work with others.
To do that, we obviously need different types of data and measures to assess
these different competencies. Now, they are not going to come all from EHR. I
would love them to come from the EHR. As a researcher it would be lovely to get
the data electronically and without a whole lot of effort.
But when we assess these we need to do it at multiple levels. We need to
look at things at the patient level, at the physician level, and at the system
level because they are all interrelated. For us to think they are not is really
making it much too simplistic.
We in statistics in my department is sort of what we call the psychometrics
department, but as statisticians, we note that patients are nested within the
physician. In other words, patients pick who they go to. That is an important
feature culturally as well as just where you live and in the kinds of patients
that you see, but they choose them. We have to take that into account when we
look our reliability, when we look at our validity, when we do our risk
adjustment, et cetera.
Physicians are also nested within the system they work and that is also
another function. There are some people and in some of our research we have
people in really great systems that aren’t doing very well. You wonder why.
They have all the bells and whistles, but what is going on there. It is more
than just the system. We know that there is a real big interaction between
these three things.
We need to make the most accurate decisions about physician’s clinical
performance that we can at this point in time. And I think things have gotten
better over the last 10 years and they will get better, but we need to do the
best job we can now and what we think of as moving the curve along. Let’s show
people where they are now and believe it or not our physicians on our board
basically say you show me where I am compared to other people and I am going to
want to move up because it is just that type A personality. We all want to do
better.
And we need to evaluate that performance improvement over time. As a nation
and as physicians as a whole, we need to look at that. Are we doing better over
time? Not just this individual physician but as a whole.
In terms of looking at the data obviously we need high quality data and a
lot of people have talked about this, but I just wanted to sort of call out
this little here is the standards that we use in our testing industry for
measurement and we really abide by that book. It is like our bible. How do we
know that we are doing a good job at the measures that we are giving you?
But we do look at it from three different levels. We talked a lot about the
data itself, the data elements, and the accuracy of those elements. Are they
complete? Do we have a lot of missing data in there? Is it filled up with the
right data? Are they comparable across systems even EHRs? Am I getting the same
things when I look at that data? And are they timely like are they old ones or
can we dynamically change them and what is in that? That is very important that
that is all accurate.
Now at the board we have assumed that what we are getting is that accuracy
although we know for sure that it is probably not. And right now at the board
we have collected data and asked physicians to collect it themselves. They have
pulled their own charts. They self report. And that actually is a very positive
thing. We thought they would really be annoyed with us to have to keep pulling
those charts, but the first 25 charts that they have pulled they said wow. I
didn’t realize this wasn’t in the chart. It is an eye opener. It really says
aha. Something is wrong with my bookkeeping. From that perspective we haven’t
spent a lot of time at that point that we recognize that it is really
important.
Then when you roll up those data elements into measures based on
clinically-based guideline hopefully, we need to assure that they are reliable,
valid, and feasible. Can we actually calculate them, get them? Do we have
enough patients per physicians as Kevin was mentioning to collect them because
if we don’t, that measure is not going to be reliable? And I will talk a little
about what we hope to do as more on a composite level where we aggregate
measures up to a composite because we don’t feel the measures in and of
themselves by themselves are reliable enough for me to make a decision about
you as a physician. Are you certified or not certified? We have really high
stakes decision that we are making and we need to make sure that what we make
those decisions and what we base those decisions on are high reliably and valid
measures.
The other thing — so I talked a little bit about this, the classifications
and decisions. We didn’t talk much about that, but for us that is really
important. For us it is whether you got certified or not. For others it is
whether you got paid more or not. There could be many levels of decisions that
you could make about a physician, but when you do that you need to have it be
very accurate because you will get people not believing in those as decisions.
The people that we think should be certified should be others that think should
be certified. There should be other measures to say yes. This is the right
person and yes, this person probably shouldn’t have been certified. We need to
make sure that those decisions are accurate. There are high consequences to not
being certified in this country. You can’t get jobs. You can’t get paid at a
higher level.
And finally we need to make sure what we base is on appropriate and planned
sample design so that — even though when we EHRs you will have lots of data
and you think you don’t need to worry about samples, but there will always be
measures where you don’t have enough patients during that annual period that
you are looking at. What do you do and how much is enough? How many patients do
we need?
I just gave some examples of the specific things that we need to do our
work. We have talked a little bit about these here in various different
settings, but we need the clinical data. That could come from EHRs. That does
come from the EHRs. We need patient data. We need the demographics of the
patient. Does the physician know the demographics of the patient or do we have
to get that from the patient?
The other week we asked physicians — we did a focus group and asked
physicians can you tell us the insurance of your patients like go get the
insurance? And they like had either no clue to where to look or just didn’t
want to. They didn’t want to go from the chart to the administrative data, but
somewhere you know they are getting paid.
Those kinds of things are really important in terms of potential patient
risk adjustors. Again, we have the debate in our board about should we risk
adjust or should we stratify. Should we show people how they are doing based on
certain patient populations or should we actually put the risk adjustment into
the measures? And that is an ongoing debate and it is sort of a little bit of
both.
We also think it is important to look at the patient self-care or what we
are calling here experience of care measures. We have found that those are
actually informative and helpful in predicting outcomes of care. We look at it
a little bit differently. We ask the patient and we do a lot of what Judy said
this morning, but we do a very specifically for that particular disease. We
will ask the patient did the doctor or somebody in the office tell you how to
check your feet, something really very focused and very actionable. And we are
actually putting those patient care measures into this composite, which is sort
of a little bit out there. I will show you that in a minute.
The physician’s data. Obviously we need basic demographics on the physician.
For us specialization is important. Are you an endocrinologist or a general
internist? What kinds of patients are you seeing and what kind of education did
you have when you got there?
Diagnostic reasoning is important to us. Are you making errors and you are
taking care of this patient, but for the wrong thing. We need to know that. I
mentioned the unrestricted medical license and disciplinary actions.
And then we need to know what kind of system you are in. You should see the
span of physicians meeting certification programs. It goes from the solo
practitioner to the people in Mayo clinics to group practices of sizes of 10 or
so. All of that is important in terms of us understanding the relationships
between all these competencies and between the patient, physician and the
system.
ABIM defines the field of internal medicine and these are the subspecialty
areas that we have now. We have general internal medicine. Everybody has to go
through and get certified in that, but once they do that they can actually go
on to fellowship training and get specialized in any of these areas. Right now
we have over 50 percent of our physicians is specializing in one of these
areas. We have a broad span of areas to deal with in our program.
These define the field of internal medicine because they are broad. We are
not talking about just thyroid disease. We are talking about a little bigger
field.
What we are dealing with is very different types of physicians, but I just
pulled out this slide to demonstrate to you that in our MOC program as of the
end of 2009, we actually have 16 percent of our physicians are still in a
practice of one physician and it is clearly defined, not working part time with
others, one, by themselves. And then if you look 10 or less we have 50 percent
of the practices are in 10 or less. It really still feels like we have a long
way to go to get to the accountable care organizations or a bigger picture of
pulling these physicians together.
We also have a variety of things that motivate physicians to participate in
our program and surprisingly this was a study that was done in 2006, but we
have repeated that study. We haven’t published it yet. But professional image
is really what drives them to the program at least this is what they say. As
you can see, required for employment and monetary benefits, again this is self
reported, is not as important to them as keeping up to date and doing well for
their patients. When we ask the question a different way of are you required to
do this for yours, we got like a 65 percent saying yes we are required for some
either a health plan, insurance companies. Somebody else is requiring us to do
that. But the idea here is that we have a wide variety of physicians and
motivations and backgrounds.
What we would like to do in thinking about assessment is we look at it from
and this is a very theoretical approach to clinical assessment, but sort of to
get you out of the mode of the outcomes and processes of care. There are many
different ways to assess and you can do some things. The bottom level is very
elementary. Do you have the knowledge? And the top level, can you actually show
it? Are you actually doing the right things in your practice? I will just talk
about these from a medical standpoint, but at the bottom level when you are
doing assessment, you can actually assess things in a broader way. I think this
is a little bit what Kevin was talking about. The things that you don’t see
normally in practice how the heck are you going to know whether if that person
comes to your office, do you know how to take care of them? One way of doing
that is by one of these lower levels of types of assessment.
When you move up the pyramid to the top, it is much narrower what you can
assess and it is less standardized. And that is what we are facing with all
these processes and outcomes of care. Everybody is in different types of
practices. They see different patients. We really can’t get very good
standardized measures. We are trying and we are getting there, but it is hard.
We can’t cover everything. We use sort of this spectrum of things to test
different types of competencies in different ways.
I will talk about two areas that we do at the American Board of Internal
Medicine and the second level is the diagnostic reasoning. We are assessing
diagnostic reasoning using clinical vignettes and that is what our exam does.
It is basically little case scenarios, snippets. Can you handle this? Can you
diagnose the person appropriately? And then we talk about the performance and
practice and —
For clinical diagnostic reasoning I am not going to spend a lot of time on
this, but why do we do this? Who cares? If you can take care of your patient,
why do you care about this? And one of the reasons is that you could test
broadly. The other is that there is a whole theory about problem
representation. If you have the right schema in your head to represent problems
and you have a really foundation of the way you think about diagnosis, you will
be able to come up hypotheses that are meaningful and not just gathered from
incidental information. You don’t just sort of pull the pieces together and go
online and look up. Well, it is this, this and this. What can it be? You
actually think about it in a very logical and representative way.
We think this is an important thing to test because if you don’t have that
and we have seen it where people can scramble and use resources, but they are
still not getting the right answer what is it that they are missing. This is a
really important thing in education. And Judy Bowen has a really nice article
in New England Journal that kind of describes that. But that is what we are
trying to do through the secure exams.
Going over to the practice improvement modules this is what we use to assess
performance and practice. And again right now it is all physicians pulling
their own charts. Eventually with the meaningful use contract that we have we
are actually going to put one of our practice improvement modules which we call
our comprehensive care practice improvement module into electronic forms so
that we can actually get data from electronic health records directly. We are
going to have a whole learning experience through that. It has been very
helpful today that we hear about that experience.
But we have approached this with is a three-pronged data collection. It is
not just about the chart data, the clinical data, but it is also about the
patient experience of data as well as the practice data. We actually use NCQA’s
practice survey in a lot of our practice information in our practice survey to
collect information about the practice. It had practically 100 questions in it
and we wanted to see whether it was related to anything else.
We take those three pieces of information and feed back in performance
report to the physician based on how they are doing and I will show you how
feedback relative to how everybody else is doing as well. And then they have to
do a quality improvement cycle based on something that they are not doing well
on. And then they remeasure it afterwards. We want to see if they have
improved. We don’t hold them to improving although all we hold them to at this
point is —
What we have and this is written up in two different papers. What we have is
a diabetes composite score which I think you actually have seen. I don’t know
how many of you were on this committee last year, but David Ruben presented
some of this last year. And basically the bottom line is that the composite
measure that we came up with is much more reliable than any of the individual
measures. And we make the classification decision based on that composite. It
is much more reproducible than a decision that you can make in any individual
measure. And we basically get fewer false-positive decisions when we do that.
We want to protect ourselves against certifying somebody who shouldn’t be
certified and not certifying somebody who should be.
The composites allow for a more comprehensive assessment. But performance
and feedback on those individual measures is still important. Even though you
say okay here is how you did relative to everybody else, you still need to know
where you need to improve. Saying it at this level but giving all the
information is very helpful.
And then we have done a classifications decision based on a very scientific
approach to what we call standards setting. We didn’t just have experts sitting
around the room and say 70 percent. We actually did a methodology that holds up
fairly well scientifically.
I won’t go through the developing composite measure steps, but we really did
base it on the data that we had in our program. In order for us to get our
expert panel to really believe what they were doing is they needed to see the
data. Tell me how people are actually doing on the measures.
We looked at 20,000 patient charts and 18,000 patient surveys so a lot of
data. We actually had more data than that now, but we based our data on that.
And we looked at individual measures as well as the importance of those
individual measures.
This is kind of what you saw last year. It has changed a little bit I think.
This is again for diabetes’ care. These are the intermediate outcome measures,
the clinical process measures, and we have patient experience measures in
there. It is part of our composite at this point.
Any one of these measures by themselves we didn’t feel as reliable enough to
say you pass or fail on it. We ended up coming up with thresholds for those
individual measures and important weights for those individual measures as
well.
What is different about our composite than others is that we give people
partial credit for doing well so that not everybody is like in or out. It is
not an all or none. We actually give them something. If most of their patients
are doing well in something, they should be recognized for that. We give them a
partial credit.
It is though a compensatory model. You can make up for something. If you are
not doing well in one thing and doing well in another although these things are
so interrelated as you know in medicine, you can make up for doing bad on
something.
What we did notice is that you can’t make up — you can’t kind of pass this
thing if it is all about patient experience measures. And the patient
experience measures by themselves did not hold up really well by themselves but
together with the rest of it it did hold up.
DR. TANG: (Inaudible)
DR. LIPNER: It is the percent of patients that need to meet this goal.
DR. TANG: And literally you said a threshold for LDL less than 100 up to
23.8 percent.
DR. LIPNER: Yes. Let me just sort of take a step back. We started with what
we were calling tell us about the competent physician and we set the —
purposely have the group think about it with a very low bar. Who would you just
like not have your mother go to? It was purposely set that way. Now it doesn’t
have to be. You could change the mentality of the group and say an average
physician, an excellent physician. But we purposely did this because we thought
it would be less threatening and it would get physicians engaged in the process
of this is easy to do and they will see that they can actually achieve this.
We actually haven’t set this in our program. All it is is performance
feedback at this point. But it was based on the data and surprisingly people
were doing pretty badly in that measure. They sort of came up with a value
without the data and then looked at the data and adjust it.
DR. TANG: So if you don’t reach 23.8, you get 0 for that row.
DR. LIPNER: No. You actually get something, but very little. You will get
something.
This just shows you a particular doctor and their performance and how they
would do. What we did do for clinical process measures is what you just said.
We said you have to achieve at least this much. We couldn’t justify giving you
anything if you didn’t meet that minimum threshold.
How did we do in terms of accuracy of our classifications for this
competency measure? We said these people are competent and these aren’t. And
basically this is — I don’t want to get into all the statistics here, but at
that dashed vertical line is where we set the cuts for. And there out of one we
were at .98, very high classification accuracy which is really, really good. We
felt really good about our composite. And partially we are there because it is
such an easy measure. It is such an easy thing to achieve and most people are
achieving it. As you can see if you set the bar at different points, that
reproducibility or that accuracy would change a bit, but still be pretty good.
And this is sort of what we came out with for those 957 physicians and I
think David showed that to you last year. But the idea here is to give feedback
to physicians. Where are you on this curve? Dr. Smith was at 66. He is not even
at the mean. Dr. Smith would say wow. I need to improve. I am not even at the
average.
But we also wanted to give them an idea of how bad is bad and we said
anybody below here really doesn’t meet this very minimal standard. And we only
had 4 percent of our physicians in that category and these were volunteers who
did this. When we actually did this again with non-volunteers, it was a little
bit higher.
And then we are giving feedback based on the individual measures as well and
we are dividing it into these quartiles to just give them a sense of where they
feel on these measures. We are also thinking about stratifying them to
comparing them to physicians like themselves, which I think will be very
helpful for them.
All in all this worked out really well. The other thing that I just wanted
to point out is the composite score interpretation was very valid. What we
looked at was endocrinologists did perform better than internists, which is
what we expect. Those classified as incompetent scored lower on their
diagnostic reasoning exam. There was a significant difference. They also have
low overall ratings in residency so from their program directors. There was
something about them that was different. And they were also more likely to be
in solo practice. It was sort of like these were things that really stood out
that made us feel like okay. It is working in the right direction.
Moving forward and this is sort of what the meaningful use grant is going to
be working on is the comprehensive care plan where we are now going to take the
challenge of not just diabetes but looking across the spectrum of general
internist practice. And we have already done a study on this where we had chart
abstractors abstracting the data. And I won’t get into the details of it other
than — let me just show the summary in the next slide. The complexity
increases across multiple conditions. But measurement of chronic disease care
and preventive services was feasible. Not every single bitty thing, but on a
general level we got enough there that we can get enough charts and enough
data. Acute care conditions were not well documented and we really couldn’t
measure them at all. And again performance of practice here was correlated with
diagnostic reasoning.
And finally basically high quality data is important and for us access to
the actual broad data elements is really critical mostly to continue doing the
research that we need to do to understand the interrelationships. If everything
is rolled up in measures and measures keep changing over time, we are never
going to understand how these things interact with one another. And obviously
the connectivities to the electronic databases are really critical as well.
DR. MIDDLETON: This is really terrific and I am sure we can go on for a long
time, but we are at the end of the day. Let me ask if there are three burning
questions.
DR. TANG: One, you answered the question that I asked David a year ago.
Thank you. I got my answer now.
Let’s go back to the purpose of this because I find it very helpful and I
want to do some offline stuff with you in meaningful use. But what do we take
away on the national quality measurement roadmap? I understand the parts of how
you assess everything from the competency to the practice of an individual
physician. How do we relate that to national quality measurement?
DR. LIPNER: It is a question that I don’t think there is a great answer for
it. I think the thing that I think we have to start with is reducing redundancy
between all the different players that are working on the same thing. I think
that is — as I sit here listening and have been in many other places,
everybody is approaching things in different ways and that is great. But
bringing that research together and really trying to understand. What do we
know now and what can we implement? We are looking for a patient experience of
care of measure. Can we come up with one that we could all be happy with or can
we come up with a variety or at least constructs that are always going to go
into these things?
Similarly with I think with the electronic health records we are a little
further along because I think the NQF has really been and we looked to others
to develop to give us the measures. We really don’t want to create the measures
ourselves although when we started this a number of years ago there just
weren’t measures in some areas and you saw all the disciplines that we had. I
think that is one that we have to also focus on, the too hard box. What can’t
we measure currently that maybe at some point we can measure and how do we
collect the data to get there?
DR. TANG: So it is a little bit of begging the question it sounds like. We
all agree that we want access to raw data in real record systems. And I suppose
the only thing and this may be similar to the conclusion we had last time under
meaningful measures. Would ABIM and ABMS be willing to be at the table since we
all want the same thing and we don’t want to duplicate work? Should we all just
work together and find a forum to do that now that we have a much richer
sandbox? We didn’t have a sandbox for any of us. We all did our own thing. Now
that the sandbox is being built should we work together on figuring out a way
not only for common measures, but sort of cross demeaning and that kind of
thing?
DR. MIDDLETON: One thought you mentioned NQF a couple of times and I think
the current work underway there in terms of utilization measures, measures of
CDS and structural measures of HIT adoption that framework of course might be
amenable or lend itself to some measurement framework for cognitive services.
That would be very interesting to pursue building upon that infrastructure.
DR. GREEN: I want to thank you for coming and also you, Kevin, for taking
the time to do this. And I want to take us back to the topic of the hearing
about developing a quality measurement roadmap that considers future
information needs and data sources. Just notice that this is an addition to the
data sources that is still immature but extremely promising, and it ties back
to NCVHS’ work in a couple of powerful ways in my view. A big one is those
years that NCVHS spent on looking at reuse of data. Collect once, use often for
multiple purposes, and that is coming back, I think, to what you are calling
out, Paul.
Another aspect of why this is an important thing I think for this
subcommittee and the full committee to consider, is what you showed us at least
three times in your slides in different ways. But it is the reach and to what
is actually the big platform with health care delivery. The small physician
office dwarfs all other. It is not a competition. We can have the integrated
delivery systems that we want, but like we heard from HRSA at the end of the
day is 16 million people. We heard from HIS at the end of the day. It is 1.8
million. What she just talked to us about starts touching almost everybody. And
this really matters to our objective here is driving quality improvement and
quality measurement all the way right to the front lines where communities meet
medicine. This is a lovely set of presentations.
With that said and I don’t want to get myself into trouble here with
anybody, but it still is an immature enterprise and we have miles to go and we
have much to learn about how an MOC performs. With that said there is a lot of
progress. A couple of examples to try to provide some evidence to support that,
The American Board of Family Medicine is a CMS registry, already made that
step. Already report once, use it three times. Get your CME credit as a
physician, do your board work and do PQRI. One quality improvement process. The
clinical community simply loves that. Just loves it.
And these questions we had about the Mayo model and that Kevin answered.
There are a lot of those experiments now that are launched and there is going
to be a lot of more. Some of them are university based. Some of them are health
professions association based. But these are destined in my view, to become
another data source and we ought to put it on the roadmap.
DR. MIDDLETON: That was the right way to end the day. Thank you, Larry. To
set the challenge for us. Thank you, Dr. Lipner and Dr. Weiss again. What a
terrific day. Three panels – one more to go tomorrow. And thank all staff and
support of everybody for staying an extra 15 minutes. We will see you in the
morning. We are adjourned.
(Whereupon, at 4:26 pm, the meeting was adjourned.)