[This Transcript is Unedited]

National Committee on Vital and Health Statistics

Quality Subcommittee Hearing on Measures that Matter to
Consumers

February 28, 2012

Doubletree Hilton Hotel
8727 Colesville Road
Silver Spring, MD 20910

Proceedings by:
CASET Associates, Ltd.
Fairfax, Virginia 22030
(703)266-8402

Table of Contents

M O R N I N G  S E S S I O N

Agenda Item: Welcome/Introductions

DR. MIDDLETON: My name is Blackford Middleton. This is the National
Committee on Vital and Health Statistics Quality Subcommittee hearing on
measures that matter to consumers. I am the co-chair of this committee with
Paul Tang. Good morning.

So thanks to everybody for coming and we have an exciting day and a half or
so prepared to review a great deal of testimony from all of you, and to learn
about how to address some of the issues pertaining to measures that matter.

In many ways, this is another step on the long journey for the NCVHS Quality
Subcommittee on defining a roadmap hopefully that is going to lead to better
measurement, processes and infrastructure and measures and outcomes, of course,
for health care in this country, both for disease care and health-related care.
We build actually on the testimony and letter we wrote approximately a year
ago, where we addressed the issues of aligning quality measurement with needs
of health reform.

At that time, we really were asking the question of what was the future of
quality measurement, analyzing measurement activities from consumer
perspectives, provider perspectives, professional organizations and
accreditation organizations and regulators, as well as payers and group
purchasers.

And we really chose that topic then, to try to look at the quality
measurement and assessment process in a more holistic fashion, and identify
needs for effective quality measurement, measurement that is central to the
care delivery process, where feedback is built into the process, understanding
the types and sources of quality data and the increasing amount of data, number
of data, that come from health IT environments, increasing reliance of
healthcare reform on quality and quality assessment and value assessment, and
increasingly focusing on the patient experience and value, and finally making
this all part of built upon and built into the health IT environment.

We had a wide variety of stakeholders testifying, and in conclusion from the
testimony, we made the following assessments. There was a significant gap
between currently available quality measures and the needs of key healthcare
stakeholders, number one. Secondly, there is a need for a shift in the way
measures are focused and developed across the various stakeholder groups. And a
failure to correct the current path would result in a massive effort and
expenditure of resources on measurement activities that do not progress toward
health reform and are not useful in supporting improvements of health care
quality and value.

So what we recommended then is the following four things. We recommended
prioritizing creation and funding for development of measures that are
meaningful to consumers, that was number one. Number two, we wanted to fund
research and development, improve assessments on the value of health care based
upon measures, and information then about cost and quality that are relevant to
all healthcare stakeholders. And we wanted to fund research that addressed
accountability and care coordination. I’m sorry, the second bullet headline is
really measures of focus on healthcare value. And then, third, measures that
focus on accountability and care coordination. And finally, looking at the
broad infrastructure of the healthcare landscape, how do we improve the
efficient acquisition and use of healthcare data drawn from a healthcare
digital infrastructure.

These recommendations in many ways really helped us to focus on the
patient-centered and overwrought term. But really, from the patient’s
perspective, what are the measures that matter. And we thought that is really
fundamentally a paradigm shift in which we’re in the midst of. We know that
providers use measures in oftentimes different ways that consumers might use
them, and that providers have both their internal types of measures in
reporting, as well as measures in reports that may be going externally to the
public.

Another controversial issue, we need to address fundamentally individual
patient concerns and preferences, of course, and without increasing the data
collection burden on the provider side. What role might the consumer play in
this. And how do we measure an assess cost and from whom’s perspective, to whom
does this cost assessment really have to be determined, so that we can really
use the measure and insights appropriately to assess value of healthcare. And
are the providers holding accountable for patient care, these were a number of
the controversial topics.

So today, a year later, we’re diving into really the first set of issues,
the measures that matter to consumers. And we thank all of you for coming and
we will have a number of different perspectives presented. And Paul Tang will
go through the agenda for the next day and a half.

DR. TANG: Thank you, Blackford. My name is Paul Tang, Palo Alto Medical
Foundation and co-chair of this subcommittee with Blackford. And I want to
welcome you, as well.

I have to say I have been looking forward to this for probably a year. This
is just a favorite topic. I think even before health reform, it has been very
clear that we need to pay much more attention and direct much more of our
perspectives towards the perspective of the consumer. So even after health
reform, of course, it’s really front and center, including our projected ways
that we will be paid. So this has been a favorite topic on a number of
committees on which I serve, and delighted to be able to have this hearing over
the next day and a half.

Christine Bechtel shared a story that gave me a little heads-up on what she
is going to say. We served together on the HIT Policy Committee, and she has
brought up a number of issues related to let’s say her position on having an
electronic health record system, which we are all a part of. So we have been
chiding her about maybe it’s time to switch. But anyway, so she is going to
share that story and I am just looking forward to hearing about that.

But our agenda for today starts out with what goes through the mind of a
consumer when they decide either to pick a doctor or to change one. And let’s
get in the mind of that, and our first panel is going to talk just about that
process, which will be wonderful to hear. And then, what kind of tools would
you like to have when you’re making those kinds of important decisions for
yourself or your family. And we go into what measures that would matter and
then decision making in the second panel.

After lunch, we talk about things that just weren’t talked about in the
past, when we worked only on paper, which is how do you know how the patient
did after you replaced their joint or worked on their heart failure. Those
would clearly be things that would matter to the consumer, but in the paper
world, had no way to find out. So now that we are untethered from paper, or we
want to be, how would we get into the minds of consumers and find out how we
are really doing from their perspective, so the use of functional status in
self-management measures.

And then, the area of patient satisfaction and experience of care, and
actually they are not the same thing and that is what we are going to hear
about in the second panel in the afternoon. And because a number of us are
measured against those parameters, what do they tell us about what is going on
in the mind of the consumer. And close out the day with hearing about patient
preferences. My gosh, should we care about that?

So we have a wonderful opportunity to understand from the consumer
perspective what matters, what information should we give them to help them
make decisions, and from the professional side, what information should we
really be taking into account while we work with consumers and patients. It is
truly an exciting day, and having read the pre-material, I know we’re going to
hear a lot of good information and look forward to the dialogue.

Before we get started, I wonder if we could just go around the room and
introduce the members who are here on the subcommittee. Judy?

DR. WARREN: Judy Warren, University of Kansas School of Nursing, member of
NCVHS, and no conflicts.

DR. HORNBROOK: Ark Hornbrook, Kaiser Permanente, member of NCVHS, no
conflicts.

DR. COHEN: Bruce Cohen, Massachusetts Department of Public Health, member of
the committee, no conflicts.

MR. QUINN: Matt Quinn, National Institute of Standards and Technology, staff
to the equality subcommittee.

DR. TANG: I’m Paul Tang, member of the committee and no conflicts.

DR. MIDDLETON: Blackford Middleton, Partners Healthcare, Brigham and Women’s
Hospital, co-chair of the quality committee, no conflicts.

MS. GREENBERG: Good morning. I’m Marjorie Greenberg, NCHS, CDC, and
executive secretary to the committee.

DR. FITZMAURICE: Michael Fitzmaurice, Agency for Health Care Research and
Quality, staff to the subcommittee.

DR. TANG: Why don’t we begin with the first panel. If you wouldn’t mind, as
part of your talk, just give me a brief introduction of your role in your
organization, and I think we’re starting out with Joyce.

I have just been corrected. We need to do introductions around the room, so
everyone knows who is here.

MS. SMITH: Heather Smith, American Physical Therapy Association.

MR. WANG: Derrick Wang, Social Security Administration.

MS. THOMAS: Sarah Thomas, National Committee for Quality Assurance.

MS. AUSTIN: Kirsten Austin, America’s Health Insurance Plans.

MR. STUMP: Dave Stump, Northwestern University.

MS. HOLIDAY: Regina Holiday, the Medical Advocacy Mural Project.

MS. JACKSON: Debbie Jackson, National Center for Health Statistics Committee
staff.

MR. STEIFEL: Matt Steifel, Kaiser Permanente.

MR. SEIDMAN: Josh Seidman, Office of the National Coordinator for Health IT.

MS. BICKFORD: Carol Bickford, American Nurses Association.

MR. HUFF: Good morning. I’m John Huff. I’m from CDC National Center for
Health Statistics.

MR. HANSON: Good morning. Jim Hanson, Dossia Consortium.

MS. KANAAN: Susan Kanaan, writer for the committee.

MS. COOPER: Nicole Cooper, staff to the committee.

Agenda Item: Understanding Consumer/Patient Health
and Healthcare Decision-Making Needs

MS. BECHTEL: Good morning. I’m Christine Bechtel and I am vice-president to
the National Partnership for Women and Families. Can you guys hear me, see me,
do you need me to stand up? So the National Partnership is a non-profit
consumer organization. We are based here in DC. We have been around for 40
years, working on issues that matter to women and families. We have a couple of
relative initiatives that are pertinent to our conversation today. One is the
work that we do with the Pacific Business Group on health, through the Consumer
Purchaser Disclosure Project, which is an effort to bring consumers and
purchasers together, and work on better aligning quality measurement, public
reporting and payment to the needs of consumers and purchasers.

We also work in 16 communities across the country with RWJ’s Aligning Forces
for Quality grantees, and we help them to engage consumers in their local
initiatives to publically report cost and quality data. So we do lots on health
care and have a significant focus on the delivery system.

I think that the subcommittee is really asking some fabulous questions today
that are important to think through on what kind of information that consumers
are looking for in the healthcare system to support decision-making on a range
of topics, whether it is doctor choice or shared decision making, choosing
treatment options. I am going to focus on one particular application of health
information seeking, which is finding a doctor.

My colleagues and I from the consumer community, Joyce and Lynn, had a
chance to talk significantly before the panel, and realized that we were all
going to really say variations of the same thing and in different sort of
context. So rather than sort of cleverly trying to say the same thing in a
different way, I thought what I would do is share my sort of story around how I
found my current doctor and what I need to do to now find a new doctor, as Paul
mentioned. And then, on the second panel, I will go into a little bit of the
work that we have done as the National Partnership and also with the Disclosure
Project around patient-centered care and measures that matter.

So I think first you have to know a little bit about me as a person to
understand me as a patient. It is kind of hard to sum up your life on a slide,
but nonetheless, here are the key elements. I’m an avid golfer, that is my
first hole in one last summer on the upper left. Lower right is the state team
semi-finals, which were played at Congressional Country Club a couple of weeks
ago before the US Open, so that was just a very exciting moment. And then, you
see some of my family members, my mom, my dad, my grandmother.

And in terms of how this sort of affects me as a patient, I love to play
golf, it’s my passion in life, but I have got back troubles. I have something
called spondylolisthesis, which creates a fair amount of instability and pain
in my lower back. And I am also hyper mobile, which means that my body doesn’t
like to stay in place for very long. And so, that means that I have to do a lot
of strength training, although I have been known to fall off of that wagon. And
I do physical therapy every two weeks to kind of get my body back in alignment.
And then, of course, I have a primary care physician, and that is really the
core, along with my trainer, of my care team.

And then, of course, my family health history is nothing dramatic, the usual
stuff that I think we all have in our families around some heart disease, and
issues here and there. But my parents are actually both very healthy. And then,
my day job, I get to represent patients and families, which is a great passion
of mine.

So five years ago, my doctor who was an osteopath and had an electronic
health record and was just a terrific physician, decided to move to warmer
climates, bless his little heart, so I started to find a new doctor. And five
years ago, I knew then a lot of what I know now, so I did all the things we
want consumers to do. I tried to use publically available health information,
and it wasn’t very good. There wasn’t a lot out there, there wasn’t clinical
quality data, there were very, very few patient feedback mechanism at the time.
So I ended up actually calling, the head of ACP and AAFP and the senior folks
at the AMA, saying can you help me find a doctor? Unfortunately, they just sort
of kept coming back to Dr. Peter Bash, who many of us know and love. But since
we worked together so closely, that is like way too close for comfort, so I
kept going.

So what I did was I actually used Health Grades and just started calling
doctors’ offices, and asking them do you have an electronic health record? And
I did that because I worked at the time for the E-health initiative and I
thought, well, this is a proxy for quality maybe, I hope. Most doctors’ offices
did not actually know the answer to that question, but I found one that did.
And it was a family practice, they had different services on site. They used an
electronic health record, they were working on a medical home qualification, so
they looked really great on paper. And that was exciting to me, and I thought
finally I am going to get the patient-centered care that we all know and
deserve.

But it turned out that I have had a number of problems with them over the
last five years. Paul has heard me talk publically about some of my interesting
experiences with them. But there is just a real lack of coordination with
anybody on my care team. They have a difficult time remembering who I am when I
come in. They really don’t use their EHR in any way other than a literally
straight medical record that happens to have electrons behind it.

I thought I would tell you a quick story of I guess it’s the straw that
broke the camel’s back. I, about three years ago, was very sick. I had a nasty
cold and I called the doctor’s office, again, was supposed to be a medical
home. And I said hey, I have a question. I have got a terrible cold, I can’t
sleep at night, I need to take Nyquil, but I can’t sleep when I take Nyquil.
It’s the old kind that has Sudafed in it, because I stockpiled it before it
went off the market. Can I take a half an Ambien with it? And the receptionist
said well, you can’t speak to a clinician, so let me write down your question
and I will get you an answer, and I said okay.

And she called back, which was very kind, and she said I have an answer to
your question. You can take Ambien and Sudafed. And I said that’s great, but
that’s not my question, and I reiterated my question. And so, she went and she
tried to get it answered, called back again and said okay, okay, I talked to
somebody else, and they said you can take Nyquil and Sudafed. I said okay, but
that is also not my question. Can we try again?

And so, I got a third phone call from a nurse this time. I happened to be, I
think, in the grocery store, loading up on Kleenex or something, and so I
didn’t hear the call. And I got a very terse voicemail from the nurse who said,
Ms. Bechtel, the answer is exactly as the receptionist said to you. You can
take Sudafed and Nyquil. So now, I have three different answers, or it was
Ambien and Nyquil.

So I have three different answers to none of the right questions. But she
also said, if you want any more information, it has been more than six months
than your last visit, and our policy is we have to see you every six months,
and so you have to come in. So I drove in there, mad as a little hornet. And I
met with a new young doctor who I had never met before, which was fine. And we
are in the exam room, and he says why are you here? And I explained my
question, and he says, well, yes, you can take those things together. I said,
great, thank you. And I stood up to leave and he said, wait a minute. Is that
all you are here for? I said yes.

And I said, I have to tell you, I need to be able to call you and get an
answer to a very simple question. You’re supposed to be medical home, I am
supposed to have some access kind of benefits to it. This was not very
patient-centered. And in that moment, his iPhone rang and he answered it in my
exam room. I thought, oh, goodness, do you not see the irony. It was a
racquetball buddy, and so they made a racquetball date, and he hung up the
phone.

And I said you know, I have to tell you, I just want to give you a little
piece of patient feedback here. You have an electronic health record, but we
are in a position where I have an acute problem. And it is only now that I find
out that I am overdue for a six-month visit. Maybe you could think about using
your electronic health record to send reminders and remind me that I need to
come in, etcetera. And he said oh, no, the systems just don’t work that way.
And if you know anything about my background, I am one of the last people on
earth you want to say that to.

And I said listen, I actually know that the systems do work that way. And
you may not have set it up, but they do in fact work that way. His iPhone rang
again, he answered it again, it was a drug rep. So I left, disheartened. I did
have a chance to speak to my main doctor later and gave her the feedback, and
actually offered to help. I told her what I did for a living and said I would
be more than happy to be part of a group of patients that helps you really
redesign on a patient-centered way. I am still waiting for that call.

Paul has heard me tell sort of experiences over the last couple of years on
the policy committee together. So I decided to use the panel together today as
an impetus to actually start looking for a new doctor. I wanted to think about
things like medical home, maybe a higher level of qualification than what I
had, EHR, positive patient experience. And so I went and looked at a lot of
websites, mostly free, not all of them were. And I also looked at mobile
applications, and I have spent about eight or so hours doing this, and here is
what I have found. This is a smattering of sort of what I found.

This is a web shot from FindADoc.com. And I happened to click on this
physician, Kevin Michael Gil, because he had a lot of patient experience
ratings. It’s not clear to me where the data actually came from. I think it is
patient-experience surveys because those are generally the CAHPS domains that
you see up there, although the slide is really small. But there are things like
communication and office staff and wait times and things like that. Although
though it says patient reviews, zero, and I looked for a good 15 minutes to try
to understand the data source, and it was not on the website that I found.

But I did like this. There is a red button and it says invite Dr. Kevin Gil
to be your doc friend. And I thought this is great, I am going to have a doc
friend. And so I clicked on it, and what I got was access denied, you have to
be a doctor or a dentist. So I moved on, I went to my own health insurance
company because when you work in our field, you have the benefit of
professional contacts. I had a long conversation with Robert Krughoff of
CHECKBOOK about this. And he said look, try your health insurance company, and
so I did.

And the health insurance company promised that if I logged in as a member, I
could see comprehensive costs and quality data. And I can tell you that if you
are looking for a primary care physician through this insurance company, you
get neither. You get zero data, other than their location and gender. And you
actually do not even know if they are board certified. If you click there, it
takes you to the ABIM website.

So Health Grades has improved significantly since I used it five years ago.
What is available for free is the patient experience information and other
things, you have board certification and gender, insurance is accepted. The
issue that I found here was that most of the ratings were a sample size of
three or four or five patients. The most I could find in the primary care docs
in my area was 24 people who had given feedback. I looked at Angie’s List, same
kind of issue, mostly patient reviews and some other kind of specialty
information, etcetera. But again, no clinical quality data whatsoever, so that
was challenging, and no sort of designations. It is a medical home, is it a
NCQA-certified, the diabetes module that they have, whatever.

So then, I thought well, okay, I’m on a policy committee, I have got to have
a meaningful user. So I went to the CMS website and looked up the list of
doctors who have been paid under meaningful use to date. Now, clearly, this
website is not meant for the purpose that I’m using it for. But I went through
the PDF document and I wrote down the 15 doctors who have been paid on
meaningful use in my area. And then, I looked them up on this website called
Vitals.com. All of them were specialists, by the way, except one, and the one
who was a primary care had a patient rating of two out of five.

And what I liked about this website is that not only does it give you lots
of information, like the other ones do around hospital affiliation, but it
actually also has the average wait time in the practice listed, which I thought
was actually very helpful. And the meaningful user PCP had an average wait time
of 35 minutes.

This is our friend, Dr. Kevin Michael Gil again, who I have no idea who he
is. Bless his heart, I have to call him, but he has a 10-minute wait time, so I
thought that that was better. It also allowed the clinician to make a personal
statement about their own interests, and so he has an interest in dementia,
which was interesting.

I was pretty disheartened because I can’t really find good meaty substantive
information here. So I went to mobile apps, and this was an interesting
experience. I found about 20 or so on my iPhone. They were mostly kind of
either location-specific or health plan-specific. Many actually not in this
country, but you could find a doctor in India, if you need one. They were, as
you can imagine, very common to the websites in terms of the feature that they
offered, although they had far fewer patient ratings than any of the websites
did.

But I do want to share an experience that I had. So I found one mobile app
called SpeakWithDoc. And this actually does not help me find a doctor, like in
terms of ratings and publically available information. But it does help me
connect with a doctor. So I decided to click on this little button that says
ask a physician and see what would happen. And within two minutes, I was on my
phone with a doctor texting.

And I decided to pose the exact same question about Nyquil, Sudafed, Ambien
that I had posed to my primary care doctor, that had taken so long to get an
answer to. And I had a lovely discussion with this person. I have no idea who
he is or was. It was very timely, we went back and forth, this was on a
Saturday morning. And there was a brief delay in the middle of our dialogue
because he had to drive somewhere. But in the end, I will read to you exactly
what he said.

He said sorry about the delay. I was driving and didn’t transfer your
question. The issue is that Sudafed can cause many patients to feel a bit more
hyper, and therefore make it difficult to sleep. While there is no real notably
terrible interactions between Sudafed and Ambien, what I have observed in my
patients is a few more reports of sleepwalking. So that is why I am a bit
hesitant about you taking them together. That reaction might not be the same
with you, though. Already I have more information from this guy than my primary
care doctor actually gave me in person. And then, he thanked me for writing in
and asked me to rate the app in the iStore, which I thought was great.

Bottom line, before I turn it over to my colleagues for a much broader view
of my own experience, is it is easy to find the kind of very barebones basics.
But it is not easy to find any kind of clinical quality data. And as a
professional who spends a fair amount of time in this space debating over
measures and the measure application partnership and other things that we do,
it was just very surprising to me that there is nothing out there that I could
find, with eight hours and my knowledge base, so it doesn’t bode well.

Also, I don’t have a sense of what kind of services that they might offer,
if they have a patient portal, for example, or they offer secure messaging or
if they have basic labs and x-rays on site, so that I do not have to, next time
I sprain my ankle, it takes me three days to sort of get the right diagnosis,
and things like that.

There were a lot of options, a lot more websites this time than there were
five years ago. But I do not know exactly who is valid and relevant. They all
used different data sets, they all have a different number of reviews, they all
have different sources of reviews. The data is not very deep, so I am back to
what we all do, which is I need to now start asking friends, calling doctors. I
will probably call the family docs again and ask them.

But I also think that my experience with this mobile application was a real
sign to me that we are really generally in a new era today. And I think it is
going to help us raise patients’ expectations about healthcare system and
physician performance, and that we need to make sure that our office-based
practices and the public information that we provide to consumers actually
catches up with what we know technology can do. So that is my story and I will
turn to my colleagues. And on the next panel, I will get into more around
patient-centered care.

MS. DUBOW: I am Joyce Dubow from AARP. AARP has around 40 million members
throughout the country. We have an office in every state. Our members are
between 50 and over, about half, so we are interested not only in the Medicare
population, but in the people who are insured commercially, as well, some of
whom are uninsured actually.

I am going to focus on older people. I think Christine’s setup was fabulous
because it frightens me that my brilliant colleague, brilliant in many ways,
could make so many assumptions that could lead her astray, and that she is just
a perfect example. First of all, I know nobody who would spend eight hours
looking for a doctor, nobody. My husband is always my best focus group because
he is not interested at all, which is why we belong to an integrated system
because we don’t have these problems. And we also have access to email, and all
week I have been accessing my doctor by email, and I get very quick responses.
The apps business terrifies me, who the heck is this guy who is giving you
information?

I am coming back to policy one, we are in Washington and I want to talk
about some stuff. I want to talk about the challenge of decision making. I am
going to focus on people who have more than one chronic illness, older people,
and talk about the challenge of decision making. Christine’s example is
perfect, and she is healthy and resourceful and smart and preserving. These are
not typical characteristics of most health information seekers at all. They
want stuff to be easy and at their fingertips, and easily understood.

So just let’s think for a minute about the challenge of decision making and
what we want people to do. We want them to make good decisions that are good
for them about the best use of services, drugs, interventions, whatever, use of
technology, to meet their personal needs and their circumstances. And to do
that, you need to have a whole host of skills. You need to be able to read, you
need to be health literate, which is not the same thing as literate. Just
remember the numbers, 90 million people have very poor literacy skills, which
addresses both numeracy and ability to read. That is not health literacy.
Health literacy is situational and it depends on how complex a decision you
have to make is. So the more complex your situation, the health situation you
have, the higher skills you need. It is harder to account for where you need to
be in the health literacy range. I will give you some numbers in a little bit.

People need to solve problems, which means they need to make trade-offs and
they have conflicting goals. This is a very, very complex task when we talk
about somebody, particularly with multiple health conditions. You also have to
have confidence in your ability to make a decision. And I would remind you as
you get older, that confidence diminishes. No matter who acute you are, it is
just harder to make a decision. Uncertainty enters into decision making, it’s
much, much more difficult. And of course, self-efficacy, the ability to manage
your care, to manage your health and to have confidence in that is critically
important. And as they say, it often diminishes with age.

So what do we know about decision making? It is complex. We heard that from
Christine and her question was simple. It was really easy, she had one quick
little question. It is very hard to have to integrate these variables into a
decision. People end up focusing on one thing. They focus on what they can
understand, and they often undermine their decision.

We have great examples of this in part D, where people do not maximize their
own interest, because they are making simple decisions. They pick on one thing
and they focus on that, because it is too complicated to weigh so many factors.
And when something is concrete, they will pay more attention to it. So that
when somebody has information on costs, for example, which they get, if you see
something about the level of a premium, they do not pay attention to anything
else. They do not pay attention to performance.

Christine was perfectly happy to find a doctor. She did not really care
about what he knew. She focused on one thing, access to that information, not
whether the quality information was good. You talked about from your
professional knowledge that you needed to know more about data sources, et
cetera, but it is not what you did.

Preferences also are not stable, and they are easily influence by how
information is presented. And that is very, very important when people are
uncertain, particularly their preferences are not stable. And we know also that
when you frame a choice as a loss, you get people’s attention. So if you talk
about the fact that somebody’s safety may be jeopardized, it is a whole lot
more affective in catching people’s attention than saying here is a high
quality performer. If you scare them and make them anxious, they are going to
pay more attention. Just think about what your own reactions would be.

There are obviously many, many barriers to making decisions, particularly
about quality. For starters, people do not believe there is a quality problem.
There is tons of data that tell us that people think that there are problems in
the healthcare system, but not with their personal doctors. It takes a while.
It took Christine five years to decide you might have had some problems with
your doctor. It takes a long time. It’s like your Congressman. Your Congressman
is good, everybody else’s is not so good.

People see choice as a proxy for quality. Christine’s decision making was
very interesting. Not once did we hear her think about narrowing her choices by
looking at a different kind of system. That would simplify the choice. It might
also get her some consistent e-access and all the rest, but that is not what
she was thinking. Now, that is not to say that she ought not have that choice.
But I think that, if you think about how people make choices, they believe that
if they make their own health care decisions in a very, very broad network,
that is better because that is likely to yield better quality.

And of course, we know that many people think more is better. Most people
think more is better. To a large extent, performance information is just simply
not understandable. And as Hibbard and others have told us, if you do not get
it, people just marginalize it. They think it is not important, so they do not
think it is worth paying attention to. As I mentioned before, there is an
enormous cognitive burden to making complex decisions, and people just sort of
tune out. They cannot process more than five or six variables at a time, so
think about what is required.

This is a very bad slide, but I just wanted to show you. These are literacy
levels, and if you look at the 65 plus right on the bottom, you can see that
almost a third of people have below basic proficiency. That means they cannot
read a medication slip, they do not necessarily understand an appointment slip.
They certainly are going to have challenges with understanding directions about
self-management. Just think about the implications of having such a large
portion of the population over 65, who have multiple chronic conditions, having
such low literacy skills.

You asked us to think about the demands that people have to make when they
make decisions, and I know Lynn is going to talk about this, too, and I am
going to buzz through it. But for starters, they need to decide which coverage
option they are going to have. In the exchanges, it is going to be choosing
among the metals, the platinums, the golds, the silvers.

In Medicare Advantage, for an over 65 person, on average she has 26 Medicare
Advantage plans to think of in 2012. Just think about having 26 plans to
consider. As employees, nobody has that kind of choice, and one has to wonder
whether there is any value to it.

Once they get past that, assuming that they do evaluate the performance
information, and any kind of out of pocket cost estimates that might be there,
on the Medicare.gov site, there are some estimates. There has to be a selection
of providers. Lots of people, particularly older people, make decisions about
coverage options on the basis of whether their physician is participating in
that coverage option. But if we want people to be paying attention, we want
them to be examining information about performance when it is available or when
it becomes available. We want them to make more granular decisions and not make
knee-jerk reactions, so that we will be motivating improvement. We will talk
about that again in the next panel. So again, we expect them to be comparing
comparative information.

We also want them to be participating in shared decision making. So they
have to participate as equal partners in selecting treatments and interventions
and drugs. And they need to be able to assess risks and benefits. They need to
rely on their clinicians to give them good and accurate information about the
risks and assessments, and I would suggest to you that the clinical community
probably has some numeracy problems itself, which is because it is hard. There
is a lot written about you present information on risks and benefits. And we
have not gotten it right, and there is a lot of misunderstanding and a lot of
misinformation. So just think about the challenges of participating in shared
decision making.

And then, of course, for older people, the imperative to participate in the
development of a care plan, so that it represents something that represents
your preferences and your circumstances, and that it is done in genuine
partnership. Often, the surrogate turns out to be a caregiver. But again, these
challenges apply to caregivers, as well as the individual patients. And
finally, if you think about it, consumers, patients, have to be able to engage
in self-protective behavior.

They need to be able to, if they are a hospital patient, to challenge that
worker, who may come in clearly not having washed her or his hands. Again, they
have to be able to ask their clinicians about the risks and the benefits
recommended interventions. They have to be able to appeal, they have to be able
to assert. This all is challenging in the decision making realm.

The current state of information, we heard great examples of it, is pretty
weak. Materials tend to be one size fits all. Again, think about older people,
think about target audiences where needs are not the same as with young people.
Christine is a good researcher, look what she had to do. We cannot expect
people to be able to do that, to be able to refer to these websites that she
did, and to pull out information that will be useful for them.

There is just no ongoing routine assessment of the literacy skills or the
decision-making skills that is done by providers. And certainly, the materials
that are created by health insurers do not differentiate typically by the
target audience. There is a strong reliance on web materials because it is a
cost issue, and it is much less expensive to be able to do it. But it cannot be
the only answer. And I thought it was interesting that the IOM committee on HIT
and safety actually pointed out the fact that IT alone cannot meet the needs of
all people. We just cannot.

I know this is an abhorrent view that people who are aging in will be more
technically proficient. Frankly, I do not buy it. I think we have to wait to
see the data. I think all of the challenges that come with aging will enter
into it and people will be more disinclined. It’s not to say that they’re not
going to use email and that they’re not going to look on the web once in a
while. But as the main source of information, I think we really need to wait.
The jury is not in yet.

Comprehensive information is rarely available. Your employer is likely to
give you information on cost, on how much your premiums will be, and maybe if
there is an out of pocket limit. You don’t see, unless you work for a really
big employer or if you’re a Medicare beneficiary, whether you get any
information on the performance of the providers that are being offered. And
even when it is, it’s not always salient and not always actionable or relevant
to the individual who wants to make the decision.

The formats are not evaluable. It means that it is not readily accessible.
They do not facilitate decision making. I put the next idea in green because
there is some hope. There are some websites that actually try to provide a
framework to help people make decisions. CMS does, NCQA does, they pick up the
old fact framework about living with illness. I forget all the headings, but
they categorize the types of information into buckets, so a framework is
provided. Some of the state report cards do that, too.

I could not find recent data, this is from 2008. But it is a very useful
series of data that the Kaiser Family Foundation and ARHQ have produced since
1996, I think, which has looked at whether people know about the existence of
quality information and whether they are using it. And in 2008, you can see
that only 14 percent saw and used quality information of any sort. The numbers
may be a little bit higher, but I don’t think it is above 25 percent anywhere.

They respond to these approaches by turning off how do consumers respond to
these approaches. We know that recent Pugh data show that African-Americans,
Latinos, adults living with disability, you can see the string here, are not
likely to go online. We know that people rely on face-to-face intervention, or
call lines, the volumes are very high. People just are inert, they do not make
changes. They do not make decisions that represent their best interests.

There is lots of opportunity for improvement, and I just have a whole bunch
of things that we could be doing. First, we have to consider the target
audience’s skill levels. There is no question about that. We have to offer
people information that will be relevant to them, and then they can actually
act on. They want information about how to find a good doctor. That is what
people want to know. They want information about patients like me. They want to
know how much something is going to cost them. Providing averages or DRGs for
Medicare, what a waste. Most beneficiaries do not have a clue what that means.
They want to know what it is going to cost them. It is true that the Medicare
Plan Finder helps you estimate how much a Medicare Advantage plan, for example,
might cost you during the year, based on certain assumptions.

But we need to do better. People want to know what it is going to cost them.
We need to integrate information and bring it all together in one place. So
now, you can find health plan information on a Medicare website, for example,
but it doesn’t tell you anything at the same place about how good the doctors
in that plan are. It’s at the health plan level. We are not giving people
information when they need it be integrating it all together. Clearly, there
are opportunities to improve communication with patients using EHRs. But we
need to remember, as I said before, that it is not for everybody.

I mentioned that we need to lower the cognitive burden by making information
more evaluable. Rank order plans in the exchange, if you are going to have
those metal categories, rank order them by quality. If they are platinum, they
are not all equal, so rank order them by performance. Use symbols that mean
something to people, that signal them. Help them make a decision. Obviously you
would want to provide drilled down information for people who want more detail.

We need a consistent framework to help communicate about quality, help
people understand what this stuff means. We know narrative is very useful for
people with lower literacy skills. Plain English, no jargon, white space
applies to everybody, not just people with low literacy. Obviously we need to
use multiple approaches, as I said before, to meet the needs of the target
population. It cannot only be in writing.

We need to reduce the me, too choices. There are too many of those. When
there are no difference between and among health plans, and when they all have
the same physicians participating, how in the heck is anybody going to make a
decision when there are just minor tweaks. To CMS’s credit, they are actually
beginning to address this problem now, by eliminating some of those me, too
products. But we need to do more of it to, again, make the decision-making
easier for people, particularly for older people.

Quality information has to be featured as prominently as cost information.
It is as important. And at a glance, interpretation would be more valuable.
People are not aware that quality information exists when it does exists. They
do not know about it. I cannot tell you how many times I have asked people if
they know that there are hospital-level, hospital-specific data available in
Hospital Compare. They just do not know. They do not know about health plan
information being available in the state report card. People are not aware of
it, this stuff is not publicized. We need to do better in doing that.

Summary of key points clear, obviously there always has to be additional
sources of help. We always have to be careful that the information that is
presented is not manipulating decisions that are not going to be in the best
interest of patients. I am not suggesting anything. I am not suggesting that
employers would ever engage in that practice, but I have seen it. I think we
need to be very careful, again, how information is presented, is as important
as the content. And I think we have to be very, very careful about that,
because obviously it is going to skew decision making.

I mentioned negative framing before. This is a very provocative idea. I
think with Medicare beneficiaries, people with lots of comorbidity, we have to
be really careful. You do not want to make people anxious when they rely on the
healthcare system. But you want them to be paying attention, and so I think we
have to figure out a way to strike a balance. We need to engage intermediaries
to help older people.

And I think finally, we need to be sure that we test materials, to be sure
that the content is understood as it is intended to be, and that people
understand what we are trying to communicate. I threw this in, I know you are
going to talk to Robert Krughoff. The fact is that we actually already have
some mechanisms for presenting cost information to consumers. Krughoff has done
this in demonstration in three cities, at the physician level. He has actually
provided physician level information, too. But this is a tool that he already
offers to federal employees and several agencies, that actually help cost out
health care, based on health status. And it is actually sound.

We have the tools to begin doing this now. This is not pie in the sky stuff.
More challenging, I think, is developing some of these tools to be responsive
to people with comorbidities and older people, so that they can make good
decisions. Happy to answer questions.

DR. MIDDLETON: Why don’t we try to go ahead and get through all the
presentations. And if we could be about 10 minutes each, that would leave us
time at the end for questions.

MS. QUINCY: I am Lynn Quincy from Consumers Union. I am a senior health
policy analyst there, and if I run over my 10 minutes, just wave at me, please,
and I will try to go really quickly.

Consumers Union is the non-profit publisher of Consumer Reports magazine.
Not everyone realizes that we have an advocacy and policy division, and that is
where I work. I have nothing to do with testing cars unfortunately. And I want
to welcome you to Silver Spring. I actually live just a few blocks that way,
and make sure you visit our AFI and go to restaurants and all that good stuff.

I am going to talk primarily right now about how people shop for health
insurance, and the reason is because that is pretty much what I know about. I
think the main reason I was invited to talk to you today is that Consumers
Union, with me as the principal investigator, did three different studies on
how people shop for health insurance with connection with some of the reforms
that are being implemented around the Affordable Care Act.

And we developed some really nuanced information that I do not think was out
there in the world of research prior to our investigation, having to do with
the barriers that people face when they shop for insurance, and exactly how
they approach the task. But of course, I have got to relate it back to what you
are doing today. The way I am going to try to do this is by reminding you that
when you go and shop for your health insurance coverage, it is really like a
gateway to those performance measures that I think most people are going to be
talking about today, which I think primarily are measures having to do with
provider quality.

I do think it would be very, very useful to keep a very clear lexicon going
here, as to any point in time, as to whether or not we are talking about
provider quality measures, or actual true plan quality measures, which are the
things that they have control over, like how good is their customer service, as
well as how both of those things integrate into a value determination.

So a quick overview, the studies were three in nature. We used some focus
groups, but primarily cognitive interviewing. And we were looking at different
aspects of a new health insurance disclosure that is going to be put in front
of consumers. And at the beginning of each of these studies, or any given
testing session, we asked open-ended questions of consumers about how they shop
for coverage. I want to share some of the things that they said.

What we found is that when consumers go shopping again for health insurance,
they are looking for a good value. Their primary concerns when they think about
value is they want to know what is covered and how much is it going to cost me.
We hear that over and over and over again, they were very, very consistent. We
sometimes heard, is my doctor in the plan, and there is some survey data to
suggest that is a primary question that consumers ask. And or does this plan
have good quality providers.

And what I want to emphasize is that consumers, at least in our testing,
they actually did not say I want the cheapest plan. They wanted the best value
plan that they could afford. That is an important distinction and it provides
you with sort of an entrée into getting them to maybe pay attention to
quality measures.

So a couple of notes from these very high level findings, that coverage
concerns do triumph quality concerns. That is something that you are going to
have to be aware of. Provider quality trumps plan quality measures, in terms of
again these open-ended questions we were asking consumers. And this is from not
my own research, but I think this last point, it definitely resonated with what
I was seeing as consumers talked about coverage and care, when they look at
provider quality information, it is not necessarily to optimize their care, but
to avoid the risk associated with below average care. And I think that dove
tails with some of what Joyce was saying. And another thing to keep in mind, as
a way to create effective measures.

The key finding, in my short time, I cannot tell you all the wonderful
things we learned. But the overarching finding is that while consumers care
about value, they cannot calculate value. And one of the main reasons they
struggle is that they struggle with the cost-sharing terms associated with
insurance, and there are many reasons why they struggle here. The jargon is
unfamiliar. People would look at terms like patients’ out of pocket limit or
out of pocket maximum. And they would not really be sure if that number there
was good for patients or bad for patients. That is the level of difficulty they
were having with these concepts.

So they may not know the jargon, even if they are familiar with the jargon,
let’s say coinsurance, they might say at the beginning that they had heard that
term and they would be comfortable explaining it to a friend. When it came to
using it in usability tests, they actually struggled and did not really
understand the term. So you have got these complex, unfamiliar topics, and then
in order to arrive at a bottom line for a consumer, you have got to roll them
all up and wave them together. So you know have what is basically a cognitively
impossible task for consumers.

And a fourth piece, for some consumers, that they are actually missing what
is called the mental map when it comes to shopping. They actually have an
incomplete picture of what health insurance is, and the point that Joyce made
that it is actually out there to help insure you against unexpected risks. Some
consumers approach the task as shopping for prepaid health care.

What this means, there are a couple of bottom lines. If you are trying to
design effective shopping tools for consumers, the starting place is you’re
actually asking them to shop with a blindfold on. They cannot figure out where
the best values are, even if you are only looking at the coverage dimension,
and you have not tried to bring in the quality dimension.

Another bottom line that I think we need to think about, perhaps your
research into the future, is simply gathering this top level information
consumers taking all their shopping energy, leaving nothing left over for
quality measures. And this is this gateway concept, can we get them through the
gateway a bit more easily, so that they can actually focus on quality
dimensions.

One of your questions is, how can we effectively disseminate information.
And I would like to emphasize this is an evidence gap, we do not know how to do
this. And when I say know, I mean in the sense of we have done consumer testing
and we feel confident that we have succeeded with the dissemination approach
that we have taken. You are going to hear me say this a lot, we need to do more
consumer testing. It is greatly an underutilized tool for developing policy.

But we do have some of the answer, based on my experience. The most
important point is where is this information coming from. In our testing, even
people who were very health insurance literate, so they had good skills, we
were giving them a great document with really clear information, if that
information came from a source they did not trust, they are not going to use
it, so you failed in your goal. And unfortunately, sorry AHIP in the room here,
but consumers do not trust health insurers. And if they think that health
insurance information, this document I was showing them, came from a health
insurer, they are going to discount it or they are going to go and get it
verified by another entity. That trusted source is a critical component to what
you are trying to accomplish. It is very closely related to this need for
people to know that the information, quality measures or any type of measure is
coming from someone like them. And I think there are a lot of ways you can
accomplish that.

Joyce already mentioned you want this information to be delivered just in
time. People cannot hold that much discreet information in their head, so your
best approach is that you deliver the coverage aid or the quality aid or the
personal assistance, whatever it is, the health insurance education in a just
in time fashion. You can’t completely remove it from the experience of shopping
for coverage.

Plain language, Joyce already said that, so I will move on. Another question
I thought you were getting at is, who is out there helping consumers shop for
coverage. And this is something that we did not ask explicitly, but would come
up in our testing, and it was pretty interesting. Almost every consumer lacked
the confidence to make decisions. Even the expert consumers said I cannot do
this, I need help. Sometimes they would say I work with a broker, but more
often than not, they are just relying on someone else they know who may not be
an expert.

And I thought this was really cute, a lot of them said they would ask their
mom. And again, they volunteered this, we were not asking them. This is not
something that we were trying to figure out. But when we asked them to make a
decision in our usability test between two planned choices, they said I don’t
know. I would have to show this information to my mom, or my neighbor who is a
nurse, or my neighbor who is nothing, but uses health care. These are the
people who are influencing their decisions about coverage, and presumably the
quality of their providers.

We do have information that when a free expert trusted source of assistance
is available, consumers are going to use it. Joyce mentioned the Medicare
Rights Center Helpline. We have a lot of experience with the ship organizations
that help people with the Part D rollout in Medicare. That is a model that can
be used, given if we can’t get consumers empowered to shop on their own, then
they have got to continue to rely on these outside experts.

Barriers to consumer decision making, there are many, many, many. And we
need a really nuanced understanding of them, and I am only going to touch on
the tip of this iceberg. When it comes to identifying value and coverage that
is a good value, we have the underlying complexity of these insurance products.
These are extremely complex products. It is not surprising that consumers have
difficulty figuring out how much coverage they offer.

And this information is not presented in a uniform way. Everyone we talk to
thought well, even though I seem to have understood everything there, I think
there is other information buried in the proverbial fine print, which means I
have got it wrong. So they are expecting complexity and feeling like they can’t
get to this bottom line. As a result of this, and as a result of how profound
the implications of their decision is going to be from a health perspective and
from a financial perspective, they dread shopping for coverage. That is an
important barrier. People minimize and avoid things that they dread. If they
are going to dread shopping for coverage, you are going to have a hard time
getting them in there, playing their role in the marketplace. Forget shopping
on the basis of quality, so this is just a really big lift.

I already talked about how if your source of information is not trusted,
they are not going to use it no matter how well designed it is, or how
inaccurate it is or how anything it is. It is really key, I have already made
this point, but consumers cannot figure out how much coverage a plan offers,
and that is an enormous barrier, because this is a huge budget outlay for them.
And you are asking them to spend this money on coverage with a blindfold, with
respect to the product.

So identifying quality providers, which I think is a little closer to what
you are trying to do here. And again, I actually do not hold myself out as an
expert in this, but these are some things that came across the transom, if you
will. It is not part of the shopping exercise for most consumers, in part
because often that quality data, as Joyce said, is completely divorced from the
information they are looking at, with respect to coverage. Figuring out which
providers in the plan is very difficult. I think when Robert Krughoff is
speaking later, he may address this. But it is actually very difficult to get
accurate provider directory information into your health plan comparison tool.

And that means, even if you have identified a good quality provider, you may
not be able to link them back up to the plan. And consumers are not going to
call every plan or check every unique plan’s provider directory to see if they
are in there. So I think this is a major barrier that requires really close
examination. Consumers need, right when they are comparing plans, to be able to
filter based on which providers are in that plan, and that information needs to
be accurate and timely, so something to think about. Quality measures are
unfamiliar, would it be relevant to my case. Those are some of the points that
we have already talked about, wanting information from someone like yourself,
or trusting information that comes from someone like you.

My feeling is that this link between health plan, the type of health plan
information you view, and provider quality, is actually quite elusive. And I
think consumers caught onto this. How much control do health plans really have
over the quality of their providers. And it may seem that, through their
provider contracting process, that of course they have control. These are
health plans, they have got lots of data. They can go out, figure out who the
high quality providers are, and contract only with them or pay them more. But
the real world actually doesn’t look like that. They are just as stymied by
lack of useable quality information as a consumer might be.

In addition, when the contracts that health plans have with providers are
typically not revealed, but in Massachusetts the attorney general did a study
where she opened those provider contracts, much to the dismay of the plans, and
perhaps the providers. And she found that there was no linkage whatsoever
between what they were paying the providers and the quality or efficiency of
the providers. It was completely related to the relative market power of the
providers and the health plan in each locality. So I think we have to address
this head-on. Where does the health plan have control over quality, and where
do they not, and not sort of overstate what we can get, using that method.

Another barrier or concern is let’s say you have got a health plan, they are
doing a fabulous job using high quality providers or put all the right
incentives in place. But I think we all know that if all the different payers
are not aligned, we probably haven’t helped consumers as much as we want to.
The providers practice much the same way, regardless of who is paying them.
This has to be a universal effort to get them all practicing the same way.

DR. MIDDLETON: If all computers just worked the same way. That was terrific.
Thank you.

MR. FLAITZ: Hi. My name is Jake Flaitz. I am the director of benefits at
Paychex. I am here with my colleague, Jim Sutton, from Rochester General Health
System.

It’s a good thing, but I don’t I will repeat much of what has been said. It
has all been terrific. I will hit on a couple of things, but we are going to
take a few that talks about what we are doing as a community in Rochester, and
how that may make sense. It is real interesting, and I could give you my
perspective as the director of benefits of Paychex.

Again, just real quick on Paychex, we have got 12,500 employees, about 3500
in Rochester, we are in 40 different states. We have got a highly engaged
workforce in terms of the importance of benefits. Employees tell us all the
time, a number of our employees tend to be primary benefit earners, maybe even
more than primary wage earners, so benefits are very important. Nine out of 10
of our employees over the last four years have done things like health risk
assessments, biometric health screens, are engaged in their health. Yet, we run
into some of the same issues that Joyce and Lynn and Christine have talked
about in a very engaged workforce.

Just real quick, I think what is really interesting and what I absolutely
agree with is, inertia is an extraordinarily powerful force. It does not take
much to create a barrier that seems insurmountable for our folks as it relates
to getting health care information. From a community standpoint, we can go
about change sort of one person at a time. We do that at Paychex a little bit.
We go about it as a company. But we think that it makes a lot of sense to
approach it as an entire community.

I would also agree with what Lynn said, or it may have been Joyce, I forget,
where the trust in insurers, from an employee standpoint, is not very high. And
quite honestly, I would like to believe it’s higher than it is, but it is not
very high for employers, either. And I think that is one of the things that we
are finding in Rochester, is that the data that we are going to be giving to
residents of the community, to our employees at Paychex, is going to come from
the community, both providers and from the community at large. And I think that
is really critical.

My job right now is to try to set the stage for what we are doing in
Rochester. I will hand it over to Jim after I do that. Our vision in Rochester
is to take data, turn it into information to create understanding through
community engagement, ultimately to affect behavioral change. Key outcomes for
us are going to be increasing access, improving quality and eliminating
disparities that exists.

We are going to brag about Rochester just a little bit and just describe
what collaboration looks like in Rochester. In a high blood pressure initiative
that we have recently launched, we have 118 folks representing 52 different
organizations involved. To get the work done that we are doing, we have
organized ourselves into six standing working committees around things like
best practices, communications, community-engagement, planned design,
behavioral change in measurement. And then, we have had two ad hoc committees,
one around demonstration projects at the worksite, and another around
sustainability, how do we keep this thing going because it costs money.

This group, our core group, typically meets, Jim is a part of that, at least
one time a week for two hours each week. I have done that for over six years
with my colleagues. We work under the auspices of our CEOs, which I think is
really critical. They are very involved with this effort, because I think it if
was just me as the director of benefits, I don’t know that I would get a lot of
attention in the Rochester community. We are really proud of what we have done
and we are really proud of what we are going to accomplish.

There are some pretty interesting things about Rochester. I will not spend a
lot of time on this slide. I will definitely keep to my less than 10 minutes,
because Jim has got to come up here, too. But you can see what we like to call
the Rochester advantage. Costs in Rochester are about 41 percent below national
average, both on the commercial side, as well as Medicare. We have got good
access to care, we believe is evidence by low insured rates. And we have got
good quality, at least as measured by the 2010 hospital value index. So things
are looking pretty good, we are starting from a good base.

Some of the reasons why we believe Rochester has an advantage include just a
really strong primary care community, in terms of the supply of primary care
physicians, advanced medical homes, GME. We have got a legacy of community
rating, which has contributed to the advantage. We have got a low uninsured
rate, a long history of collaboration. We have also done a good job at
controlling capacity in Rochester, maybe too good of a job. Our hospitals
typically run well over 90 percent occupancy. We have done a good job on the
outpatient side of control and capacity, as it relates to things like MRI, CAT
scans, even sleep labs through an organization called CTAB, that I happen to be
a board member on.

When we started looking, this slide takes a look at what the Rochester cost
advantage, and what might be the opportunity for us. You can see the top bar is
the Rochester health care costs. The second bar is if we understood this
hypertension initiative. We believe that potential savings to the community now
are in the area of $1.23 billion. Our goal in Rochester is to have the
healthiest community in the United States. Pretty lofty, but we are pretty
aspirational in Rochester. Savings for the community would be in the
neighborhood of $7.5 billion, if we were to achieve that.

Here is the bad news. So we are starting from a base, but everyone else, I
think, every other community in the United States, we are seeing significant
increases in our health care costs. And quite honestly, this trend is just not
sustainable. It is not sustainable for businesses and it is not sustainable for
our employees or for other residents of the community. So it is something that
we have to address, we do not have a choice.

So I want to talk just briefly, before I hand it over to Jim, to talk about
the high blood pressure initiative that we are doing. We started out with a few
other initiatives. One was to create a regional health information
organization. We like to believe, and have some folks outside of Rochester to
tell us, it is the highest functioning RHIO in the state of New York. We have
got over 600,000 community members that have decided to opt into the RHIO. Now,
for the life of me, I do not understand why we could not pursue in New York,
and with that model, I think it would work a lot better. But we have had to do
an opt-in and we still have been able to get 650,000 people roughly in a
metropolitan area of a little over a million.

So we feel like we have set the stage for the transference now of health
information. A lot of providers are also involved. We launched an initiative
where some of my colleagues from Xerox, Kodak, and Bausch and Lomb had quality
black belt folks available to the three local health systems, to institute some
lean six sigma initiatives. They had already been in process. We estimate some
of the financial benefits to that to be about $24 million over a three-year
period.

Wegman’s, another member of our collaborative, I should just pause for a
second and say that this started out as a business initiative back in 2006.
Paychex, Wegman’s, Xerox, Kodak, Bausch and Lomb and the Rochester Institute of
Technology, were the companies that were involved. Wegman’s generously made an
initiative that they had started called the Eat Well, Live Well, available to
the entire community. They are pretty popular these days. It is a step in kind
of food programs, so you are counting, you are trying to get 10,000 steps a day
and trying to eat five cups of fruits or vegetables.

As I mentioned, they made that available in Rochester. We have done it at
Paychex. About 5000 of our 12,500 employees across the country participate. In
Rochester, we have had 400 local organizations with 200,000 employees
participate in that effort. That was sort of our wellness effort.

And we also increased the fill rate of generic medications. Now, Paychex at
the time, it was not an issue. We have had an extraordinarily high generic
utilization rate. That was not true in the community. We tried to extend that
to the community and have, in fact, done that. And we estimate savings
somewhere in the neighborhood in the community of about $200 million a year by
increasing the generic fill rate.

It is a real interesting thing what we were doing, and I am not sure if this
is going to happen or not. But it seemed to me that, as we were doing things
like the RHIO, which was focus on health IT, and Eat Well Live Well, focused on
wellness, generics focuses on medication, that we were both too abstract and
too definitive almost at the same time. It was not making a lot of sense, so
the question was how do we bring this together?

As a community, how does it make sense to the provider community, how does
it make sense, most importantly, to our employees, their dependents, to
citizens in Rochester. We felt like what we needed to do was to focus on a
medical condition and bring those elements together, to advance health and
health care in Rochester. And with that, I am going to turn it over to Jim.

MR. SUTTON: Nice segue, Jake. My name is Jim Sutton. I am a practicing
physician assistant. You would be proud of me, Christina. I was checking an
email of my patient while you were lecturing. I am also the director of
community medicine for Rochester General Hospital. RGH is the second largest
health system in our geographic area, the third largest employer. I am also the
chair of our metrics and measurement subcommittee of this high blood pressure
collaborative.

You have probably had time to read this slide, and in an effort of saving
time, I just want to get to kind of the meat of what I want to talk to you
about. Again, I think it is preaching to the choir a little bit, as to why we
chose high blood pressure. The devastating impacts of its complications, as
well as the potential of lowering blood pressure values through better evidence
practice.

I am sure you have seen this slide, it has been out for about a decade.
Interesting that half of the determinants of health come from health behaviors
and inadequacies in medical care. So we looked at this quite a bit in the
beginning of the collaborative, and started asking ourselves how do we go after
that large wedge. And we sort of asked ourselves, and with apologies to Dr.
Wagner, what would Ed do. And we kind of took the chronic care model and
disassembled it, and put it back together. I think the only thing that remains
there is the colors.

And we asked ourselves as a community how could we get in that sweet zone
right in the middle, that really special place where you have increased
customer awareness, best evidenced based practice, and all the components in
the community for better self-management. If any of you had a chance to hear
Don Burwick’s address in December, or read the manuscript, Don would look at
this and I think the takeaway in his message was you need to do everything. It
is not time for pilots anymore. You need to throw everything at this, including
the kitchen sink. And probably in many cases, the kitchen sink also.

We took an approach of one simple word, engagement, the Webster definition
underneath it. And if you look at these two columns on the left side, you can
look at some of the more traditional community approaches, some that have been
somewhat successful, most have failed in history, and look at what we are
trying to do in Rochester, in terms of community engagement. Rather than
focusing on the individual behavior of the citizens of Rochester, we are
looking more at institutional and organizational policy, rather than inviting
the community to the table. In our case, the community is the table and this
plays into a lot of what Lynn was talking about.

And when I do the next session, I am talking about our metrics and
measurements. The information is coming from the community, the changes are
coming from the community itself. Rather than us suggesting external solutions
to the problem, we are letting the community come up with the solutions
themselves. And there is a small focus on programs and services, but really
what we are looking at is building trusting relationships inside the community,
and asking them how can they engage into making themselves the healthiest
community in America.

Just briefly to wrap up, and I apologize, I will probably go over this very
quickly. Some of this will go into the next presentation. In terms of community
interventions, we started with an attitude survey, and we over surveyed into
lower socio-economic groups. We had a research firm do these surveys at bus
stops to compare this data to the internet portion of our survey.

And a couple of interesting things came from this initial survey. One was
that awareness is not a problem. Everyone was clearly aware of the fact that
high blood pressure is a silent killer, and clearly aware of the fact that
changing your behavior can forestall those problems that come with high blood
pressure. The problem is people were paralyzed with this inability to actually
make a change in their behavior, to make those healthy choices.

Another interesting tidbit from this survey was we found that most of the
consumers got their health information from the pharmacy, believe it or not. So
in terms of our community interventions, there was a vendor in our area that
had switched out all of their kiosks to put a newer model in, in traditional
places where they had kiosks. And the old kiosk was fine, it just did not have
the functionality of the newer kiosk. And we were actually able to get a low
cost alternative to purchase all these kiosks and pull them out of the
warehouse.

I started putting them in areas throughout the community, including barber
shops and salons, churches, in the workplace, at small independent pharmacies
where often they could not afford a kiosk. We started engaging the faith
community through some demonstration projects. We have had several
demonstration projects now going on in Rochester. These are based on
self-determination theory.

We happen to be lucky that, at the University of Rochester, we have some of
the perhaps world experts in behavioral change, Ed Deci being one of them. And
in these demonstration projects, we have taken peers, for example, in churches,
they could be deacons or peers in the church. In the workplaces, they are peers
in the workplace, and the same thing in barber shops and salons. And we have
ran them through some training, where they have learned, of course, about
healthy behaviors. But they have learned more about goal setting and how to
coach their peers in setting realistic goals and in achieving their goals. And
then, they have been let loose to go work with their peers, to help mentor them
and coach them along into their goals.

For example, my goals might be much more high level. As a practicing PA, my
goal may be to not get out of each room unless I have addressed the blood
pressure values in some way. As a director, my goal is to try to get to the
collaborative meetings every Thursday of my life. Someone else’s goal may be
just to obtain insurance or to make that first doctor’s appointment. And I
think to move beyond just giving them information, to activate patients, to
actually do something about it. The goals have to be realistic to them, and
they have to have a coach or a mentor come along with them and help them reach
those goals.

I think I am going to skip the clinical interventions. When I get into
metrics and measurement, I will be talking more about our community-wide
database that we have created to guide this project. And also, talk about some
of our academic detailing and our plan design. So in an effort to be respectful
to everyone, I think I will just hold off right there and let you guys go into
some questions. Thank you.

DR. TANG: Thank you so much to the panelists. This was very, very
informative, and depressing and inspiring at the same time. I think you really
summarized it well, the first few speakers, in terms of measures that matter.
And it is clear that currently, we have no measures that matter.

Regardless of what the measures are, they are at the bottom of the
consideration. You had talked about coverages, number one, I get that. And
looking for trusted sources, needing comparable information, I think
Christine’s story just sort of illustrated all of that. And then, Joyce and
Lynn sort of came up and said, well, based on surveys, that is exactly what
happens to folks. So we have to find a way of neutralizing the top
considerations, and that is why actually the health insurance change, if they
come to fruition, might help do that, so that some of these other measures
could float to the top and be considerate, if we fix them.

So thank you so much, and also the point of it has got to be measures that
matter that concern people like me. And we do not do that with population means
right now. It has been a terrific panel. We have five minutes to just sort of
circle around, but thank you for providing the written testimony and your
slides ahead of time, because I think those will have persistent value. And it
was nice to end on an up note, with Rochester in the sense of gosh, can we
actually get ourselves together. And it is just so inspiring to hear what you
have done. I visited, which is how I discovered Rochester. And when I heard
that the CEOs meet once a week, it just blew me away. And now, I remember that.
But anyway, let’s go around to see if other people have quick questions.

DR. COHEN: Just a couple of comments. This was great and I totally agree
with Paul, depressing and then inspiring at the same time. It struck me in the
conversations about how heterogeneous, when we say consumer, it is certainly
not one size that fits all. We have the assertive, smart consumer who has
access to the internet and all of these techniques.

And then, we have other consumers, citizens and residents who do not have
those kinds of abilities. And the question is designing different sets of
information to meet the heterogeneity of the consumer population. That just
makes the challenge even more complicated. And I guess I would like us to be
thinking in terms of not one size fits all, when we provide information. We
certainly need to do it in a multiplicity of ways, so that is one point.

The other point that I think Jack and Jim just raised is the difference
between providing information and activating decisions on the part of
individuals. I think we are at the bottom of this food pyramid, and just trying
to figure out providing basic information, the next step and challenge for all
of us will be figuring out how to activate individuals to use that information.

And then, the third point is the last presentation really is a wonderful
connection between some of the other work we are doing in the National
Committee, around trying to understand what information is available for
community health assessments. And integrating the notion of population health
emerging as sort of more than the sum of individual clinical measurements. I
really am interested in how we get from providing information to individuals to
make better choices, and how that ultimately rolls up into helping communities
decide what their priorities are, and intervening to make their entire
population healthy. So I look forward to this continued conversation. Thank
you.

DR. FITZMAURICE: I am struck by the need. Well, first of all, I really
appreciate your testimony. It kind of synthesizes everything. It says there is
a lot of work to be done, and that whatever work has already been done, we, as
consumers, do not know an awful lot about it. And I heard that virtually
repeated, repeated, repeated, but that we are trying very hard.

As a consumer, I am thinking I start looking when I have an illness for a
doctor or for a specialist. And so, I need a quality measure for a particular
illness. If I have arthritis, if I have a broken bone, if I have heart disease,
I need to have some quality measure that looks in my community for who is good
at making the diagnosis, who is good at managing the treatment through
medications, other specialists, hospitals, rehab, homecare.

And insurance companies are looking at that cost, but I am not aware of an
awful lot of research, looking at the quality of care and the quality of life
that results. Do I get a better quality of life if I follow this provider and
that pattern of care, or this provider and that pattern of care? So I am
looking at even beyond of what is needed to achieve more successes into, what
would be maybe the ultimate success for my looking at providers.

DR. HORNBROOK: Two context notes. First, historical, Ernie Sayward help
found the Northwest Region of Kaiser Permanente, and then he went from Portland
to Rochester, to start the Rochester Experience, and it has been something
unusual ever since.

The other thing is, I just want to remind us of the context that very few
relatively small portion of individuals have direct negotiation with insurers.
There is usually an employer, between them and the insurer, the insurer makes
most of the decisions in this country. And it is a perfect set up for
confusion. In fact, the whole marketplace is designed to keep the patient
ignorant for two reasons. One, the businessman, the employer, does not want to
deal with unions and workers advocating for specific insurance benefits, and
driving up the whole cost of the package. Their interest is keeping their
insurance costs lower. And the HMO does not want to have a lot of micro
discussions about benefits, because they use the small adjustments of various
utilization parameters in the benefit structure to achieve a target price.

And if you have a large group telling you, like California purse, this is
the price I want to pay, this is the price you are going to meet, then Kaiser
Permanente, for example, has to figure out how to meet that price because there
is such a huge place in the market. And for the individual market, of course,
it is really only feasible for people who are rather young, because the price
goes up so fast, unless you really cross-subsidize it. And then, the uninsured
never even deal with insurers. It is interesting that none of you talked about
how consumers talk to their employers.

MS. DUBOW: Actually, I think that is a really, really, really important
point. I think that employers really get off without a lot of the
responsibility that ought to be dumped at their feet, frankly. An insurance
company will sell whatever you want to buy. And employers who are looking at
the bottom line are not paying attention to the best potential experience for
their employees, they are looking to save money. That is not to say that I do
not value the contributions that employers make towards the cost of coverage.
But I believe that employers really are a locust of responsibility that is
often ignored.

The other thing I would like to point out, Lynn talked about the fact that
it is hard to identify the places where a health plan can be held accountable.
I don’t agree, because I think we have to force accountability at the health
plan level, because that is where the contract is. And I think that the fact
that we have health plan measurement, when CMS contracts with a Medicare
Advantage plan, it’s holding that plan accountable. Never mind that it is hard,
but that is the source of accountability and that is where our expectations
have to at least start.

So that when we look at the opportunities at the health plan level, we are
talking about workarounds in the absence of the infrastructure that is absent
in the provider community. The community supports the arrangements with
community services, the IT connections, there is lots that a health plan can be
held accountable for, in addition to performance, which I admit is challenging,
and consumers are interested at a more granular information. But the health
plan cannot be let off the hook, nor can the employers.

MR. QUINCY: I agree, I just want to be realistic about what exactly they are
doing.

MR. FLAITZ: I was just going to, as an employer, maybe give you one
employer’s perspective. And I will admit that it is not consistent across all
employers. But I mentioned that benefits to our employers are extraordinarily
important. They tell us that every day, particularly their health benefits. And
to give you an idea of how important it is in our response as a company, let me
just describe our coverages right now.

We have got a $200 deductible plan, a $400 deductible plan, and a $600
deductible plan. The majority of our employees are in either the $400 or the
$200 deductible plan. We have ongoing discussions with our employees to say,
our goal, my number one goal as the director of benefits at Paychex is to
provide the absolutely best plans I can that are affordable for both the
company and for our employees.

The other thing, I think, from a health standpoint and a healthcare
perspective is that my own personal opinion is that the impact financially to
the organization, in terms of the health of our employees, which relates then
to health coverage, is much greater on an indirect basis than it is for a
direct basis, that the costs of deductions and productivity and absenteeism are
much more significant than the direct spent.

The other thing that I think is critical, from a benefits perspective, as an
employer is we are absolutely certain that it both improves our attraction of
the employees that we want in our organization and the retention, based on the
benefits that we provide. So again, I am not saying that all organizations are
like Paychex, but that is our perspective.

DR. WARREN: I had three different observations from the panel. One, one of
my other tasks on NCVHS is I co-chair the standard subcommittee. And we had an
interesting task last year to come up with a health plan ID. And so, when I
heard you talking about health plans and the complexity and the confusion, I
thought we went through all of that. And I thought I knew what a health plan
was until we tried to define it for regulation. So very, very complex, and we
are still trying to figure that out.

The next thing is my own experience and where my own work is, which is
implementing EHRs. And I am still amazed at the number of clinicians who see it
as nothing more than a replacement for paper documentation, and that is the way
they see it. They do not see the vision of how it can coordinate care and help
make better decisions. And so, I think we need to do a much better job with
that.

And then, the other piece, the educator in me, as I was listening to you,
especially I believe it was Joyce talking about different ways to give
information. And then, I thought most of the clinicians that I know and work
with, and the educators that I know and work with, I do not think we are
teaching our students, med students, nursing students, etcetera, how to
evaluate what is appropriate and how it is appropriate to deliver the message.

And if they have more than something that is on an internet, which right now
most of them are depending, it is going to be on the internet, so your comment
of one size fits all, I think, is very appropriate. And we may have our own
literacy issues within the healthcare professions to deal with on how to
develop these tools, and then how to select which ones to use. So we probably
should think about that as we continue our deliberations.

DR. MIDDLETON: We have extended our time just a few minutes into the break.
And we would like to ask for your forbearance because I just have a couple of
burning questions. And then, we are going to give some time. Perhaps we will
break in just three or four minutes, and start at five after. The same people
will continue.

But one of the observations I had was certainly we have to consider this
proxy decision maker. Oftentimes, in my own patients, it is never going to be
that patient who is going to be the decision maker, you just know how it is. It
is the same way I am with the auto mechanic. Something is wrong, just go fix
it. And I would ask those who wish to comment on how do we address those
measures that matter for the proxy decision maker, where that is the case. Are
they the same or is it different?

MS. QUINCY: I think it might have been Bruce who was talking about the
heterogeneity in consumers or people out there, whichever term you want to use.
So I think we do not know, it is likely they might not be the same. And I don’t
have really any great insights, except that I was compelled by what James was
saying, which was a community-based approach. In other words, do not put all
the burden on that decider. Provide this supportive environment that provides
fewer pathways to go wrong.

MS. DUBOW: It depends what the issue is, first of all. But everybody assumes
it is the woman in the family who makes the coverage decision, or the caregiver
who helps make treatment decisions for an older person, for example. Or the
employer making a decision about what the health plan coverage will be. I think
that these proxies are usually just, if at all, one step ahead of the patient
or the consumer, and they need the same kind of information. I don’t think we
need to worry about differentiating at this point. I think they need to have
the same types of information that are, as Lynn points out, accessible, easy to
use, understandable, actionable and appropriate.

MR. FLAITZ: I don’t know if you were thinking of this or not, but one of the
proxies that I think we can do a better job with is actually plan design. Plan
design, as it looks right now, we essentially cover everything the same, all
inpatient care is covered the same, all outpatient care, with distinguishing
and making it easy for employees and their dependents to understand what is the
best pathway for them. And I think as employers and as health plans, we have an
obligation to take a look at that and improve upon it.

MS. BECHTEL: I understood your question to be specific on quality measures;
was that generally the context? I think the different contexts, completely
different answers. And I actually think it gets back to Mr. Fitzmaurice’s
question around disease-specific measures versus others, and the absolute
differences across consumers and their proxies. And so, I tend to lean towards
wanting measures that are not necessarily condition-specific, because I think
it is going to be hard to have a huge proliferation of measures that somehow
accommodate everyone. Do I think I am going to find measures of how a good doc
is on spondylolisthesis? No. I didn’t even know how to say that word until 10
years ago.

But what I do think is that measures that, as Joyce talked about, give a
better sense of the clinicians’ diagnostic skills, their approach to treatment.
So for me, it is kind of this combination of functional status, quality of
life. At the end of the day, did the outcome improve, regardless of the
condition, combined with what was the experience like, because what we know is
that there is a direct link between patient experience and an impact on health
outcomes.

DR. MIDDLETON: That is terrific. I just have to get one last one and then we
will take a good 10 minute break. It is striking that across the country, you
see many examples, or an emerging number I guess, an increasing number of
examples where something happened in the community, some seed, some fire, as
Berwick would call it, was lit. And then, things happened.

So in Rochester, what happened, what was it that those CEOs saw or when did
they meet, and why did they even sit around the table, the very first time. It
strikes me, if you get that right and get it going, then all of the rest of it
could be simply downstream.

MR. FLAITZ: I think the impetus was probably twofold. One was cost, that got
their attention, and in a survey that the Rochester Business Alliance or the
Chamber of Commerce does, health care cost has come up as a number one issue
for employers, probably for at least the last 10 years. I think the other issue
that had happened, too, is what had gone on in Rochester. I came to the
Rochester community in 1995 and had a long history of collaboration. And then,
went through a very competitive phase that existed in a lot of other
communities across the country. So we kind of changed our path, and we find
that that did not work, or least it wasn’t consistent with sort of the
Rochester’s DNA. And there was a desire among the CEOs to move back to a more
collaborative model.

And then, I think as part of that, what they saw was from a business
perspective, we could not do this on our own. We could not impose our will,
both on whether it was on our employees or their dependents, or on the provider
community or on other organizations. The only path forward was to do it
collaboratively. It is hard work, it is real hard work. And I think the one
thing that I have learned is, I think businesses often have a real need for
speed. I wish this was a sprint, but it is not a sprint. It is an
ultra-marathon, probably at best. And yet, we have got to go as fast as we
possibly can, and still kind of pace ourselves.

So I think it was those two things. I think cost was the driver, but we
recognized the sole focus on cost was not going to get us to where we wanted to
be. We have got to be looking at essentially improving the health of our entire
community. Quite honestly, it makes what I do at Paychex easier, my colleagues
at Xerox and RIT and Kodak and Wegman’s and Bausch and Lomb are doing the same
thing. It makes it even easier for our group if the entire community is doing
it.

DR. MIDDLETON: Let me just echo Paul, say thanks, and why don’t we call it.
It’s 10:57, we are going to call it 11:00 and reconvene sharp at 11:10. We will
have to stop sharp at 12:30 for the lunch break. Thank you, everybody.

(Break)

DR. MIDDLETON: Hello David. This is Blackford Middleton and Paul Tang, and
the NCVHS Quality Subcommittee. Thank you very much for joining us. We are just
reconvening. This section is on measures and data to support health and
healthcare decision making. As we are starting at 11:10, I am going to ask
everyone to be just a little bit, 110 percent speed. And we should be finishing
up on time at 12:30, which we will need to keep to so everyone can have a break
for lunch. And is this the correct order of speakers? Take it away, Dr. Lansky.

Agenda Item: Measures and data to support health and
healthcare decision-making

MR. LANSKY: Can you hear me okay now? Thank you Blackford and Paul and
everyone for letting me join you today. I am sorry I cannot be there in person.
After this morning’s great start, I want to come with a measure of hope and
encouragement. I think we have a lot of experience to say that the measures
consumers care about can be developed and used. I think it will take a
commitment by this and other groups, but I do think it is possible. We know
enough to move this agenda along pretty rapidly, if there is a will to do so.

What I wanted to do in my time today was primarily talk about a methodology
to produce the kinds of measures that consumers are asking for, as input to the
overall process and strategy we are all considering. I am really representing
two groups here. I am currently working with the Pacific Business Group on
Health, which as you see on this slide, is a number of large national
employers. And they have a very strong interest in providing data, as we heard
on the last panel from Paychex, directly to their employees to help them make
better decisions, both about their insurance coverage and their provider
selection and their treatment selection.

And we have done a number of projects and activities to help the companies
and their employees in this regard. Over 20 years or so, PBGH has published
websites, publications, worked for state contracts. We currently provide a
health plan choice software to about 2 million people to help them choose their
health insurance coverage, as we heard earlier this morning.

And we have been working the last couple of years with direct measures of
physician quality, generating measures for about 15,000 California primary care
doctors primarily, about their individual quality scores. But alas, that data
is based on claims data, which is of course the only source we have that is
pretty much universal, and has many problems we are all familiar with. But we
have been trying, as an organization, to go down this path for a long time.

The next slide also mentions the work that that I done for some time. Many
of you are familiar with, from the Foundation of Accountability, I think Joyce
mentioned this morning, which was the consortium of patients, consumers and
purchasers for about 10 years, working on developing quality measures with a
very particular eye toward what consumers were looking for and patients were
looking for. In the course of that 10-year period, we did about 100 focus
groups, 55,000 surveys of patients, many interviews and other formats to try to
understand what patients were looking for. And then, we spent about five years
looking at how consumers would use this information to make decisions,
including on behalf of the organizations listed here.

I want to try to capture some of these learnings from that time, and just
present it for your consideration today, as sort of the launching pad for
obviously what would need to be another round and generation of additional
research. But I think it gives us a foundation upon which to build. The next
slide summarizes very high level findings that I want to share with you from
that work. First, patients, when we talk to them in depth and across a broad
range of experiences and health conditions, generally believe that quality is
shaped by their doctor. And they want information about their doctor and the
types of doctors that they get care from.

The other abstractions all of us worry about, health plans and ACOs and
delivery systems and medical groups, are just abstractions. They do not really
speak very well to most patients. We can have an argument that they are
important sources of variation and performance, but it is not easy to translate
that to consumer decision making.

Secondly, patients with chronic illness associate their specific condition
and the expectations, the care processes and the outcomes associated with their
condition, and that is what they want to know about. They are not that
interested in generic performance domains, although I will give you some data
in a minute about that. They do understand that quality is multi-dimensional,
and they are interested in all of those dimensions of quality, so that is an
opportunity for us to use various kinds of information to help support better
decision making.

They do want quality information when they are making decisions. They want
to go to good doctors and doctors who are accessible and doctors who
communicate well and doctors who know what the best science is and follow it.
There is a thirst for that, if we can find a way to deliver it. But it has to
be relevant to their specific decisions. At one point, we did a series of
decision maps where we traced all the way from pre-diagnosis to long-term
outcome. All of the decisions that patients need to make in the course of their
care, and what information they wish they had at each of those decision points.

And the point of that exercise is that it is doable, it is knowable. As
Christine said this morning, it is challenging to then deliver information just
in time for those specific decisions, and the type of information or metric
that we think is reliable. But it is not impossible to do that mapping and
target information. And obviously, in other sectors of our society, marketers
do that extremely well.

People want to use performance results to guide their interaction with their
doctors, more so than to choose a new doctor. And they also do not want to be
told, despite the correct discussion about negative framing that I think Joyce
presented, they do not want to be told that their doctors are bad. They want to
be told their doctors are good and there are places their doctors can improve,
and here are the places you can ask your doctor to be attending to and
improving. It is a very complex use of information. Much as clinicians have a
complex use of quality data, so do patients. It is not simply a thumbs-up,
thumbs-down kind of a desire.

We asked patients very broadly what their general notion of quality was, and
they gave us the words that you see here to describe good quality care, a lot
of it being about the doctor, and the words you see here to describe bad
quality care. So they do have some broad expectations and conceptions of what
is good and bad care, and some of these things we heard in Christine’s story
this morning.

We also then drilled down and talked about what specific expectations of
care people have voiced. They want a doctor who is experienced in their
particular needs and disorder, can answer their specific questions. Their own
experience of the care interaction is as important to them as technical aspects
of care, which of course they do not feel as competent to judge. And they want
to have a partnership with their doctor, they want to be communicating about
these quality issues, not simply looking them up on table and voting them up or
down. And the sidebar of quotes, this happened to be from some diabetes
patients, gives you just a flavor for the kinds of things people seem to be
looking for, as they think about what is quality for them.

The next slide takes that kind of inquiry a little bit deeper, and takes the
specific example of HIV patients. And I think what was interesting about this
set of groups we did with I think three communities and a wide variety of
patients of different ethnic backgrounds and socioeconomic status and types of
conditions with their exposure to HIV, was that we did a panel of experts,
people who were very sophisticated, clinical and research experts in HIV care,
and you see the way they ranked. This was some years ago, but this was the
time, 10 or so domains of quality that we asked experts, which of these should
be included in a quality measurement set.

And then, we took the same list to a group of patients, a very diverse group
of patients, and had them rank the same potential quality measures. And what
was interesting to us is where they differed. In some ways, there are some
similarities, in the overall higher ranked items and the lesser ranked items.
But you see particularly the experience of care involvement in decision-making,
and the measurement of functional status, which I know you are talking about
later today. Patients ranked it third most important, and the clinicians ranked
it tenth out of 11.

So in general, what you observe from the patient ranking in this particular
study is first of all, patients can do it. They can tell you what is most
important to them. Their judgments are not irrational or not responsible. They
are sensible and they generally concord with the kinds of things that experts
care about. But they are different, and valuing the patients’ votes and
recognizing from the patient point of view these three things, a reduction of
the opportunities for infection, being involved in a respectful communication
about decision making, and knowing whether or not their ability to function may
be maintained. Those are the three most important quality measures to this
group of patients. It is an important opportunity to listen to the voice of the
patient, as we craft the measurements that are going to be used for public
reporting and otherwise.

The next slide goes back to something I think Joyce mentioned, the
opportunity to group information in ways that make sense to people. And again,
this was an older framework and probably should be revisited. But my suggestion
is that it is possible to talk to consumers and patients, and ask them how they
think about their care, what stages of health and illness experience they go
through in life, and then categorize the information we all think is valuable
on quality into concepts and domains that make sense to people. And at some
point, of course, you can create composites which specifically give grades, if
you like, to these domains.

This first categorization, the basics, staying healthy, etcetera, was
developed to try to capture at a large aggregate level, such as a health plan
or an ACO now, the broad domains of healthcare performance. And this is a
little paradoxical, because for someone who is primarily concerned about their
heart disease or their child’s asthma, they may not be looking at all five of
these domain. But if you are looking at the performance of the health system,
you may want to look at broad domains that communicate them broadly. I think
this is an important debate for the committee and others to have.

The second set of domains are really meant to cover the grouping of
information that you will see in the next slide, into three clusters that could
be composites for the most part. So are my providers following the best known
evidence, the best clinical practices. Am I getting an experience of care, or
likely to get an experience of care, that is respectful and that I can
understand what is going on and I can be involved in my decisions. And thirdly,
am I getting good outcomes, getting good results.

Those three big domains are voiced by patients from every health condition
that we explored in that previous work. So we have tried to keep addressing
those three domains, and if you look at some more recent dashboards, for
example, Dartmouth’s work on developing a dashboard, maybe John Watson will
speak to it, they also continue to use these three general types of domains,
although, often now, with the added one of cost or efficiency.

I thought I would give you just a flavor for how this plays out. When we
talked, for example, to many, many breast cancer patients and tried to
understand what they thought were both good process measures, the steps to good
care, what were the experience of care measures, and what were the outcome
measures in those three clusters, it was possible to populate a set of
measures. Here you see seven or 10 measures that would be constructed from a
measurement set around breast cancer care. And this covers actually a fairly
broad set of stages of the treatment process. And if you drill down into that
experience and satisfaction box, or the results box, you will see a number of
subdomains that people were concerned about. So these measures spoke both to
what patients said and what experts and researchers said.

The next slide gives a second slice of that for a less acute illness here,
chronic asthma care. And again, you will see the three buckets saying process
measures, experience measures and results measures being reported. And in each
case, you can imagine taking the sub items, the A items, B items or C items,
and grouping them into a composite that would be something that patients could
relatively, simply understand as a rating of performance of their doctors or
their care system on caring for asthma.

And again, if you look at the detail here, the kinds of things patients
talked about wanting to know were very heavily weighted towards system
management, and can my child go to school, can they play sports, can they
maintain regular activities. Those were the critical representations of quality
for patients when we talked to them about asthma care.

The next slide is a little bit of a caveat, and I think this raises a
question that comes out of the comments Christine made earlier. When we were
doing this work at FACCT, we really wanted to believe that we could identify a
set of competencies or constructs that were generally true of a practice or a
care system or a health plan even. And we actually went and did this work in
both Minnesota and Indiana at a fairly granular level, down to the medical
group level, and discovered that our hypothesis was unfortunately not true.

And if you look at the detailed scores on this small table, in this case
these were actually small health systems, whether their delivery of patient
education score was consistent across conditions. And if you look at the three
health systems here, one, two and three, and you look at which one scores
highest, let’s say for asthma, you can see that plan three got the highest
score in patient education, but plan two got the highest score for diabetes
patient education, and plan one got the highest score for heart disease
education. And they all performed lower in asthma patient education than in
diabetes patient education.

There was no easy way, and I think this data when we did it in Minnesota for
care systems was actually even worse. There was no ability to say yes, care
system X consistently performed on the competency of delivering patient
education, and we can now tell people in the community, you can have confidence
that care system X will do a good job, regardless of which condition you have.
It did end up being specific to the condition. That may or may not still be
true, and it may be more true in some communities or some care systems than
others. But I think it is a caution to us all about how our beliefs about the
care systems we interact with may or may not actually be empirically true, and
maybe if nothing else, it is a reminder to verify before we assume that our
measurement theories are going to be realized.

The next slide just very quickly recapitulates the way FACCT went about
developing measures, and this is where I want to say a word of hope. This is
very simple, these six steps. And really, the key thing here is to begin with
consumer input, talk to patients, talk to consumers, talk to people,
caregivers, others who are affected by the illness or the treatment process
that you are concerned about. In parallel, look at the available literature and
the methodology, and convene the experts.

And our approach was always to have the very constructive interaction
between the experts and the patients, and go back and forth between them. We
always were really encouraged by both groups ability to understand the
legitimacy of the other point of view. There was often considerable variability
between the two points of view, but a very constructive dialogue. And I think
even more so now, there is a great respect for the voice of the patient that we
want to encourage. And I think this kind of an approach can be taken forward in
the future, to begin with the voice of the patient, validate with the voice of
the patient throughout, and generate a set of measures that are mutually
acceptable.

The last slide, just to summarize a couple of the points I think that we
learned in our work. First, keep listening to the patients. Their voice should
be the primary voice determining what is measured. They are the ultimate users
and payers of our healthcare system, and we should make sure that they are
getting what they need to know from the work we all do. And we should respect
that voice and can understand its legitimacy, even in the context of perhaps
disagreements from experts and clinicians.

It is unfortunately true that what matters to patients is what is relevant
to their specific needs and circumstances. It is possible, given that, though,
to still group data into some kinds of measurement sets that speak to a
patient’s questions. People do not want to know just whether the HBA1C is being
monitored. They really have a set of issues around their diabetes care, for
example, that needs to be answered as a cluster.

Obviously, the data sources, I know, is a question that we all wanted to
talk about today. The data sources are a significant problem. The existing data
sources that are often accessible, even EHRs as they become more accessible
now, were not designed to capture the information that patients want to know. I
think we have to say that to ourselves over and over and over again. Even the
EHRs that we are all working so hard to proliferate now do not capture the
information the patients want to know. So it is not possible to have measures
that matter to patients if the data sources do not capture the data people want
to know. We have to come back to that question. I know certainly on the policy
committee, we are having those discussions now.

Finally, think all of the work we did suggests that we have to think about a
measurement system, not about measures. Individual measures that we do spend a
lot of effort fine-tuning are really only sensible in the context of an overall
understanding of people’s experience of care and need for information. I will
stop there and look forward to some discussion later on. Thank you very much.

DR. MIDDLETON: Thank you very much, David. It was terrific. We will move
right along and have some questions at the end.

MS. DUBOW: I will just start here. I think David so nicely said, I agree
with what David said, that we know what we need to do, we just have to get on
with it. I think you are going to hear a lot of us saying the same thing, in
terms of the kinds of measures we are looking for. Christine already
foreshadowed that by talking about functional status, etcetera.

I want to just remind us that there is value to measurement, to consumers
that goes beyond their own decision making. So when clinicians use information
that results from good measurement, patients benefit because they get better
care. The opportunities for quality improvement that are identified through
measurement illuminate our understanding of areas that need attention. We
cannot do payment reform if we do not have good measurement. We need to be able
to align the incentives properly, but we need the results of good measurement.
The use of measurement for judgment, and also for reassurance, I do not think
we should ignore the opportunity for reassuring the public that some care is
actually pretty good by just presenting information, the results of
measurement, and finally, surveillance.

We measure, even though consumers do not use information. And I think that
often the fact that consumers do not use the information is a rational response
to the lack of usable, actionable information. I talked about the
information-seeking burden before, but public reporting has value. The results
of publishing the measurement information is valuable, whether consumers use it
or not, because we know that providers are using it. And we also know that even
if just a small proportion of people who use the measurement, we can see
improvement. So consumers benefit even if they are not directly using this for
their information.

If we do expect the measurement results to be useful to consumers, it has to
be meaningful. So we need to be sure that the measures that are published are
consistent and valid and reliable. It does not help a lot when consumers are
confronted with dueling scores. The various websites that people go to that use
different data sources simply confuse, and I think they essentially result in a
turnoff. If people do not see the same information that is consistently
presented, they do not know what to make of it, and they just blow it off.

It is very useful, the national efforts around standardized measurement,
that have an underpinning of good evidence, and that are consistently
specified. I think it is very, very important that we use these kinds of
measures in our state and local initiatives, as well. It is not to say that we
cannot have additional measurement at the local level. But when we use
measures, we do not need 25 measures on whether blood pressure is controlled.
We need consistent measurements so that people can get a score and be able to
rely on it.

People want to have confidence in the results, so that the data ought to be
audited or at least somebody ought to be overseeing data integrity. And the
reports need to be timely, the measurements that report on performance for
three years ago is not necessarily relevant to current decision making.

You asked us to think a little bit about measure focus, and as I said, I
think you are going to hear the same thing from most of us. Patients are
interested in outcomes. Functional status is absolutely key. I thought that
David’s slide about where symptom relief fell when compared to others was
interesting. I think people are interested in symptom relief. These are all
testable, though. These are all researchable questions. I think that obviously
we need to talk to patients, as David said, to understand what people are
looking for. But I think that we know from a lot of work that has already been
done that people are looking for outcomes that address quality of life and
complications.

I cannot remember, one of the members of the committee talked about
information that spans an episode. That is truly the most patient-centric way
of looking at something, the way a patient experiences care is across an
episode. If you have a heart condition, seeing isolated measure does not help
somebody understand how to address her condition. So we have to have measures
that look longitudinally across settings. That is how a patient with chronic
condition experiences care, and the measurement that we report to patients,
that we provide to patients, ought to be looking at that.

Patients are interested in knowing whether their preferences are recognized
and valued and solicited, and that is an important measurement for consumers.
Same thing with support for self-management, although again on David’s slide,
it did not say that. We know that from older people, that was an HIV example
that he gave. It is very, very important for people to understand that they
participated.

As I mentioned earlier, people are interested in physician level
performance. I know that this represents a methodological challenge, but people
want to know how good their doctor is, period. They are looking for a good
doctor, and that is the information they want to know. And they want to know
how good is their doctor. Who is the best doctor to do X, and how good is my
doc. And as we talked about before, cost is really very important.

This is another list that does not represent patient reporting outcomes. But
I think these are measures that are important largely because they look at
issues around efficiency and resource use, which I think are ultimately very
important to patients, because the money ultimately comes out of patient
pocketbooks. People, not employers, we all pay for it in the end, as either
taxpayers or users, because these costs all get passed along. So inappropriate
use of emergency rooms, for example, medication reconciliation to avoid
inappropriate or duplicate contradictory prescribing. Hospitalization, Arnie
Epstein wrote a very interesting article recently about how the propensity to
hospitalize is really more important than the readmission stuff. It is the
propensity to begin with. So we are interested both in avoidable readmissions,
as well as the hospitalization itself.

You can see that information about inappropriate use, overuse of services
that are costly, that are potentially safety issues for patients. And then,
below the line, if we are looking for quick helps right now, maintenance of
certification on an individual physician is very useful. Plain old
certification is no longer a good standard, because virtually all physicians
are certified. But whether they are maintaining their certification, that is
something that we ought to be saying. There is a whole host of physicians who
are grandfathered, that is not so useful. People want to know that their
physicians are keeping their skills up to date, and that they are meeting
certain standards that the boards are prescribing.

So maintenance of certification would be a good thing, just as maintenance
of licensure if it were an ongoing process at the state level, would be a very
useful measure. And then again, for people with chronic conditions, knowing
that a plan, a provider is using community resources appropriately, making
these non-clinical services available. That is a helpful indicator for
patients.

Data collection should be parsimonious again because ultimately consumers
bear the cost. I do not think we should waste time with what I call junker
measures. And so we need to be very careful about the measurement system that
we develop. We should not be wasting our time with process measures that do not
have a known relationship to an outcome. We should not be paying for stuff
around documentation. Sure, they are important, but these are not measures that
ultimately are salient and actionable for consumers. They are assuming that
this stuff is happening.

Again, to be parsimonious and to be efficient, ideally the information is
going to come directly from the practice of the clinician. I cannot emphasize
enough the importance of having information on race and ethnicity. We have made
progress. There are standards, but we are not getting these data out and we
cannot address the disparities in health care if we do not have the data
collected, so they are absolutely key. I might add that there are other
disparities, aside from race and ethnicity, like gender. We know a lot about
geographic variation, but socioeconomic status. There are other bases for
disparities, and we need to understand them. Age, how could I forget age.

We know that most data now are coming from administrative data because they
are there and they are cheap, and they reduce the collection burn. But if they
do not have any clinical information attached to it, they are not really useful
in giving consumers the information that they need. Obviously, we all hope that
at some point that we are going to get to the point where we can get what we
need out of the electronic record. But we are not there yet and those systems
have not really been designed to give us the kind of information we want, as
David mentioned.

Patient reports, the suite of CAHPS instruments, is a very, very important
platform to amplify and to go beyond. I notice in the current call letter from
Medicare Advantage plans, CMS is talking about enhancing CAHPS to get better
information on care coordination at the physician-clinician level. That is
really important and people will really value that. Is the information at the
visit, so that the patient doesn’t have to provide the information again. Those
are the kinds of questions we can ask patients and get some really good
information. The patient is an excellent source of a lot of information we
cannot get any place else.

Key gaps are, as I say, from a patient perspective where they want to know
physician level data. I know, I have a colleague from NCQA here, we cannot
always get physician level data. Sometimes we have to settle for the practice
level. But the most salient unit of analysis for an individual patient is the
physician, and we cannot forget that. Patient-generated data, this as a source
is a goldmine that we need to tap and we need to use. And again, cost to the
individual, we need measurement in that area.

There are data, as well as implementation gaps. At one of the last MAP
meetings that we had, Christine took great umbrage that somebody said that
patient experience was a gap area. And she argued, and I agree with her, that
it is not a gap area, it is an implementation gap, not a data gap, not a
measurement gap. We have CAHPS instruments that can measure the physician, the
clinician level. It is just not used, and I know that Robert Krughoff will talk
about this later, I have to believe he will. We have this instrument and it is
just not being used. This would be really, really useful information for
patients.

We need measures on shared decision making. I know that the Foundation for
Informed Decision Making is working on this, but it is a challenging prospect.
We need to do more in this area. I think the bottom line here is we need a lot
of research. Somebody took issue with the idea of having condition-specific
information. But I think people want to know that when they are making
treatment decisions. They want to know that. I do not think we can afford it.
We have huge gaps in geriatrics that we are just not addressing.

Again, the information on race and ethnicity is key. And we need measures on
appropriateness, because we do not have them. I think this is a research area
because I think we have a lot of work to do in this area. I think the bottom
line is that we have to start with what we have and keep getting better as we
do it. The big challenge at all of the quality measurement tables is whether
something is good enough. We need to decide what is good enough and get going.
It will get better as we use it. I think we already have a history of that. I
think we can start with refining the measures we have and keep moving and keep
improving. I think that is it.

MS. BECHTEL: Thank you very much and great presentations by Joyce and David.
I am going to move pretty quickly through some elements of mine, because I
think they are well-covered in some ways. One of the questions that the
subcommittee asked was what measures would be meaningful. We argued that, of
course, just as David Lansky said, and Joyce as well, that we actually have to
start by figuring out what is meaningful to patients by talking to them. So we
did that, and what we found was that we have these great definitions from
places like the Institute of Medicine and PlaneTree. But when we talked to
clinicians and others, there wasn’t a real sense, but what does it mean in
practice? In the real world, what does that mean?

We did a couple of things to pull together. First, we started with a set of
consumer and patient advocates from the state and local level, who their sole
purpose is to help patients navigate the healthcare system every day, and get
their needs met, so very close to patients and their families. And then, we
actually did a series of focus groups in a national survey that was designed
not really to define patient-centered care, but actually to figure out how you
talk about delivery system change publically with consumers for things like
medical home and health IT and payment reform that we have no real good public
language for.

But to do that, you had to start by talking about the healthcare system. And
so, we were able to pull that work together and organize it into what I would
describe as more of an operational definition of patient-centered care,
according to consumers. So it has four domains, whole person care, coordination
and communication, patient support and empowerment, and of course ready access.
And when we sat down with patients and their advocates, one of the things that
we heard over and over again was, I am so tired of being looked at like a
collection of diseases and body parts. And if you do not understand everything
that impacts my ability to get and stay well, you cannot possibly make
treatment recommendations that I have any hope of following.

So you really need to understand me as a person. You need to then not only
record that, but remember that in the future. People talk over and over again
about sort of showing up at the doctor’s office for the millionth time and
getting the dreaded clipboard and being related to as if you are coming from
another planet.

The other thing I would say, one of the most common comments to come out of
the focus groups that we did, was look, I just want my doctors to talk to each
other, very simple. And we, inside these rooms, know that there are lots of
reasons that are complex that that does not happen, but it is a very simple
request. And so, they talked a lot about the notion of help me find a
specialist, don’t just say you sort of need one, even help me get the
appointment. Make sure that they have my information ahead of time, make sure
that you understand that person’s recommendations and that I get it, too. Help
me move around and navigate the healthcare system in a smooth way, because you
guys are the experts here, right.

They also talked about patient support and empowerment. And this was well
beyond the sort of compliance adherence debate. This is about, and Joyce
mentioned it as well, self-efficacy. Do I believe that I have the capacity to
change, I can change, do I have the tools that I need to change. And then, is
there a partnership with my clinician where we can set goals that matter to me
together, where the goal is not get your core strong. The goal for me, if you
are asking me, is going to be play 36 holes on a Sunday without pain. That is a
big goal, but I am motivated.

And then, of course, an environment of trust and respect. I was really
surprised over and over again, when we talked to patient advocates and to
consumers and focus groups, how there is still just a real basic lack of trust
and respect in a lot of people’s experiences, whether it is respect for their
privacy or respect for any sort of facet of their health conditions or who they
are as a person, or even the way that they are received in the waiting room.
And it has an impact, all of these domains, on clinical outcomes. And access is
the fourth domain, and again, very much impacting clinical outcomes.

So consumers really wanting to have more and better access outside these
really constrained five minute, seven minute office visits, phone calls, things
like that. Wait times, brief, accommodating limited mobility, cognitive
impairment and language barriers or cultural differences were also very clear
and prevalent access problems that people are still encountering.

I took this for the purpose of the panel, and tried to organize some
examples. And you heard Joyce talk a lot about different kinds of measures, and
David, as well. But just some different things that would matter more to
consumers, organized by the framework that I just gave you. So under sort of
whole person, high quality care, things like patient-reported outcomes, and
condition-specific composites. So it is sort of that like me, but also
concordance with values and preferences. Did the care I get and the way I get
it really match with what I was looking for in my own values and life
circumstances.

Coordination and communication, big gap area, we all know. We have some
measures, but they are not great. What I really want to know is how well are
those doctors talking to each other. How well are they functioning as a team,
or doctors, nurses, all clinicians. Patient-supported empowerment, we do have
some activation measures, and Joyce mentioned shared decision making and
measures of decision quality that we are sort of getting to, but not quite
there yet.

And then, of course, we have actually a fair number of access measures, as
well. Some of them were reported in, for example, the websites I looked at my
search, sometimes language and sometimes wait times. But there were not cost of
care or sort of those structural access things that were reported that I could
find. And I had an arrow for CAHPS because that does have a lot of questions
that relate to each of those domains.

What you see on your screen is actually what I thought was, this was
something that I came across while I was doing my website homework to find a
new physician. And I just thought it was a terrifically wonderful illustration
of the patient-centered care domains that I just illustrated. This is a patient
who has multiple chronic conditions, who keeps them on a piece of paper that
she prints out at her computer, takes them in. And the physician that she sees
does medication reconciliation, some degree of formulary checking, and she
talks about how he knows and remembers all that we have gone through together.
She talks about feeling as though I am his only patient that day, he smiles
when I make a joke, there is a lot of respect. But I loved the last line. And
she said, I go in and my blood pressure goes down. I smile and I know that I am
going to get well. Great example of patient-centered feedback, and that came
from one of my websites.

As Joyce talked about, a lot of these measures are either way too basic or
they just do not exist at all. And I think they also, in part, maybe why they
do not exist is because improving performance on many of those kinds of
measures requires that clinicians do things they don’t get paid for in fee for
service. They do not get paid to do care coordination. We have all these
accountably debates, and a lot of these measures, you have a patient
involvement role as they should. And so, need I say more.

It also requires a major culture shift. I come back to my case story of one,
but the fact that I am offering to help my doctor’s office improve and change,
and it is why I hung in there so long because I was so hopeful that we could do
something together. No. So we have to really shift our culture, and it starts
by viewing patients as the center, inviting their involvement in quality
improvement and in measure development, as Joyce talked about. It also, of
course, requires infrastructure that is out there. CHECKBOOK has figured out
how to have a data platform for surveys. But forbid that that is somehow at the
individual physician level.

So the business case, obviously who would invest in these kinds of measures
when they are impacted by multiple clinicians on the care team and on the
patient. And there is not always a clear case that a health plan or an employer
is going to get sort of like a bigger market share from them.

Finally, I do want to say, we have to address patients’ low expectations.
When we had these focus groups around care coordination, and they talked about
carrying binders and binders of paper around, from doctor to doctor, and nobody
would look at even the perfectly beautifully organized summary sheet that they
would present, and they could not get them to think about even talking to
someone else.

The things that they described were really unbelievable. But they were not
in a position where they were like, we have to fight this and this has to
change. Their position was, that is just the way the system is. They were
absolutely resigned to it. It was really fascinating, from an advocate’s
perspective, to see. But as I thought about how do we change those
expectations, the current suite of quality measures I find to be relatively
uninspiring.

Of course, you guys have probably seen, but I liked it that RAND looked at
the 11 different kind of payment reform models that they think are sort of
coming out of the Affordable Care Act. And these are the ones that could have
some real potential to change the health system for patients, and found that we
still have gaps in the five areas that you see listed.

And then, finally and I am really going to go very fast through this, you
can find a much fuller version of this document online. But consumers and
purchasers came together under the disclosure project, and built a set of 10
criteria for creating measures that matter for patients and consumers, for the
purposes that you see listed, whether that is decision making about choosing a
doctor or their care, or enabling performance-based payment.

So the first three are, first of all, get their voices to the table,
consumers and purchasers. Use direct feedback, patient reported information, to
measure performance, and build a comprehensive dashboard that gives you, like
what David described, a better, broader sense of what is happening, that
multiple people can imagine themselves fitting into. And I will show you an
example of that in a second.

Focus on an area where there is great potential to improve, and ensure that
measures generate valuable information. As I was looking at the websites, I
again had some expert guidance from Robert Krughoff who said, hey, go to the
United Healthcare website, because they actually do have cost and quality data.
And I did, but every primary care physician that I looked at had the exact same
two star rating, every single one of them. They were all high quality, low
cost, according to this site, and that does not help me. You might as well have
zero information.

And then, finally constructing measures that assess what happened at the end
of the day. Not just were these measures where the clinician recommended care
that was in concert with guidelines, but also at the end of the day, was the
patient able to follow those guidelines, and did they understand them. And
Joyce talked a lot about needing the individual level. We are in sort of this
really new area of shared accountability. And so, I think that we are going to
have to have some information about individual members of the team, and how
they are performing and how they are faring, in order to make this work.

I forgot my example of a patient-centered measure dashboard, which is
organized by actually the triple aim. But you can see it is clinical outcomes
and patient reported. And a lot of the domains that David and Joyce really
talked about, coordination, some structural measures like use of technology,
total costs, but something that gives them more coherent picture for pictures
to understand. Thank you.

MS. QUINCY: I realized later that this title is very ambiguous. But it was
intended to mean consumers trying to select a health plan or the decision
making around health plans. Actually, I do not think I need to say this again.
But I am going to talk about the decision making around selecting a health
plan, because I think it might be a gateway to getting to that provider
quality.

This is actually a big part of the studies I put up earlier, was how can we
help consumers with their health plan selection process. And you may recall
that a huge finding from those studies is that they really struggle with trying
to figure out the patient’s cost sharing burden. And there are some areas, for
example, of aggregate measures, some of which do not exist now, that could
really help consumers. So I hope this is a little bit hopeful. It is a we can
do this type of message, even though it is not in the equality realm.

The first one is something called coverage examples. This is required by the
Affordable Care Act. It is something that consumers have not seen before. I
have an example on the next slide, and for specific medical scenarios it shows,
for example, having a baby, how much the plan would pay, how much the patient
has to pay and also what the starting cost of that care is. It was absolutely
remarkable about how this new way of presenting health plan information
completely changed patients’ perception of what it is they were buying, and I
will talk about that more when I have got the coverage example out.

Another thing required by the Affordable Care Act is that health plans in
the individual and small group markets are going to be arrayed into tiers,
based on an overall measure of coverage generosity called actuarial value. We
also tested this and this also is a new thing, and it proved extremely helpful
for consumers as a way of navigating their choices. It was not the only measure
they relied upon, but it provided that evaluable measure that Joyce talked
about earlier, which means you can rank things. And so, it provided a way to
navigate choices. They could say I am going to look at all of the platinum,
which is the plans with the most coverage, or I am going to start with bronze
or whichever strategy they adopted.

Not to mention Robert Krughoff’s name yet again, but another thing that you
will see hopefully in his demo is the fact that what I would call a corrected
patient out of pocket maximum. There is a wonderful study called Coverage When
It Counts. I think I have the title slightly wrong, but it looked at health
plans, and it looked at a breast cancer scenario, and it actually went to the
200 page policy, and it said exactly what would this plan pay for this medical
scenario. And for plans that appeared on their face to be very similar, they
had similar actuarial values as it was measured, and similar out of pocket
maximums. The cost to the patient at the end of the day was actually different
by thousands of dollars.

You cannot rely simply on the reported out of pocket maximum. You have to
take into account all of the exceptions and so on, an exercise that was
difficult for these PhD researchers, forget the average consumer. But tools
that are available that correct that out of pocket maximum, or rules that
require that it be a hard and fast out of pocket maximum, these are all things
that help with that difficult determination of what the patient’s out of pocket
costs are going to be.

I talked to you before about the difficulty of figuring out on a streamline
centralized basis, which providers are connected with which health plan, so
that is that next bullet. Where do go for unbiased expert help. Consumers have
absolutely no idea. We know many, many of them, no matter despite your best
efforts to improve the tools available to them, are still going to need
personalized assistance. We need to help them figure out where that is. I think
that is a real stretch for them now, to figure out how to go and find help.

Things are going to change, as you know, we hope in 2014. Some of the
cost-sharing traps that exists now are going to go away using the new rules.
But there is still going to be pressure to keep premiums down, at least we hope
that will be true, and we can imagine that the ways that insurers adjust their
products in order to keep the premium price attractive might change a little
bit. And two areas that I think are going to loom in much greater importance
compared to now are measures of network adequacy, because ability to vary
cost-sharing does not go away completely. But it comes more constrained, so we
might expect plans to turn their provider networks and perhaps limit them more.
So we are going to have to provide patients with reliably timely measures of
network adequacy.

Another one that we should expect to see is greater use of inside limits.
This is a benefit design feature whereby they limit the number of visits or the
number of prescriptions or something like that. You are no longer going to be
allowed to have dollar denominated limits, so we could expect this to be
substituted in. We have got to make sure that that information is clear and
readily available to consumers.

This is an example, it is actually slightly out of date. What was put into
final rule looks a little bit different than this, but this was the slide that
I had. This is the coverage example, this is going to be on the new health
insurance disclosure form. The final rule only included, for now, two medical
scenarios, having a baby and managing diabetes. They are going to hopefully
return back to the treating breast cancer example. When we provided this new
way of showing health plan information to consumers in our studies, compared to
traditional health plan information, which I would characterize as showing the
deductible, showing the coinsurance, little discreet pieces of information, the
reaction was absolutely phenomenal. And I am going to tell you why because I am
hoping it will convince you that you have to do consumer testing about
everything.

First of all, it shows you how much medical care costs. Having a baby costs
$10,000. People do not understand how expensive medical treatment is, and that
is why they can buy policies today that might have a 20 or $30,000 annual
benefit limit, and think they have got good coverage. That sounds like a lot of
money to most consumers, and they simply have no idea what it costs, for
example, to contract breast cancer.

It shows what you pay, but it shows the bottom line. In the case of this
plan, it does not cover having a baby, you would have to pay $10,000. So
actually, let me go to the breast cancer example. You would have to pay $3,200.
A lot of that is associated with this deductible. Well, consumers, they cannot
generate that rolled up number on their own, so this was very helpful. But here
was the biggest surprise of all. It also shows, mathematically not too
difficult to figure out, what the plan pays. And think about your traditional
health insurance disclosure. It is all about what you have to pay. You, the
consumer, has to pay the premium. You, the consumer, has to pay the deducible,
you have to pay the copay. And you are not reminded that you are getting
something for all of these payments. You are getting insurance coverage.

People in this room probably readily understand this. But the consumers in
our studies, whereas before they would have said this exact plan, they would
not consider buying it. When they saw how much the plan was going to pay for a
breast cancer scenario, they said, oh, my gosh, this is actually valuable. That
deductible that I really detested before, suddenly that does not look so bad,
because I am getting all this coverage. This is the same information they were
seeing in a different format, but it was presented a different way. The effect
was profound, it actually changed people’s decisions about whether or not to
insure themselves in our hypothetical scenario.

Joyce already hinted at this on what she said. It is important to keep in
mind that some measures, they are very important to measure, but they work
indirectly. And a wonderful example is the fact that there are state insurance
commissioners out there, measuring health plan financial solvency. This is
critically important. Consumers are never going to look at it, even though it
is publically available. But somebody had better be measuring it because we
want solvent health plans out there. And they do a great job, nobody needs to
worry about the solvency in their health plans.

Other things that only a few consumers might care about, but it might be
really important still to measure, are rates of claim denials or denials that
are overturned on appeal. We talked before about revealing those provider
contract terms. Do these provider contracts actually go out and pay for
quality, which is what Joyce wants. And I want to keep Joyce happy, I don’t
know about you guys.

Customer service, again, apologies to our AHIT member, but a survey just
came out. Customer service on the part of health plans is rated very, very low.
And consumers care about that, it makes all the difference to them. We need to
develop measures of customer service, so that plans will perform better, even
if not every single consumer goes out and looks at that measure.

This is the key message. Everyone so far has talked about the patient voice,
we need to listen to what the patients have to say. That means you have got to
go out and do consumer testing. I had a little video, but I think we should
skip it in the interest of time, that showed five minutes of snippets from our
consumer testing. It is really compelling when you hear things in their own
voice. It is much better than me saying it.

I thought of a couple of other things that I wanted to say about deficits
and gaps as we talk to everyone else. I think that we need a way to measure the
gains in consumer welfare. This is a difficult exercise, although the IOM has
attempted this in the distant past. When we talk about the cost of data
collection, when we talk about the cost for producing this new health plan in
summary form, or when we talk about the cost of spending more on a great health
plan comparison tool, we must weigh that against the gains in consumer welfare,
and we do not have good tools to do that right now. We do not have a way to
quantify the gains to consumers and weigh them against those costs. And as a
result, it is too easy to dismiss those consumer gains. This is not easy, but
it is going to be very important as you debate policy recommendations.

The other thing that is very important, another project that I have going
on, is an exercise to measure consumer’s health insurance literacy, not health
literacy, but health insurance literacy because today’s measures of health
literacy do not actually address all these difficulties that I have been
talking about today. And when we talk about trying to target our messages to
consumers, we need really robust measures that describe these different
dimensions of literacy, so that we can target our communications, so that we
can see how well they are doing. I think that is self-evident. I will not
debate that.

The other thing, I guess I wanted to give you one more example about why we
need to do consumer testing. I bet everyone, before I said we did all this
testing, knew that health insurance was hard. But when you go and do consumer
testing, I am going to give you a single finding. We learned that there are
three discreet things that are difficult about coinsurance, and that is what
you do not know until you do consumer testing. We learned that A) many people
are actually unsure who is paying the displayed percentage on the page.
Particularly if it is zero percent or 100 percent, they are like, well, is the
plan paying that or am I paying that? That is a pretty bad start. Second, even
if they knew how was paying the coinsurance percent, they did not know what it
applied to. It applies to something called the allowed amount, which is
essentially unknowable for consumers. So you actually cannot get to how many
dollars you have to pay.

Third is that consumers, there is actually a lot of them that struggle with
percentages and cannot apply them, and we see this in other walks of life, like
the excellent work that is being done around mortgage disclosures by the
Consumer Financial Production Bureau. We had consumers where we would say,
okay, coinsurance is 50 percent, the allowed amount is $1000, what is your
share? They could not answer that question. And that is where, going back to
measuring, literacy measures, numeracy measures, rolling that up into usable
scores that tell us how to do our work is very important.

Let’s not rely on the data that is out there now, but let’s really start
with what should we be measuring. And I think this is important. Consumers,
let’s not be constrained by the past. I do not think we yet know what consumers
can learn to use if we do all of our homework, all of our consumer testing and
figure it out. I just put up trans fats, but consumers have actually learned to
use a lot of what are really sophisticated measures. But we have figured out
how to present them simply, like your miles per gallon cars. They go out, they
use it as they were supposed to use it, and it all works. There may be
undiscovered measures out there that really are going to work, really are going
to do the job. I think that is it. If you are only going to read one study of
all the ones I published about the consumer testing, that is the one you should
read.

DR. MIDDLETON: That was terrific. Thank you very much.

MR. SUTTON: It is me again. I am going to finish up with a presentation that
only a technocrat would appreciate. Two years ago, my workgroup, the Metrics
and Measurement workgroup, was tasked with trying to measure all the work that
we are doing in Rochester. Just a couple of things, before I get started, one
is that we spend so much time in Rochester arguing with each other about
whether we are doing the right thing and are we measuring the right thing, and
are we going the right directions. It is certainly a joy to get out of
Rochester and present this data, and have some people go wow, I really think
you guys are kind of on the right track. Although, it is quite intimidating
being around these three ladies that have done some excellent analysis.

Also, just to let you know, we have mentioned a lot about high blood
pressure. What we are doing in Rochester is trying to build a roadmap on who we
can be a healthier community and how the community itself can kind of pull
itself up by the bootstraps and fix some of their own problems. Hypertension
just happens to be the first car going down this roadmap, and we think and hope
this is going to go on forever. Once we get past hypertension, we will move on
to whatever is next, whatever the community tells us is next.

If any of you have done any work looking at Frank Konabee’s work on lead and
lag measures, our lag measures are lag measures because we will not really know
if we have gotten there until they are there. And if we did not get there, you
cannot go back in time. Our lag measures are those incidents of the
complications of high blood pressure, of course being heart attack, stroke,
heart failure and end stage kidney disease.

We have gathered this data from what is called a New York States Sparks
data, which is the statewide planning and research cooperative. This is
mandated data that all hospitals have to report to New York State Department of
Health. For one of the first times, this is public data. We have taken it back,
looked at Sparks data for the four hospitals in Monroe County. We have broken
it down by ethnicity, socio economic status, and have targeted these
conditions. So we have been able to take some public data and bring it back for
some use for the community.

We have also engaged the health plans into giving us the per patient, per
cost. The patient with high blood pressure, they are still working on that
data, we do not have it yet. And then, what I am going to spend some time on
today is what we call our community-wide community database of blood pressure
values, currently at 64 percent. Our goal is 85 percent. We sort of stole this
from Kaiser Permanente in California, that has, through a project, reached 80
percent, their goal in the blood pressure values. That is considered best in
class. We want to go above that as a community.

That final measure, the percentage of hypertension in our community that are
at goal in their blood pressure values is somewhat of a rallying point. It is
that one measure that the entire community can look at and say, this is our
goal. I am looking at our lead measures, which are more kind of on the ground
work to see if we are going to get to our lag measures. Some of these are
broken down by workgroup, by team. These are measures where individual citizens
can say, what is my piece to the puzzle, what can I do to affect that larger
goal. Things such as knowing what your blood pressure value is, that the
percentage of the community, they have a plan with their doctor. The percentage
of their community that get their blood pressure checked regularly, and also
the percentage of physicians in our community, they are engaged in some type of
quality improvement practice.

This is actually our clinical registry. We are trying to stay away from the
word registry. We think that, in some cases, has some bad connotations where
people think they are being tracked. This is de-identified data. The way this
is worked is it is volunteer.

The three large health systems in our community, as well as many independent
community health centers, and many private physician practices that have EHRs
have been uploading this data to the collaborative. It is de-identified, what
they are giving us is the patients with high blood pressure, what their last
blood pressure value was and the date of that, when they were last seen in the
practice, and the number of people coming into their practice that are new to
the practice and have a diagnosis of high blood pressure.

This slide is a little bit old. We had this database now for a year. Our
first sweep, we ended up with 56,000 patients into the database. Our last
sweep, we are sweeping it every six months, we now have 72,000 into it. We have
had more practices engage and want to upload this data. This represents about
40 percent of the community. By the end of this year, 85 percent of the
community is going to be on EHR. We think we are well on our way by 2013 to
have essentially practically every blood pressure value in the Rochester area
tracked through this database.

Race and ethnicity of the database is pretty close. We are off a little bit
by our African-American population and Latino population. That is actually the
fault of Rochester General, my employer. We are currently on paper charts. In
2012, we are making the transition into our EHR. We wanted to be a part of the
database, so we participated by grabbing all this data out of the paper chart
manually, and then reporting it to the database. So we had to sample our
patients, we could not do every patient we had in our system. We have a very
large system.

And Rochester General serves predominantly the urban part of Rochester, so
we have a high African-American, high Latino population. We have given, I
think, 10,000 blood pressure values to the database. By the end of this year,
that will probably be closer to 30,000, and then the ethnicity of this database
will more closely match the ethnicity of the entire population in our
community.

Socioeconomic status is actually very close to our community. And also, if
you look at a comparison of Monroe County to the comparison of the database
itself, you can see it is pretty darn close in terms of patients that we are
looking at in the suburbs, patients in urban population and patients with a
crossover. We think we are really well on our way in terms of developing this
community-wide registry, that is working at all the aspects of blood pressure
in our community. And then, presenting it back to the community in terms of the
percentage of patients that are to goal, so they can have this rallying point.
Currently, it is 63 percent. Our last sweep was 64 percent of patients to goal.

I know this is very painful, but I am going to do it anyway. However you can
do this, either look up on the screen or look on your individual sheets,
because we have done a lot of work on this. And this schematic is really a
representation of how we are measuring the work of the collaborative in our
community. As I talk you through it, and I can do this really in a minute, I
guarantee it, I am hoping it will make sense to you. If you start up on the
top, we have about a half million residents in our community above 18 years of
age. We know based on national data, if you look just below that, we know about
172,000 have high blood pressure.

If you go over to the left side, we can go through this really fast because
this is stuff we do not know. These are patients in our community that are not
seeing a doctor, or patients in our community that are seeing a doctor, but do
not have an EHR. Or perhaps they have an EHR, and they have not engaged yet to
be into our registry. So we have a lot of not available, not available as far
as data is concerned.

At the bottom, these are some of our workgroups. The community engagement
workgroup, they have been tasked with engaging these residents into
understanding the importance of going and seeing a doctor, and getting into a
practice. Also, our best practice workgroup is working to engage practices that
do have EHRs, to upload that deidentified data into our community database. And
as you can see, our first sweep, we had 56,000, now we are up to 72,000, so
that is starting to work.

If you look over onto the right side, our most recent sweep shows 72,000
patients that we do have good blood pressure data on. And again, this data is
their last blood pressure value, and the date of that value, the last time they
were seen and how many patients in that practice are new patient to practice
with a diagnosis of high blood pressure. And if you see the column kind of in
the middle, which are new patients first seen in this practice, we think that
is very important, because as we see practices reporting this data, showing new
patients into their practice, this is a sign that we are moving patients off
from the left and into the right.

For example, our first sweep showed 460 new patients to practice in our
registry. The second time we swept it, there were 924 patients into it. And if
you continue down the right-hand side, you could see patients that are seen
within the last 13 months. We chose 13 months because a perfectly
well-controlled hypertensive patient would be told to come back in a year in
many circumstances. And if we measured a year, we were afraid that we would
sometimes miss these good controlled patients, so we have a one month window,
and went 13 months. And again, you can see the percentage to goal and the
percentage not to goal.

If you sweep along the bottom left there, you can look at all the workgroups
that we have put together in our community, about 118 people. On the left-hand
side, these are very diverse workgroups that have representation of the ethnic
groups, representations of the community, non-profit sector, the business
sector, that are working to engage the community to get into a practice, to
make a treatment plan with their doctor, and to take better control of their
high blood pressure.

As you move more to the right-hand side, you see workgroups that have
physicians in them, data analysts experts into them, that are engaging
practices into quality improvement programs, these are peer counselors,
physicians that have taken academic detailing and going into practices and
saying, hey, through our registry, we have noticed that your percentage of
patients to goal is not quite as good as someone else’s. Is there something we
could do to help you. And that we have a community-wide effort, sweeping all
the way across, over to the far right-hand side showing the percentage of
patients in our community that are to go on high blood pressure.

We think that if we can move that from our baseline of 63 percent up to 85
percent, that incrementally each year, we will see a five percent decrease in
the incidence of the complications of high blood pressure, which Jake mentioned
early on, is probably worth about $3 billion over a five-year period of time.
Thank you.

DR. MIDDLETON: Terrific. Why don’t we open it for discussion and questions?

DR. HORNBROOK: I have several different comments, but this last chart was
fascinating to me. I am thinking how this could be replicated as the basis for
beginning community action. And in theory in Massachusetts, if we have
everybody insured, or pretty much most people insured, if we have an all payer
claims database that includes all commercial plans, Medicaid and Medicare data,
we have the information needed to essentially recreate this document,
everything except the key, which is the workgroups to address the issues. Is
collecting the data intended to stimulate the workgroups?

MR. SUTTON: No. You are right. I think the building blocks are kind of there
through this in any community. Perhaps Rochester is the first community to kind
of get there and try it and see what happens with it. The data is there for
data purposes only. It is to let the community know where we are at, and where
we think we should be. The workgroups are to try to move it from the left to
the right. It is the way it should be. It is probably the way we should be as a
nation. But it is work, and I think some of the other ladies were very good in
emphasizing the trust factor.

This data is coming from our community, and community members look at it. We
are hoping that they trust that we do not have any secondary intentions in
giving this data. And who knows, maybe we are wrong in how we are gathering
this data. But if we are wrong, we are wrong as a community. This is a
community-wide effort to kind of put a mirror in our own face and see where we
are at. And we are hoping that there is a level of trust that comes with it. I
am leery of the fact, fi this same data was coming from outside of the
community or from an insurer or from an outside agency, if the trust factor
might not be there.

DR. HORNBROOK: If I may just continue on with a question for everybody. It
was clear to me that the responsibility for the plans and insurers to collect
this patient-focused information or information from patients is imperative if
we want to provide information back so individuals can make better decisions
about their care.

One observation and one question, how much of the information really is
clinical and how much of it is not. And for the piece that is not clinical,
what incentives can we provide to the insurers and the plans to collect that
information. And what incentives can we provide to them to focus on the need
for inputs from their patients to direct their decisions?

MS. BECHTEL: I will take a little first stab at that. I think I am having
trouble with the clinical versus not clinical distinction, and where that line
might or might not be drawn. And the reason is that, I think for a long time,
people will say that there are domains of the patient experience survey, for
example, that are not clinical. And I disagree, because when we have talked to
patients, they will say things like, well, if I cannot understand your
treatment recommendations, or if you do not understand me as a whole person, I
cannot get to better health outcomes, because the recommendations you give me I
do not understand or I do not have any hope of following, because you just said
that you need to join a gym and I cannot afford it, or you need to take seven
days of bed rest and I have five kids.

I think there are two major myths that we have to move through, if we are
going to shift our culture to be more patient-centered. And one is, that there
is not a link between patient feedback and the clinical outcome, which there
is, and there is great literature that does support that. And also, that we
have to use that information to improve, and that gets to the second myth,
which is we, I think, assume that oftentimes clinicians know what is important
to patients and what they want. And yet, there is a pretty great body of
evidence that says that that is not always the case. You have to ask.

The other thing that I would say is I would add to your list, you talked
about plans and employers collecting patient-reported data. And I think
providers have to be first on that list, because number one, they have to
understand what is happening with their patients, and number two, they have to
use it to improve. So I think it is multi-stakeholder, multi-faceted, and I
think we have to better believe in that relationship between feedback on things
that do not appear to be clinical and the relationship to the clinical.

MS. DUBOW: I don’t want to repeat what Christine says because I agree with
her, but I want to talk about your observation or your question about
incentives. I think, for starters, the cost of data collection and reporting
should be considered a cost of doing business. We are past the era of no
accountability. And I think purchasers and regulators have every right to be
requiring data collection and public reporting.

With respect to what is right, what are the right measures to collect, what
is the right information to require, obviously it is a combination. You have
heard us give you strings of ideas. I will not repeat them. But I think that as
far as the source of what the right measures are, not only should we determine
from patients the information they need, I think Lynn made a really important
point during her second presentation. And that is the capacity for the public
to learn when we provide information that will help them. They can be educated.
These are things that can be taught. You can tell somebody with information why
something of clinical nature is of importance to them, in terms of
self-management, of taking care of themselves.

I think we have not done enough work in helping people understand why some
of the clinical stuff is really very important and why they ought to be
thinking about it. But not only do we need to ask patients, but we need to help
and educate. My bottom line is, this is the cost of doing business in the
21st century. This is what you have to do.

MS. QUINCY: To be totally responsive to your question, I think if you cannot
do this with 30 measures, but if there are some prominent measures that is
going to be in the first screen full of information that consumers see when
they shop for health plans, or there is an annual report card, anything that
becomes a prominent measure is, I think, incentive in and of itself to perform
well along that dimension.

MR. SUTTON: I don’t want to sound like a broken record, I think what
motivates everybody to be a part of this is they are putting their piece into
the puzzle. Early on in our collaborative, we looked around and said, wait a
minute. We do not have insurers at the table. Why aren’t they here, as equal
partners, looking at what we are going to do? Perhaps today, insurers should
have been here.

I think if you give people an opportunity to participate, they will want to
play their part, they will want to put a piece into the puzzle. And I think
that is what motivates them, what could motivate them to be a piece of this.

DR. TANG: I have a question to try to reconcile. Fortunately, we have the
same group basically reconciled the first panel with the second. In the bad
news-good news for the first panel, the bad news is we did not have any
measures that mattered to consumers. The good news is they did not have time to
go consider those anyway, because coverage came first, etcetera.

So now you gave us some attributes of a good measure that would help
consumers. Is it the case that if we build it, they will come, or will all the
other stuff that you talked about in the first panel still get in the way? Or
what are the preconditions for having measures that would be used and consumers
would find useful?

MS. BECHTEL: For which purpose?

DR. TANG: Choosing a health care team that would improve my health, me and
my family.

MS. BECHTEL: I am not sure that we can yet draw the conclusion that
consumers do not use quality measures for that purpose broadly. We certainly do
have evidence that is pointing in that direction, but that data that we have is
really on the hospital side and it is really for Medicare hospitals. And I
think the decision making around inpatient and outpatient care is very
different.

But the fact that, when we are talking about clinical quality measures and I
cannot find any on my local physicians in the nation’s capital, it is
disconcerting. But I think I would come back to what Joyce talked about, which
is that the cognitive burden is really high. Even leaving my case aside, with
having to shift through multiple websites, everybody has different
functionalities. The data sources are completely different, but fundamentally,
the data I am looking for is not even there.

I think we have to get to it, even if we were to use measures that matter,
it has to be, like Joyce said, five or six kind of factors, at most, and
understanding and working with consumers, as Lynn talked about, very early to
say what is it that you care about the most, how do we get drilled down
information in the areas that matter, and how do we make this less complex to
weigh all of these different factors. They do need to know do they accept my
insurance, I do need to understand their specialty, their name and location. So
if we are talking about five variables, and those are three, we are in a little
bit of trouble. We had better make the quality information and the cost
information really easy to find and meaningful. It has to show variation.

I was so struck by the fact that I can just go on a website, and the one I
could find has two stars for everybody. I was like, really, okay.

MS. QUINCY: That question is very important because you do not want to
devote resources to something that people do not use. But a well-designed
consumer test can provide the answer, and it probably would not actually be a
focus group approach. It would be an approach that involves usability testing,
trying to simulate the real world, which I will not go into detail. But there
is a well-designed test that would provide the answer to your question.

MS. DUBOW: We haven’t talked about patient engagement from other
perspectives today at all. We are focusing really on using measurement. I think
really one of the key aspects of patient engagement is engagement at the
behavior change level, in terms of one’s own care. I think we heard about
educating physicians around motivational skills, communication, helping at the
training level, clinicians to better communicate with physicians, which in turn
will produce more activated, more engaged patients. And I think when these two
engagement of patients from the behavior side, in terms of taking care of one’s
own health, and the continuing efforts to provide public information for people
to use in making decisions, come together, we will have synergy there and we
will be making this thing move forward.

I talked a lot about having to target stuff. I do not think public reporting
is the primary way to engage patients. I think it is necessary, but it is not
sufficient. And it is not what is going to change the system. Clinical behavior
is going to change health care and improve it. But we do need to get people
engaged in their care, and that has to happen at the individual, personal
level.

MR. FLAITZ: I think one of the things that is interesting, and Jim did
mention, and maybe this gets to your question, Paul, when we look at the high
blood pressure initiative, I think one of the fundamental things we ask
ourselves, from a patient perspective, was it is it important for people to
know your numbers. We hear a lot about that, right, know your numbers, know
your numbers. For many people, that can be confusing. My blood pressure is 126
over 78, where am I? So we have talked about taking that, rather than saying
know your numbers and what those mean, know your zone. Are you red, are you
yellow or are you green? It is important to know that, and to really simplify
it.

One of the things that we found, both within our work in the collaborative,
and in our work at Paychex, we will constantly say, make it simple, make it
simple. And we find out it is not nearly simple enough. To engage the folks
that we need to engage, I am not as worried about the folks that are already
engaged. We can get them the information, they can carry on their own. And one
of the things, just to talk specifically about Paychex, that we have found, in
an open enrollment training thing that we have done over the last two years
with a company called Jelly Vision, is that engaging highly interactive, sort
of video training modules make all the difference in the world. We put out this
easy, simple forms to use, in terms of understanding what your cost share may
be, one page. At Paychex, you have got to take a math test and pass it to work
here. The question was not math skills, it was just the unattractiveness and
how uninteresting that one page was, to go through that determination. Doing it
in a whole different manner has made all the difference.

DR. MIDDLETON: If there are no other questions, let me say on behalf of the
whole committee and the subcommittee, this has been terrific. Thank you for
sharing your insights, your expertise and your generosity with your time. We
would love to have the opportunity to pose follow-up questions remotely, if the
case may be. Thank you very much.

(Whereupon, a luncheon recess was taken at 12:30 p.m.)

A F T E R N O O N S E S S I O N

DR. TANG: Thank you for reconvening. We have still two more very interesting
panels coming up to round out our discussion about measures that matter and
beyond measures, really. This next group is going to talk about functional
status and self-measurement measures. These truly are concepts that matter to
consumers and patients, so anxious to hear from our two guests.

Agenda Item: Use of Functional Status and
Self-Management Measures

MS. SMITH: Good afternoon. I am Heather Smith, Program Director for Quality
for the American Physical Therapy Association, and I am also a physical
therapist. Thank you for inviting the association to participate in today’s
hearing.

I just want to start by giving you a little context about the American
Physical Therapy Association, which is a professional organization representing
the interests of more than 80,000 physical therapists, physical therapy
assistants, and students of physical therapy. The association’s goal is to
foster advancements in physical therapy practice, research, and education and
to further the profession’s role in the prevention, diagnosis, and treatment of
movement dysfunctions and the enhancement of the physical health and functional
abilities of members of the public.

Today I am here to discuss functional mobility measures that matter to
consumers. I think we can all appreciate quality measurement of healthcare is
complex. It has been said that quality is in the eye of the beholder. As you
can see here depicted, an ideal measure would ensure patient- and
family-centered care for the consumer while providing them with outcome
information. It would be of diagnostic use to the clinician, dieting and
informing treatment interventions. Lastly, it would allow the payer to capture
outcomes across the continuum of care, facilitating a true pay-for-performance
health system.

In our role as physical therapists we perform evidence-based screening,
evaluations, and assessments for musculoskeletal, neuromuscular,
cardiovascular, and integumentary conditions and provide interventions that
focus on function and mobility to improve an individual’s quality of life.

We have extensive experience with measures of function and mobility,
including measures that are clinical-administered as well as those that are
patient- or self-administered. The association believes strongly that
identifying and utilizing meaningful patient-centered functional mobility
measures will better demonstrate the impact physical therapists have on
functional outcomes across the continuum of care.

Physical therapists are an essential member of the healthcare team who
provide evaluation and treatment for individuals who have a variety of
disabilities and conditions, as seen here, across the entire lifespan from
birth through end of life. Physical therapists treat individuals in a number of
different practice settings, including hospitals, skilled nursing facilities,
home health agencies, rehabilitation agencies, and private practice outpatient
clinics. This variation in conditions and settings creates and challenge in
identifying the ideal measure of functional mobility.

The differences in the post-acute and outpatient settings include the
location of care as well as variation in conditions, severity of illness, and
the functional limitations of patients. In general, patients in the post-acute
setting have a higher severity of illness with greater restrictions in their
functional mobility.

Currently, functional mobility measures are included in Medicare required
data collection tools in the post-acute care settings. As you can see in the
middle of this slide, long-term care hospitals will be using the Continuity
Assessment Record and Evaluation tool, or the Care tool. Inpatient
rehabilitation facilities use the Inpatient Rehabilitation Facility-Patient
Assessment Instrument. Skilled nursing facilities use the Minimum Data Set.
Home health agencies use the Outcome and Assessment Information Set, or OASIS.
In addition, there are several measures of functional mobility included in the
outpatient setting in the Physician Quality Reporting System.

The post-acute care data collection tools are not uniform in their
functional mobility measure items or their scoring systems, as seen here on
slides nine and ten, making comparison across the continuum difficult. The
sensitivity of some of these instruments to detect change in the patient’s
function is also a concern.

In examining measures of functional mobility, there are categories of
measures: global measures, impairment or condition measures, and body region
measures. I am going to focus on global measures. Global measures are general
measures of functional mobility that can be used for a variety of patient
populations.

The benefit of global measures is that they are applicable to a large number
of patient conditions and may be utilized in both the post-acute and outpatient
settings. There are three self-report measures that I will discuss today that
fit this criteria. They are: the Outpatient Physical Therapy Improvement in
Movement Assessment Log, or OPTIMAL; the Activity Measure for Post Acute Care,
or the AM-PAC; and Focus on Therapeutic Outcomes, or FOTO. All three of these
measures have been recognized in the Medicare Benefit Manual.

Beginning with OPTIMAL, this is developed and tested in the outpatient
setting. It is administered via paper. It includes a global rating scale and
has two domains: difficulty performing activity and confidence performing
activity. As you can see here, these are examples of the mobility items
included in OPTIMAL. These items are rated on a five-point scale. This is a
snapshot of the global rating scale as the patient would see it as it is
included in the OPTIMAL assessment tool.

The second measure is the Activity Measure for Post-Acute Care, or the
AM-PAC. It is appropriate for a variety of post-acute care settings and has
been tested in the outpatient setting. It utilizes Computer Adaptive Testing
with item response theory. It is NQF endorsed and uses risk adjustment. It also
uses the International Classification of Functioning definition of activity
limitation. It has three domains: a mobility domain, an

activities of daily living domain, and a life skills or applied cognitive
skills domain.

For those of you unfamiliar with Computer Adaptive Testing, this uses a
computer algorithm to pre-select the items that will be administered to a
specific patient based on responses to previous items. It has several
advantages. It reduces the test burden while increasing the test precision
because test items are selected to match the patient’s functional ability
level. Patients are not required to respond to irrelevant test items. It takes
less time to complete the domain.

Again, I have included examples of the mobility items for the AM-PAC. These
items are rated on a four-point scale. You will note that the mobility items
have increased detail compared to OPTIMAL, and this level of detail is
necessary for the Computer Adaptive Testing.

I thought this was a great picture, so I included it in the slide deck. It
is an excerpt from a patient feedback report generated by Phillips Lifeline,
which uses the AM-PAC. Here the patients’ responses to the mobility items are
displayed using a colorful dashboard to provide the patient with knowledge of
their functional mobility. I think all of us can relate to how easily this is
interpreted.

Lastly, we have focus on therapeutic outcomes, or FOTO. This is appropriate
for the outpatient setting, but it is also now being used in the skilled
nursing facility setting. Again, this tool uses Computer Adaptive Testing and
is NQF endorsed. It does utilize risk adjustment, and FOTO is also a registry
and provides national benchmarking for participants. The outcome domains here
are focused on conditions, and they include orthopedic, neurologic, and several
other specialties. These are examples of mobility items included in FOTO. These
items are rated on a five-point scale.

This is another excerpt from a patient feedback report, this time generated
by FOTO, using the patient’s response to provide them with knowledge of their
functional mobility. Obviously, this is in a graphic fashion. It does provide
information not only about where the patient is functionally currently, but
also provides an estimate of the number of visits to achieve a predicted
improvement in their functional score.

Barriers to implementation that are practice-oriented include resistance to
change by practitioners, which is often encountered when introducing new tools
with increased time needed to administer and interpret results. Certainly,
implementation barriers exist in employing tools that require computer
technology, specifically cost and access, as well as issues with regard to
proprietary methodologies.

Barriers to implementation that are administrative in nature include
regulatory barriers due to the use of multiple tools in the post-acute care
settings, as I discussed. Attribution issues for measures are problematic, as
metrics in the post-acute care setting are attributed at the facility level
versus clinician attribution in the outpatient setting. Lastly, risk adjustment
methodologies across multiple care settings would be a challenge.

Strategies that may drive the use and adoption of these functional mobility
measures include consideration of the use of a single functional mobility
measure across the post-acute and outpatient settings to better evaluate care
across the continuum. Funding and incentives for computer technology
implementation may encourage adoption of computerized measures. Lastly,
educating clinicians about the benefits of using global functional mobility
measures is an essential component of gaining measurement support.

In conclusion, the ideal for quality measurement continues to be an
evolution in process. I think we have heard this morning about the need to
really truly understand what is useful to the consumer. I think that there are
several viable self-reported global functional mobility measures that exist for
consideration in the post-acute and outpatient settings. There certainly are
barriers to implementation, but I believe there are strategies that will help
us to overcome these barriers. Lastly, we recognize that there is a need for
additional research in this important measurement area. Thank you.

DR. HORNBROOK: Can you tell us which of the instruments are public domain?

MS. SMITH: OPTIMAL is in the public domain. The rest of the instruments
really are proprietary.

DR. HORNBROOK: So they charge a fee per use.

MS. SMITH: That is correct. Phillips Lifeline — and I took that snapshot,
and I just want to make this clear — has that in the public domain for use
with patients, which I personally think is a pretty phenomenal feat, to allow
patients to assess where they are at. But that is the only place that I know of
that it exists in the public domain currently.

DR. HORNBROOK: How do you handle pain scales? None of these have pain scales
in them, so pain is measured separately?

MS. SMITH: Pain would be measured separately. In fact, it is included in
measures in the outpatient setting as well as included in all of the tools that
I discussed in the post-acute care settings. Yes, in this instance it would be
separate.

DR. FITZMAURICE: I want to follow up on Mark’s perceptive question about the
cost for these. Does that mean that when a nursing home or a home health agency
uses MDS or OASIS, they have to pay for each questionnaire that they fill out
and send in?

MS. SMITH: That is a good question. I actually do not know the answer to
that. I would defer to CMS to answer that, but I believe because it is
required, there is no cost to them. That is part of the issue with the barriers
of these tools, is that although excellent may exist out there with
methodologies that would allow us to assess a patient as they change over time,
they just are not necessarily easily implemented right now because of the
issues I discussed.

DR. HOUGH: Good afternoon, Chairman Middleton and Tang and NCVHS members and
guests. My name is John Hough. I serve as a statistician at the US National
Center for Health Statistics in Hyattsville, Maryland. It is an honor for me to
present these remarks today during your hearing. I commend this quality
subcommittee for addressing the timely topics.

During my remarks I will focus on the International Classification of
Functioning, Disability and Health, the ICF, and with great thanks to Heather
for giving me this nice segued lead-in. The NCVHS has a rich history of
involvement with the ICF, therefore my emphasis today on the ICF in meaningful
measures of patient outcomes simply continues a long line of investigation by
the NCVHS into the ICF.

For example, your subcommittee on populations prepared a report in 2001
entitled “Classifying and Reporting Functional Status,” which called
for adoption and broader implementation of the then brand new ICF. In 2006 and
2007 your full committee received many sets of recommendations stemming from
the Federal Consolidated Health Informatics Initiative, including the report
from the CHI Disability and Functioning Working Group. Using that working
group’s report, your full committee recommended to the secretary that the ICF
be considered as a CHI-endorsed vocabulary in tandem with SNOMED CT for
transmitting data in the functioning and disability domains.

Finally, a few years ago in June 2009 our colleague Dr. Lisa Iezzoni from
the Institute for Health Policy at Massachusetts General Hospital addressed
your full committee with remarks entitled “Are the Stars Aligning for ICF
in the United States?” These are just a few examples of the active work
already invested by NCVHS in topics related to the ICF.

This slide shows the objectives. I will introduce the ICF with a few
schematic diagrams. Then I will focus on two functional assessment instruments
that have been developed explicitly on ICF concepts and codes. The first is the
trademark product called the PAR-PRO, in which “PAR” stands
participation and “PRO” stands for the familiar patient reported
outcomes. The second is the trademarked product called the AM-PAC, which I am
also grateful to Heather for introducing. It stands for, as she said, Activity
Measure for Post-Acute Care.

Then I will describe the enormously rich opportunity for ICF-oriented
research in this area using information from the CMS Minimum Data Set for
Nursing Home Residents, Version 3.0. If time permits, or during discussion, I
will address additional opportunities for this kind of ICF-oriented research to
yield more measures that matter.

This quote from the ICF itself is helpful. It reads, “The overall aim
of the ICF classification is to provide a unified and standard language and
framework for the description of health and health-related states.” What
is the ICF? It is a classification of characteristics of health involving
either functional impairments, activity limitations, or participation
restrictions. The World Health Organization has published ICF since 2001.

This slide answers the question how should I think about the ICF. There are
two answers. First, we think of ICF as a conceptual framework. Applying the
conceptual framework alone without using ICF coding is all right and
encouraged, but ICF is also a hierarchical coding structure. The conceptual
framework is substantially enhanced when ICF coding is applied as appropriate.
Therefore, coding is even more encouraged. WHO designed the scheme of ICF
coding to work in tandem with ICD coding. ICF coding can be either simple or
complex, but there are coding guidelines that govern the coding structure.

This slide describes what ICF is not. Pertinently, right up front for
today’s discussion, ICF is not an assessment or measurement instrument. But my
testimony today is that new next-generation functional assessment instruments
can be developed explicitly from the ICF conceptual framework. Several such
assessment instruments have already been developed and validated, and I am
reporting on two today.

ICF is not a tool for disabled peoples’ advocacy. However, with better
classification, comes better and more rigorous analyses of problems associated
with disability. Hence, the ICF conceptual framework enriches and sharpens the
statements and observations of disabled persons’ advocacy organizations.

ICF is not electronically transmittable at this time. This fact is the
single most important limitation to implementing ICF in the US today.

It is not easy to explain because it covers some difficult concepts about
the human condition. Moreover, it is not entirely finished, given that one
domain called personal factors is still under investigation. In fact, in its
2007 report entitled “The Future of Disability in America” the
Institute of Medicine also defined six specific areas of improvement for the
ICF.

This slide shows the four ICF domains in a quadrant diagram: body functions
in the upper left, body structures in the lower left, activities and
participation in the upper right, and environmental factors in the lower right.
We presume that each of these domains is constantly interacting with the person
and his/her health condition, if any. The two functional health instruments I
will describe focus exclusively on the activities and participation domain.

This slide shows the same quadrant diagram with the 30 chapter headings in
all four domains. I will not cover these headings in detail, but I will simply
focus on activities and participation chapter five, entitled
“Self-care.” It is in the upper right quadrant. Self-care refers to
what we might more commonly refer to as activities of daily living such as
bathing and toileting.

This slide shows the familiar ICF interactive model. It expresses the
assumption that all four domains — and one’s own set of personal factors in
the lower right of the diagram — are constantly interacting to create or
intervene against a situation involving disability.

This slide shows two sample ICF codes. I will focus only on the code on top
from activities and participation domain. This is a code about a person’s
ability to walk. The code stem includes all the characters to the left of the
decimal point. The qualifiers are the characters to the right of the decimal
point. This code stem, d4500, refers to walking short distances. We add the
qualifiers to describe the degree of impairment in terms of the balance between
a person’s capacity to engage in an activity and their actual performance of
that activity.

In this case the interpretation of the full A&P code, d4500.2314, would
be walking short distances, moderate difficult in performance with assistance,
the first qualifier digit 2; moderate difficult in capacity without assistance,
the second qualifier 3; mild difficulty in capacity with assistance, the third
digit 1; and complete difficulty in performance without assistance, the fourth
digit, qualifier 4.

This is complicated, but as you can tell, this level of scrutiny is really
detailed for the right reasons. Such coding enables us to interpret a lot about
this individual patient or groups of similar patients. Now let me demonstrate
this kind of coding using a photograph.

This slide shows a teacher at a blackboard writing with a piece of chalk.
This gentleman does not have any hands. Instead, he utilizes prosthetic arms.
This is a good photograph because we can apply ICF codes to describe this
gentleman’s body structural impairment, his prosthetic arms, which the ICF
calls environmental factors, and his ability to manipulate a piece of chalk to
write on a blackboard.

In this slide we have added the body structures and environmental factors
codes. On top, his body structures code would be s7302.413, for structure of
hand; complete impairment, total absence, both sides. His environmental factors
code refers to the prosthetic devices. These would be coded as e1151+4, for
assistive products for personal use in daily living; complete facilitator. This
slide presents our selection for his activities and participation code. It is
d4402.14, for fine hand use: manipulating; mild restriction in performance with
assistance: complete limitation in capacity without assistance.

This summary slide simply presents all three of those ICF codes in sequence.
Even with only three codes, this set comprises an ICF profile. It describes a
person and their situation quite clearly and explicitly.

The next two slides simply present hypothetical ICF profiles to illustrate
how ICF coding works in tandem with ICD coding. We have hypothesized patients
with vision impairment, specifically macular degeneration, and multiple
sclerosis. We have superimposed our ICF code selections for these various
conditions in the schematic diagram of the ICF interactive model previously
shown.

In this slide we look at macular degeneration. The ICD codes from both
ICD-9-CM and ICD-10 are shown at the top. Then in each box representing the
various ICF domains we have inserted the relevant ICF codes. For example, a
person with macular degeneration, a representative activity limitation in the
middle box might involve a severe limitation in watching people or sports, and
there is a code for that, d110. A representative participation restriction on
the far right might involve restricted performance in acquiring, keeping, and
terminating a job as a corollary of the person’s vision impairment, and there
is a doe for that too. It is d845.

Similarly, this slide shows ICF coding for multiple sclerosis in tandem with
ICF codes in the ICF domains, again focusing on activities and participation in
the middle. We might anticipate that a person living with multiple sclerosis
would experience limitations in activities associated with dressing. There is a
code for that, d540, and restrictions in the form of participation on the
right, which ICF calls preparing meals. Luckily, there is a code for that,
d630.

My purpose in showing these examples of ICF coding is to demonstrate the
breadth and flexibility of this classification for emphasizing those
characteristics of health that actually might be most important to patients.

This slide introduces the first of the two newly developed functional
assessment instruments I will describe explicitly based on ICF concepts. This
is the PAR-PRO. “PAR” here stands for participation. PAR-PRO is a
trademarked product developed in the mid 2000s by a team led by Dr. Carl
Granger and Dr. Kenneth Ostir on behalf of the Uniform Data System for Medical
Rehabilitation, the UDSMR.

These investigators used item response theory to design a 20-item instrument
for home and community participation among patients in conventional
rehabilitation settings. They wrote “The development of the PAR-PRO was
guided by the principles, definitions and domains of the ICF and designed for
use in both disabled and non-disabled populations.”

This slide shows a few of the relevant chapter headings from ICF activities
and participation domain on the left in alignment with several of the topics
covered in the PAR-PRO on the right. For example, the ICF chapter heading
“Mobility” is represented on the PAR-PRO by an item referring to the
patient’s degree of the use of public transportation systems. The ICF chapter
heading for “Interpersonal Interactions and relationships” in the
middle is in parallel with an item on the PAR-PRO referring to socializing
inside and outside the home and intimate relationships with significant others.

The PAR-PRO can be used to investigate patients’ participation status in
both rehabilitation and conventional acute care settings, as well as in
community settings among persons in the non-institutionalized population. This
instrument has been very well validated, and it is incorporated in the overall
UDSMR suite of assessment instruments.

This slide simply illustrates the first page of the PAR-PRO paper
instrument, showing such patient-oriented categories as patient satisfaction in
the lower left and home and community participation profile in the upper right.

This slide introduces the second of the two newly developed functional
assessment instruments explicitly based on ICF concepts. This is the Boston
University AM-PAC. It stands for Activity Measure for Post-Acute Care. Drs.
Alan Jette and the late Stephen Haley at Boston University developed the AM-PAC
with a team of experts. They pioneered the innovative combination of
Computerized Adaptive Testing and item response theory in the first functional
outcome instrument designed to assess patient outcomes across an entire episode
of post-acute care in several or many settings.

The AM-PAC is a true system. It features a suite of tools and instruments.
It is a functional outcomes system that can be used across post-acute care
settings. It consists of a comprehensive list of 240 functional activities,
which together are called the item bank. AM-PAC has already been translated
into more than 20 languages.

Drs. Jette and Haley wrote, “Unlike traditional functional outcome
measures which are disease, condition, or setting-specific, the AM-PAC was
designed to be used across patient diagnoses, conditions and settings where
post-acute care is being provided…and for examining functional outcomes
over an episode of post-acute care, as patients move across care
settings.”

This slide shows a representative output that an investigator using the
AM-PAC would see and use. This particular graphic image shows the expected
performance at each stage in the basic mobility domain. There are similar
graphic outputs for daily activity and applied cognition. The goal of using the
AM-PAC is to utilize information provided by the patient at intake or an
earlier admission to a facility, and then build on that information to present
ICF-oriented functional status outcomes at every stage of the rehabilitation
process.

For example, here the AM-PAC scores this patient’s present and future
abilities to engage in vigorous activities such as walking a mile, standing
from a low chair, reaching overhead, and using the bathroom. This kind of
information is invaluable for case management and persons involved in care
planning and post-discharge planning.

This concludes my overview of the new functional assessment instruments
based on the ICF. I want to introduce one existing data collection instrument
that, when mapped to the ICF concepts and codes, could become a very valuable,
meaningful measurement for patients.

This slide introduces the CMS Minimum Data Set for Nursing Home Residents
Version 3.0 ,or MDS 3.0 for short. MDS 3.0 is a quality initiative. The
development of the new MDS 3.0 represents an enormous leap forward in CMS’
ability to monitor quality of care in our nation’s nursing homes. Moreover, the
data collected from the MDS 3.0 has already been “translated” into a
LOINC database, thereby enabling mapping to ICF concepts and codes.

This slide simply lists the overall goals that CMS had expressed when it
embarked on developing the MDS 3.0. Pertinently, the fourth bullet, which I
have highlighted in red here, pertains to our discussion today — increase the
resident’s voice by introducing more resident interview items.

Indeed, MDS 3.0 is chock-full of patient-oriented items, for example, their
levels of comfort and satisfaction with the care delivered in their facility to
their degree of participation in group activities. All these items are ripe for
mapping to ICF concepts and codes, and such a study is very feasible at this
time.

This slide shows a simple screenshot from one page from the 33-page MDS data
collection instrument in paper form here. We are focusing here on the patient’s
preferences for customary routine and activities. The questionnaire items refer
to the resident’s level of satisfaction with characteristics inside their
facility such as the resident’s preferences for independently selecting their
own bedtime, having snacks, and using the telephone in private.

To sum up, the ICF represents a conceptual framework and a hierarchical
coding structure. It can induce and express those characteristics of
patient-oriented functional status that are most meaningful to patients. The
PAR-PRO and the AM-PAC are two representative newly developed and validated
functional assessment instruments derived directly from ICF concepts and codes.

Moreover, today’s MDS 3.0, a data collection instrument related to quality
of care in nursing homes, can be easily mapped to ICF concepts and codes,
yielding a situation in which we can collect ICF-oriented data from an existing
robust data collection activity. This type of research is underway, and more
mapping studies using the ICF are warranted. Thank you, ladies and gentlemen,
for your attention. I look forward to the remaining presentations and
discussion.

MR. STIEFEL: Good afternoon. I am Matt Stiefel from Kaiser Permanente. I am
Senior Director for Care and Service Quality at the National Headquarters,
longtime colleague of Mark Hornbrook. I think we started in the northwest
region about the same time around 30 years ago. We go back a ways. What I am
going to do is talk a little about the framework we are using for measuring
member and population health at Kaiser Permanente, our portfolio of measurement
systems, and then specifically the question you asked about some of the
barriers and our strategies for overcoming those barriers.

Here is a framework. There are many such frameworks showing the determinants
and outcomes of population health. This is one that we are using, because I
think it helps shed some light on measurement and show some of the causal
pathways, specifically when we are talking about the questions of
self-management and functional status.

Importantly, the functional status measures are at the right side of this
diagram. From the things that we typically measure in healthcare, we tend to
not reach that far in our standard measures of health when we get to health and
functional status outcomes, and even further right, wellbeing measures, which I
will not talk about today, which, of course, measures of health contribute to,
but are not the same as.

I will talk a little bit about the barriers. One of the most important
barriers is provider resistance to measuring functional and health outcomes. I
think an important point, though, is while I believe there is value even there
in the one-to-one care, there are lots of other types of value for measuring
health that go well beyond one-to-one patient care. Even for one-to-one care, I
believe there is lots of value in shared decision-making and monitoring an
individual’s progress.

For population care management, though, I think there is tremendous value in
stratification of populations and doing risk assessment in enabling outreach
and predictive modeling. We have done some work with the Center for Health
Research in Portland looking at the additive value of patient self-reported
measures to traditional diagnostic history predictive models. In fact, while
both do a very good job at prediction, some of them do better than either one.

In surveillance, the outer ring is the accountability measurement, measuring
against the mission of the organization. Or pay-for-performance-type measures,
in example, is now — Medicare stars in using measures from the Health Outcomes
survey in maintaining or improving physical or mental health. I think for the
first time a major payer is paying for outcomes.

In research and evaluation when you think of comparative effectiveness and
think about what we mean by effectiveness, I think that is where we begin to
move rightward in that diagram that I talked about in measuring measures that
matter to consumers, health and functional status. There is not a good analog
in the real world to quality-adjusted life years that are used so ubiquitously
in health services research, but I think there ought to be.

I call this a portfolio as opposed to a system of measurement for population
health. We are moving toward a system, but right now I think it is still just a
collection of measures. I am only going to talk about one of them, our Medicare
total health assessment, in detail, but I will give you a kind of high-level
overview of one system’s attempts at measuring health and functional status.

For the commercial population we have an online tool in the member portal
called the total health assessment, developed in collaboration with
HealthMedia. It is a tailored health assessment that produces tailored
messaging and even tools in programs for members in, for example, stress
management, weight reduction, smoking cessation, back pain management. I will
talk more about the Medicare version of that total health assessment in a
minute.

In our Colorado region we are taking a deep dive at looking at the health
assessment for our own workforce because we have some unique assets there in
having lots of information, not just the clinical information and diagnostic
history, but a lot about obviously where they work, their absenteeism,
productivity, disability. We are trying to take that whole model and look more
deeply at the health of our workforce.

One of the things we are doing now is looking at the relationship between
health behaviors and outcomes associated with both cost and health status. One
of the things that we have found is that there is pretty significant variation
across work unit, supporting the hypothesis that your boss might make you sick.
It is just an observation at this point. We do not have a causality there.

In mortality measurement for a long time we have measured the traditional
measures of mortality, and so have been able to show, for example, in our
Northern California region that if you are a member of Kaiser Permanente,
cardiovascular disease is no longer the leading cause of death.

But that standard mortality measurement is not a measure that is terribly
useful or meaningful to consumers. Life expectancy, I believe, is, and so we
are now working to convert our mortality measures to life expectancy measures,
and in a few cases moving to what I think is the ultimate population health
measure, healthy life expectancy or a combination of quality of life and length
of life and mortality amenable to healthcare as well.

We are also testing in a variety of settings the single question,
self-perceived health. We tend in clinical settings to ask people how they are
doing. This is just a way of making that more systematic and writing down the
answer to that question and tracking it over time.

There are a lot of examples of condition-specific measures in the program I
just highlighted here. For depression we have now developed a little dynamic
assessment tool, an online tool, where we use the PHQ-2 that branches to the
PHQ-9, given the response to the PHQ-2. For hip and knee replacements we are
using a combination of the condition-specific functional status measurement
with a generic tool, the VA version of the SF-12.

Exercise as vital sign is now being used ubiquitously around the program. It
started in our Southern California region. At every office visit members are
asked standard questions about days per week and minutes per day of active
exercise.

We are in a number of settings also using a measure developed by Nico Pronk
at HealthPartners called the Optimal Lifestyle Metric, a simple zero-to-one
scoring of smoking, eating, drinking, and exercise. Those four account for 40
percent of mortality. We are gathering that now almost as vital signs in a
number of areas.

Shared decision-making — I guess it takes a long time to learn. We were
piloting tools doing shared decision-making probably 25 years ago, and we are
starting again. I think part of the issue is that the technology to support it
has changed. I remember when we did them 25 years ago, we were using these
interactive video discs to do it. Now, of course, it is all online and makes it
a lot easier.

Community health assessment — I just point this one out as a tool for you
to consider for gathering this information. With a new requirement for
hospitals and community health needs assessment, it is an opportunity to
standardize that measurement and gather a lot of information about health and
functional status in our communities.

This is the question you asked. I will just go through these three
categories of motivation for consumers and patients, motivation for providers,
and motivation for purchasers. For consumers and patients, I think first is
kind of the theme of this session, measuring what is important to them.

I think this particular subset of the session, measuring functional status
and tools for self-management, is right at the heart of what matters to people,
and also information relevant to shared decision-making. Integrating the
information into customized, personalized action plans that we not just hand to
them but help them with over time.

To administer in multiple modalities, I think we are learning from the
Medicare assessment. We would like them all to use our electronic tools. As you
heard earlier today, not all of them do, especially Medicare members, and
probably will not for a long time. In order to reach all of them, we must
develop multiple modes. For the Medicare assessment we are using the Internet
interactive voice response tablets by hook or by crook, including face-to-face.

Dynamic assessment and parsimony is another key element here. We do not need
to ask every person every question every time. In fact, I think with the
technology today we are able to do much more sophisticated dynamic assessment
and ask only those questions that are needed when they are needed.

Providers are probably the hardest nut to crack here. At our Care Management
Institute our motto for a long time was making the right thing easier. That is
the point of the work, is to integrate it into the electronic medical record
and provide a workflow, but especially to make this not appear to be an added
burden, but actually support by offloading a lot of the stuff that would
historically go on or not go on in the exam room, but to do the assessments
outside the exam room and to do a lot of what we call proactive encounters,
fill care gaps, make referrals, develop care plans before the member even comes
in for the visit.

A lot of the resistance that we have found — and it is probably general —
is that providers either do not have the time, do not know what to do with this
information, see these measures as not being very responsive to clinical
interventions. The most troubling is not believing people when they tell them
how they are doing. That is a tough one. John Ware talks about this a fair
amount. One of my favorite lines of his is, if you want to know how Mrs. Smith
is doing, unfortunately you are going to have to ask her.

Payment for outcomes makes a big difference. I mentioned earlier the
Medicare stars and HOS changing the game. Standardizing measures, another from
John Ware. He uses the analogy to temperature in thermometers, where the
initial argument several hundred years ago, was what is temperature. People
debated not how to measure but what it is. Finally, they lit on some standard
benchmarks like the boiling point and freezing point of water, and then the
debate changed and the challenge changed to making good thermometers.

Where we are with health assessment, I think, is still in arguing about what
health is as opposed to the thermometers that we use to measure it. The
timeliness of reporting to physicians is an issue also.

Motivation for payers — payers are, in this domain, probably further ahead
than the clinical care delivery system in seeing the importance and need for
this measurement. An example of that is the mother of all payers, Medicare. I
bring this one up because I think this is a potential game-changer, this new
Medicare-covered benefit for an annual wellness visit that includes a health
risk assessment and personalized prevention plan as part of that.

We are really riding this horse because I think it is a way to change the
way we care, starting with our Medicare members. What we are doing with the
Medicare health assessment system is basically we have developed an instrument
using items from the public domain, a lot from Promise and other places, for
these kind of standard types of content.

The modes of administration, as I mentioned, we are testing all different
modes. In our Colorado region fortunately we have a lot of our members, even
our Medicare members, who use our member portal, so about half of the responses
so far have come through the secure email route. As I mentioned earlier, to
address the issue of physician workload, a lot of this is done outside of the
visit, and a lot of the proactive encounter work in making referrals,
developing the plan is done before the member even comes in for a visit.

The data are automatically uploaded into our electronic medical record as
well as our population care management systems, and then used for screening and
outreach in the other population care applications that I mentioned earlier.
The information automatically pre-populates this personal prevention plan. That
is then discussed during the visit with the provider, and the provider adds to
it and gives it to the member as they leave the visit.

We are not quite there yet, but we see this as the front end of a more truly
dynamic health assessment system. I do not think you are going to be able to
see these items, but basically the system works where members at enrollment and
then annual on their birthday are given this total health assessment. That
information is coupled with information about the social and physical
environment, medical history.

Healthy members then may not be bothered by us until they come in for a
visit. Unhealthy members we would reach out to, to either do more dynamic
tailored assessment or actual clinical outreach. Then when they come in for
visits, the information is updated. It is shared and discussed with a physician
during the visit, and an action plan is developed post-visit.

I do not think I will spend a lot of time here, but the idea is that the
types of measures for population health and types of settings in which it is
measured can be part of an overall system where at the highest level we are
measuring the broad population outcome measures. Then as you drill down, you
get into the behavioral risk factors. For members you would want to see
something — maybe the analogy is like for banking you get a balance sheet and
a prospectus for your health, including a plan for a financial plan.

Another sort of John Ware analogy here — he uses this very elegant ruler
analogy to looking at a system where at the different levels for population
monitoring, group level outcomes monitoring, and patient level management it is
all part of one system, just the granularity changes for the specific type of
measurement.

At the population level you may be looking at a single measure for measuring
overall population health, when you get down to group level or population
management, a multi-item scale, and then for the individual a more tailored
dynamic assessment system. But the point is that each point in that ruler is an
unbiased estimator of the more generic performance.

I am not going to go through a lot of detail here, but it is information
that is made available to you. This comes from two very large systematic
reviews from Kazis and from QualityMetric looking at over the past 20 years
thousands of studies using principally the SF family of tools. That question
that I mentioned that was the big stumbling block with physicians — is there
anything that we do that has an impact on health? This first one is impact on
physical health measures, and the second one is impact on mental health
measures.

Just some observations from this is the first one you will see is that aging
one year, in fact, is bad for your health once you are at Medicare age. Then
you will see also, as you would expect, the major clinical conditions have
obviously significant impact on self-reported health status.

For the surgical interventions I guess it is good news that the major
surgical interventions do have a significant impact on functional status. It
does not really do us much good. To improve our Medicare star scores, we are
not going to go out and do a bunch of hip and knee replacements. But
nonetheless, it is good to know that they move in the right direction.

For the medical interventions you will see a lot of drug-related therapies
showing up. I think that is partly the availability bias of the funding for
studies like this tend to come from pharma.

The third and probably most useful is that the behavioral interventions,
exercise and behavioral therapies, do seem to have a significant positive
impact on self-reported health status. Those are things that we can do
something about and invest upstream in to improve the health of our members.

The last point I will leave you with is that the ultimate reason for
measuring functional and health status is that is the ultimate purpose of
healthcare articulated in our mission and vision statement. I believe it would
be fun to do a study of how many health systems have a mission statement in
which they say their mission is to improve the health of their population. But
we tend to not do a very good job of measuring that health from the patient’s
point of view. Thanks very much.

DR. TANG: Thank you. I think the fourth person on the panel, John Wasson
cannot be here, but he sent in some testimony that will be available. We will
open it to questions. On one slide you said it cannot be transmitted in
electronic form. Of course, that caught our attention. Can you explain why? It
looked like you had some alphanumeric, and maybe that is part of it. Explain
why, because it looked so wonderful in terms of the way you were able to
essentially tell a story just with a bunch of numbers.

DR. HOUGH: Thank you, Dr. Tang. It is a very nice comment and a good
question. I will do my best to answer it. I do not think it is quite
answerable. In brief, there is very little profit motive at this stage. I do
not mean to cast all motives with that paintbrush, but until it is part and
parcel with reimbursement, it may be difficult to fully electronify. Or, stated
another way, electronification stands at the research stage now rather than at
the application stage.

On the other hand, there are folks in our meeting room here as well as our
colleagues through NCVHS who are moving rapidly towards electronification. I
will point to our colleagues at Regenstrief Institute who are moving rapidly
towards developing a clinical LOINC analog of the entire ICF.

It is moving forward, but it is not moving forward rapidly. I will leave it
at that. It is a complicated answer, and I am pleased to take it up with you
later. Or perhaps our other panelists might have experience with it and could
answer as well.

MS. GREENBERG: Thank you, John, and thank you for your presentation. It is
available in CD-ROM and it is on the UMLF. I would say it is about as
electronic maybe as other classifications. But I think the main issue is that
in addition to these code stems that John showed, it also has qualifiers. In
order to have a version that includes every permutation or whatever, that would
be the benefit of LOINC-ifying it, as it were.

You have got the classification, but you do not have a version that has a
separate LOINC code with every qualifier. That would make it more useful. It is
in the UMLF, just as ICD, SNOMED, et cetera, but it requires more to easily put
it into a system, particularly if you want to use all the qualifiers. But it is
not rocket science. As John said, it certainly could be done, and there are
people working on it. It has been recognized by meaningful use and others as a
standard, but it is not required by anybody, so that is why the work probably
has not been done.

DR. HOUGH: I will just use the adjective that we were familiar with the
stimulus bill. It is a shovel-ready project. It is ready to go. It is not lack
of motivation. It is lack of personnel resources and money. Bluntly, it is not
an extramural research fundable topic of interest at the moment either.

DR. TANG: It was a very interesting example you gave that showed the
flexibility and the dynamic range of this coding system. I assume you can do
the same thing with someone with heart failure. You can really describe in much
more detail than we just have this AHA classification system. What is going on
with this person’s life and how can we improve it, and can you tell? That seems
like some richness to this kind of thing.

DR. HOUGH: That is well said. I will just make one quick comment, and then
take your question. In clinical applications I think the real advantage is
almost a graphic description using these qualifiers of the patient’s course,
and that is not necessarily to say the patient’s course of improvement. In
fact, if the patient does not improve or stays the same for a long time, the
qualifier digit does record that.

I will stipulate that the coding examples I showed in my slides were
point-in-time, only one at a time. That gentleman who had the prosthetic arms,
his functional mobility probably is not going to change very much over the
duration of his life, but a person with a mobility problem and such who goes
through the continuum of disablement, good days and bad days over a long
period, the code can reflect the change over time as well as at one point in
time.

DR. TANG: I will just say that one of the main reasons we are having this
hearing and looking at this topic is so that we can motivate change. This seems
like such an opportunity. It is timing. The funding situation — as you
described, it just was not possible ten years ago, five years ago. But this is
in a very new era, and we have an additional tool like meaningful use that can
help push these things. We certainly are aggressively going in the direction of
measures that matter to consumers. It is good timing.

DR. WARREN: The first question I have is, is there a license fee for ICF?
You knew that was coming.

DR. HOUGH: It is a wonderful question. I am glad you have asked it. Yes,
there is a license fee. The copyright holder is the World Health Organization.
My understanding is licensure fees are negotiable or that there is no flat fee.
I was fielding this question from an American colleague last week, and both of
us agreed that, indeed, the only people that would matter in the discussion are
attorneys.

But the point is there is a negotiated license fee. I believe that that
would also pertain to an electronic version licensed to a health system or a
health plan. The only analog we have seen so far to date is exactly as Ms.
Greenberg mentioned, namely the incorporation of ICF into the UMLS, which
involved a negotiated license fee.

DR. WARREN: Even there, there is the codicil. It may be in UMLS, and UMLS is
accessible, provided you pay the license fees for the terminology you download.
It is a nice little codicil in there that protects the MLS.

The second observation is I have actually been working with two of the
physical therapy faculty at University of Kansas and we have been trying to
explore how to make this into a EHR. The problem is it is a classification. It
is not a terminology. The coding scheme violates most of the good practices in
terminology, because the code should have no meaning. You could look at all Jim
Cimino’s work. It really does make a difference because as ICF evolves over
time, they may run out of numbers in their codes, which is where we got into
with ICD-9, and ICD-10 is going to have the same problem as it progresses.

The thing that I wanted to comment on is there is a combinatorial explosion
because of the modifiers, so if you were to turn it into a terminology, you
would have to address that or you would have to take the classification and
turn it into an assessment tool, which you have said it is not. But it
tantalizes you with the ability of turning it into an assessment tool, and then
you can generate the extensive coding that you were showing in your examples.

DR. HOUGH: That is well said. I won’t suffix those remarks except to say
that the combinatorial explosion factor is daunting, and yet, as Marjorie has
indicated, it is not rocket science. We can handle it. I am happy to be reading
a book by a colleague right now whose thesis is that ICF works better with DSM
than with ICD. It is much closer.

Even though that author is not taking it as far as your comment has
suggested, namely to make this classification into an assessment instrument, he
is essentially suggesting to rehabilitation counselors in their application of
the ICF that they do exactly that, use it as an assessment tool in tandem with
DSM.

MS. GREENBERG: Just to expand, first of all, it is was in the UMLS freely
available in the US in the same way as the agreement with SNOMED for a very
small amount of money comparatively. It is a classification. It is not a
terminology. It is not a vocabulary. It is a classification. In any event, that
contract has lapsed, but I think it could be easily renegotiated. We were
paying $50,000 a year. This was not a rounding error, to say the least. But if
there was an application and interested agencies or organizations, I think that
could be resolved, and it could be available in the same way. I will not go
into details, but some of that is under discussion.

DR. WARREN: That is why I asked the question, because that could be a
recommendation that the subcommittee makes, is that that license be renewed to
try to provide stimulus to people to figure out how to do this and to put it
into EHRs, et cetera.

MS. GREENBERG: We did have funding from NCHS, SSA, and ONC, but the latter
funding dried up. But in any event, again, this is not rocket science or high
finance.

The other thing, though, is that there is a project to map ICF and SNOMED
CT. Although the project has not been funded in the way that the project to map
ICD-10 and SNOMED CT has been, there are people already working on it. I think
the problems you described all are definitely solvable.

DR. MIDDLETON: A couple thoughts, but first a question. What is the usual
use case for the ICF? Who is the user?

DR. HOUGH: That is a fantastic question. I am grateful you have asked it.
The best answer is researchers or, I will say, practitioners in the
functionally assessing disciplines, physical therapy, occupational therapy,
speech/language pathology, and pediatrics and gerontology, as well, on the
medical side.

DR. MIDDLETON: Has any work been done, to your knowledge, that has applied
perhaps a naïve end-user-friendly front end to the tool so that I or my
mom could use it and do a self-assessment?

DR. HOUGH: That answer, I am afraid, is no.

DR. MIDDLETON: Once again, I am a convert. I strikes me that we are going to
have outstanding phenotypic understanding of a person’s health state. We are
going to have increasingly outstanding genotypic understanding of your genetic
profile. We are increasingly going to have an outstanding behavioral profile
assessment. We do not have in this mix perhaps that fourth pillar that is this
functional profile. If we really intend to contextualize a lot of things we do
in health and medicine and prevention, et cetera, perhaps this needs to be one
of those four pillars.

DR. WARREN: So users of the system, when the Consolidated Healthcare
Initiative came forward to NCVHS for us to validate, it was DOD and VA that
wanted to use ICF as part of the Wounded Warriors Program, because they were
seeing such disabilities and injuries, they needed a way to finely code so they
could document how well they were. That is why we recommended it to them.

MS. GREENBERG: At SSA also, the Social Security was also interested. There
is actually an RFP currently going through the process now from DOD, for some
development work with ICF. It is the first federal agency that has put out some
money, which is not a lot, but it looks like a lot to me compared to what has
been out there.

You said you need to turn it into an assessment tool, but this is the whole
thing. Assessment tools are being and can be built on it. They do not have to
use the coding system if a code can be underneath. They just get applied,
because you develop the question and the answer that goes to the right code. It
is usable as a coding system, but I would not expect consumers to use it. But
the questionnaires they respond to can be coded to it.

DR. MIDDLETON: They could use the tool, as could the doc, as could the
nurse, any non-expert developer. You just have to have that front end.

DR. COHEN: This is following up on this conversation, a question and then a
general observation. There is an impressive amount of work going on in
measuring functional status. What is the state of the art of providing this
information to consumers so that they can use this information?

MR. STIEFEL: We are using it in this personalized prevention plan to our
Medicare members now. The Medicare Total Health Assessment includes several
function status outcomes. Then that plan is shared with the member.

DR. COHEN: So a member goes online or works with a case manager, completes
the assessment, and then talks with someone on staff about what this means in
terms of the medical care they receive or lifestyle changes they need to make?

MR. STEIFEL: Yes, in this case, in the Medicare one, in the annual wellness
visit, which then from that produces the personalized prevention plan.

MS. SMITH: From our standpoint, I would say that it is probably variable
across the profession. There certainly are a lot of functional outcome tools
out there, and I did not even talk about condition-specific or
body-region-specific tools, although I did mention those categorizations of
tools, but there is not a standardized way in which that gets reported. That
really takes place between the clinician and the patient.

DR. HOUGH: Unfortunately, my answer is negative. I am not aware, other than
these examples that have just been described.

MR. STEIFEL: For our hip and knee replacement work we are doing four
assessments, one pre-op, and then at three months, six months, and twelve
months post-op with the condition-specific functional status as well as generic
health status.

MR. QUINN: Using the ICF coding?

MR. STEIFEL: Not using the ICF, using standardized tools. I think it is the
CUSE(?) tool and the HUSE(?) tool plus the R12.

MR. QUINN: I was just going to say that every primary care visit I have ever
had started with the question, how are you, tell me about what is going on.
This could be a way of encapsulating that conversation into a code or series of
codes that could then be tracked over time or could be assessed. I am wondering
if there has been any use of this coding system as a way of measuring that for
individuals or populations.

The other is thinking about my mom who recently had back surgery. Before the
surgery, she was given the alternative treatments. The doctor said on average,
most people feel better and have less pain or are more functional after a few
months. So she did not really know whether she was on track. Are some surgeons
more successful than others, and not just surgery but surgery plus physical
therapy plus the rest? I was wondering if it has been used in that way either
for individuals or for providers or for teams of providers or for populations
to track those things over time.

DR. HOUGH: Sadly, my answer, unfortunately, is again no. I will point out,
though, in response to this set of questions that, indeed, as has been voiced
in discussion and in our slides, the ICF is not a classification of disability.
Instead, it is a classification of health. Stated differently or stated in
qualifier terms, health, if you will, is the zero point. On anyone else’s
dashboard or speedometer, it is to the far left.

Stated another way, to the individual patient who receives a description of
why this type of coding from a classification might be of interest to them as
individual patients, I would be curious to know, for example, some gradation of
my health. I feel really good today. I feel healthy. I am a zero today, and I
want to stay a zero as long as possible. I know that sounds funny.

I do not want to be a one or a two or a higher level that indicates greater
severity. To the patient or the layperson, I think the capacity to know that
their doctor is referring to them in terms of their health rather than their
disease is the manifestation that is a positive attribute of ICF. For lack of a
better term, it is patient-centered.

DR. HORNBROOK: One of the things we are missing here are elements of dealing
with behavior. Many of Kaiser’s costs come because people will not change their
behavior. They will not stop smoking. They will not lose weight. They will not
exercise. That does not necessarily show up on the ICF because they are not
asking about motivational issues, they are not asking about your motivations
for not wanting to do certain things. Our challenges right now are basically
how to interact with our own members, our own patients, our own society, and
getting them to do the things that make them feel better and actually stay
healthier longer.

DR. COHEN: I think that is a really astute observation. I was struck by the
last slide about Kaiser’s mission is to provide high-quality affordable health
services to improve health. It is fundamental, but I do not know whether that
— when we step back, a lot of this motivation has to do with healthcare
reform, but in fact, most of it is medical payment reform at this point in
time.

The incentives for those who are concerned with medical payment reform might
not necessarily be the same that are of those who are concerned with improving
that health in populations. Until we make one equal the other, it is going to
be really difficult to figure out how to create the incentives in the system to
really improve populations’ health.

I thought it was wonderful that trying to build in a measure of functional
status really looks at, hey, maybe I can save money by doing things other than
additional traditional treatment interventions and actually improve the health
and wellbeing of the persons that I am serving. I think it is as much about
cultural shift as anything that we are required now. It has always been easier
to intervene in treatment because those interventions might not be successful,
but they were clearly demarcated and explicit. Trying to, as Mark said, get
people to do things that improve their health and wellbeing is a real
challenge. Some insurers and plans have not embraced that as part of what they
should be doing.

MR. STEIFEL: It is a great point. Several years ago a little “and”
found its way into the mission statement — to provide quality affordable
healthcare services and to improve the health of our members and community. The
“and” is important because there is a limit on the contribution of
high-quality affordable healthcare services to the health of the population. It
is not just nominal. It is causing a change in the way we think about
population health, and to reach into the upstream behavioral factors certainly,
but even further upstream into the social and environmental factors. So a lot
of our community health work is now in schools.

DR. TANG: Heather, I think you said OPTIMAL was the only non-proprietary.
The other ones were NQF-endorsed. I thought one of the rules was that it cannot
be proprietary or carry a license fee and be NQF-endorsed. They are an
exception to that?

MS. SMITH: Yes.

DR. TANG: The cost is going to come into our discussion in terms of trying
to make recommendations.

MS. SMITH: For instance, FOTO was required to share the proprietary
methodology, but in order to implement and build that will cost someone else
money down the road as well.

DR. TANG: Thank you so much for this very enlightening panel. We are going
to do the 10-minute break again and reconvene at 2:50 please.

(Break)

DR. MIDDLETON: We are going to reconvene. After the break, the first session
is use of patient experience and satisfaction measures in assessing whether
consumers or patients achieved their goals and expectations. We have another
distinguished panel.

Agenda item: Use of Patient Experience and Satisfaction
Measures in Assessing Whether Consumers/Patients Achieved Their Goals and
Expectations

MR. SHALLER: Thank you very much. It is a pleasure to be here. I appreciate
the invitation to add to the testimony for the subcommittee hearing. I am a
member of the National CAHPS Consortium and have been involved with CAHPS
research since 1995. What I would like to do today is review some background on
patient experience assessment and reporting and pay particular attention to
some research that we have done recently that looks at how patient comments can
fit into the picture of CAHPS’ patient experience measurement and reporting.

There are several approaches to measuring patient experience. One is the use
of standardized patient surveys. There are proprietary tools to do that. Most
of them focus on satisfaction. There are public domain instruments as well.
CAHPS is probably the best example of that.

There is also a growing activity around patient comments. What I mean by
that are user-posted online narrative anecdotes that you typically see on
public websites. We will talk a little bit more about that in a moment.

I just want to also mention other approaches beyond surveys and comments,
internal use by providers and all systems that involve targeting rapid-cycle
surveys for PDSA improvement using focus groups and interviews, using
walkthroughs and shadowing, and mystery shopping. There are a number of
qualitative aspects of assessing patient experience that go beyond patient
experience surveys.

Many are familiar with CAHPS, the Consumer Assessment of Healthcare
Providers and Systems. It has been around for 15 years. It is the most widely
used survey set of tools for assessing the patient’s experience and care. It is
endorsed by the National Quality Forum in 2007. It has been funded by AHRQ
since 1995 and has a number of members in the consortium that include AHRQ, the
Centers for Medicare and Medicaid Services, teams of researchers from RAND, a
member of the Yale team, and our supported contract is with Westat.

The number of surveys that are based on CAHPS has been growing since the
beginning. There are both ambulatory surveys and facility surveys. There is a
quick list here.

All of the surveys that belong to the CAHPS family have some core design
principles in common. They include the focus of these assessment tools around
those topics for which consumers or patients are the best or, in some cases,
the only source of information. It is a very important complement to more
objective or clinical-based measures of patient care.

We focus on patient reports, whether things happened or didn’t, and the
experience of interacting with health professionals or the healthcare system.
It is not about ratings. It is not about satisfying. It is about whether things
happen or not from the patient’s point of view.

All of this is based on really very rigorous scientific development and
testing and a lot of input directly from consumers and patients through focus
groups as well as stakeholders within the healthcare system. All of this
information and tools and supporting documentation is in the public domain.

Who uses CAHPS surveys? There are a number of users. With respect to the
CAHPS Health Plan Survey, which is the oldest, just about every health plan in
the country is probably being assessed by a CAHPS survey. All Medicare
Advantage Plans and the fee-for-service system are subject to CAHPS surveys.
Commercial plans that seek accreditation from NCQA. About half of the state
Medicaid agencies in the country have historically used CAHPS. That will expand
to all of them because of the recent CHIPRA legislation, which requires them to
collect and report CAHPS health plan data.

Hospital CAHPS has expanded rapidly since 2008. It is now being used by over
3,800 hospitals in the country and will continue to grow, particularly with
value-based purchasing that is going to be kicking in in 2012 by CMS.

The newest ambulatory survey that we have in our portfolio is the CAHPS
Clinician and Group Survey, or CG-CAHPS. This is just getting started. It is
actually just really taking shape through regional public reporting
collaboratives, such as Aligning Forces for Quality and chartered value
exchanges around the country.

Department of Defense and Veterans Administration use CAHPS surveys at the
ambulatory level, and in growing numbers of practices and systems that are
adopting one or another version of CG-CAHPS in response to seeking
patient-centered medical home recognition or the new ACO accountable care
organization regulations that have been issued. Also, the American Board of
Medical Specialties has a new approach to maintenance of certification that
allows the use of CAHPS surveys for assessing communication skills as part four
of MOC.

What are some of the obstacles to implementation of at least the clinician
group, this newest form of CAHPS surveys, getting at patient experience within
medical practices or with individual clinicians? One of the biggest issues is
there really is no clear national signal yet, unlike HCAHPS, where Medicare has
made it very clear that that is what they are going to require for reporting
and now payment. There are a lot of mixed signals with respect to ambulatory
care surveys at the practice and site level.

Practices are dealing with multiple demands, things that they are trying to
do internally for improvement tied often to compensation for their physician
staff, and then dealing with multiple conflicting requirements for public
reporting and accountability. There is lots of noise in this system.

Cost is another big issue. I think there is promise here in moving toward
more electronic methods of collecting information, primarily through email
invitations and other IT applications.

We also have an issue of growing respondent burden. There are a lot of
surveys out there. Until we kind of coordinate and rationalize some of this, I
think it is going to continue to depress response rates. I think if we can
create value for patients as reporters on their experience and assure them that
they actually are getting something out of completing these surveys, I think
that will help address some of the barriers in terms of response rates.

There are some myths, too, that are out there that need to be addressed. One
is that patient experience is a nice thing to do, but it is not really
essential. I think, at least, in certain medicine acknowledge that it is an
important aim on par of all the others that we are trying to achieve in the
healthcare system.

Another myth is that patients will not answer more than 10 questions. We
actually know that that is not true, that once you engage a patient in
answering a survey, it can be 90 questions long and we can still get comparable
response rates.

Another myth is that what you do for accountability and public reporting
really does not match or cannot be used for improvement. We are finding that
that is not true either. The greater alignment that we can produce in the
system so that a same survey instrument can be used for both applications, I
think it will improve effectiveness, lower cost, decrease response burden. We
will get more out of this altogether.

Finally, providers worry that it is impossible to improve these scores. It
is very difficult. Patient experience is unlike some clinical process measures
where there are pretty straightforward ways of improving process.

With patient experience, really you have to fundamentally transform your
organization from the top down and implement specific tactics and strategies,
but in the context of an overall organizational transformation. It is very
difficult, but it can be done, and we are collecting more and more evidence
that that is true and also some strategies for actually doing it.

I want to shift gears quickly now and talk about consumer response to and
engagement with all this information we are collecting through patient
experience surveys. There is a lot of public reporting of CAHPS at the hospital
level through Medicare Hospital Compare. I mentioned the Aligning Forces for
Quality, the 16 markets around the country that are required in terms of their
funding from Robert Wood Johnson Foundation to achieve public reporting of
Clinician Group CAHPS.

Physician Compare is going to come online. It is online, but it is supposed
to add an experience of care measure in 2013.

NQF’s measurement Application Partnership just issued their report
advocating that CG-CAHPS be used for all federal measurement and reporting
programs. That is kind of a significant endorsement and additional lever.

In Minnesota we have a mandate now that all practices will collect and
report CG-CAHPS. That will be made available for the public. Growing numbers of
hospitals and health systems are doing this on their own anyway.

Here is just one example. I could show you a lot of examples. You have
probably seen many of these yourself in terms of one way of public reporting,
some domains of the CAHPS Clinician Group Survey. This is off of a website in
Minnesota called Minnesota HealthScores. It is sponsored by one of the Aligning
Forces programs. You can see bars that show percent achieving the most positive
response for these various domains of the CAHPS, including access and
communication, courtesy of office staff, and an overall rating.

Here is a problem. We have a lot of examples of this. We also know that very
few people are really paying attention to this. There is very little
patient-consumer engagement in the public reporting of all these CAHPS and
patient experience survey data that we are collecting.

One thing that is in stark contrast to that is this growing proliferation of
websites such as Angie’s List or Yelp or Vitals or you name it in your market,
where people go online. It is similar to going on Amazon and some other sites.
You are invited to register a comment and also rate your provider at the same
time. This is happening, and it is happening a lot. I think sometimes the
public may confuse these kinds of ratings and self-reported information on a
website as a good way of assessing patient experience.

Actually, the extent to which this is happening is kind of remarkable. This
is from a recent Pew Research poll. We know a lot of people who are connected
to the Internet go online searching for health information, about 80 percent.
This poll reports that 16 percent of those online users have actually looked up
reviews of a doctor.

That is about twice as many as were reported in the Kaiser Foundation poll,
the most recent one available, about those Americans that have actually seen
and used information on healthcare quality. It is a much higher percentage of
the public going online looking up reviews, 4 percent of which have actually
registered their own reviews. This is a mixed bag.

Here is an example. I could show you a lot of examples. This comes from the
Yelp site in San Francisco. This is one physician who happens to be a
practicing doctor at Kaiser Permanente. This is one rating and one score. He
gets one star. The comment I do not think you can see. I will read it out loud
for you. “0 Stars. What an ass. My first bad experience at Kaiser. A
complete waste of getting off early from work. He was rushing me and acted as
if I was wasting his time…He was very rude and not professional.”

We know through other metrics that Dr. Okuhn is actually very competent,
very compassionate. He scores high on all their measures that are internal and
much more valid than this particular single posting. Someone had a bad
experience and wanted to make it known.

Here is the issue with posted reviews. There is no scientific sampling going
on here. These are totally from wherever. They are usually very small numbers
of people that go onto these sites and make these kinds of comments. They can
be either kind of wildly favorable or wildly negative.

The other side of the story is that there is a lot of power in this because
we know that this is attractive to consumers. While we have difficulty getting
the public to pay attention to metrics and statistics and scores, people do
gravitate to these kinds of narrative reviews.

I am going to talk a little bit now about some research that the CAHPS team
has done recently. We wanted to know can we leverage the power of patient
comments along with patient experience survey measures and other measures of
performance. How do we do that? How does including patient comments and public
reports affect understand and use particularly of CAHPS’ other measures of
performance, and how do they affect the quality of consumer decision-making?

We put together, starting two and a half years ago, an experiment that we
called SelectMD. We created a website. It is fictitious, but it is very
comparable to many of the public reporting websites that you see out there in
terms of content and format and functionality. We had a tracking system that
allowed us to monitor the use of the system. We sampled a representative sample
of online users through the Knowledge Network, a cohort of about 850 people,
all with access to the Internet. We put them into six study arms.

The randomization was treating them for different conditions of this website
that had varying degrees of information, some just with comments, some with
CAHPS and comments, some with CAHPS and technical measures of quality like
HEDIS and comments, and then varying degrees of complexity in terms of number
of doctors, from a set of 12 to a set of 24. We asked these respondents a bunch
of questions beforehand and then after they had cycled through the website.

Just to give you a sense of what this looks like, kind of the landing page.
We gave them a brief tutorial once they were enrolled in this study. We had
them typically choose the type of doctor, enter their zip code, how far they
were willing to travel. We sort of made this kind of sponsored by The Better
Health Coalition so it had kind of this sponsor reality to it.

Then they were put into one of these six study arms. This particular one has
all of the measures, both kind of an overview performance and service quality,
which is CAHPS treatment quality, which is HEDIS and patient reviews. I am not
going to go through all this, but just to give you a sense of —

This is the overview page. You see doctors listed on the left. We are using
a star rating system, which is pretty common. We allow sorting by gender and by
experience that the doctor has. They can single out specific doctors to look
at. In the end, they had to choose a doctor based on whatever arm they were in,
whatever information they were exposed to.

Here is the comment part of it where if they selected patient reviews, they
would be able to read these comments. These are modeled after actual comments.
We tested them rigorously. We made the comments correspond to CAHPS scores, so
positive comments were correlated with positive CAHPS scores, and the emotional
valence was pretty randomly distributed. Whether they were very strong or very
weak kinds of comments was something that we basically distributed evenly
throughout the various comments. Then we even let people put a comment of their
own in if they wanted to. In the end, they had to select a physician based on
the information they were exposed to.

What did we find out? I could show you lots of tables, and we are still
sifting through the findings of this, but basically what we found out is that
when you include patient comments, it increases people’s engagement with the
website. They do more on it. They probe deeper. They spend more time on it.
They are much more likely to say I would use a website like this in the future
and/or recommend it to their family and friends.

We also found that including patient comments leads to people paying less
attention to these other metrics that we think are maybe more accurate in terms
of CAHPS or HEDIS, but they are not any less likely to trust these measures,
and they find when they are exposed to comments that it is actually easier to
use and understand the CAHPS scores. So we have a couple things going on.

We also learned that when you include patient comments and you factor that
into an assessment of what was the real quality of the doctor they selected,
they selected the lower-performing doctor based on CAHPS scores and on overall
quality using CAHPS and HEDIS combined. This is kind of a conundrum. It is a
little bit of a mixed situation.

Here are the implications for public policy, public reporting moving
forward. Patient comments do add value. We found that to be true. People are
more engaged in public reports, but when you put them in, they may crowd out
these other kinds of measures that we really want people to pay attention to as
well.

Our research is going to lead us into working on reporting strategies that
find ways to leverage the positive parts of comments without detracting from
these other important dimensions of performance that we want people to engage
in as well. I think I will end there and hopefully have time for questions.

DR. COHEN: Just a quick question so I can understand this. Where there was a
difference in the rating between the quality measures and the ratings, the
reviews, people trusted the reviews more than the other measures?

MR. SHALLER: They trusted them equally. They trusted CAHPS and HEDIS. They
used patient comments more. They were overall more engaged in the website when
there were comments included in the display. When the respondents that were in
those arms where comments were included, when they had to end the experiment by
choosing a doctor, the average was a doctor that they chose that had lower
CAHPS and HEDIS scores. I do not know if patient comments also lead to worse
decision-making, but that was the way the comments related to the quality of
the doctor that was selected.

MS. HOLLIDAY: I am Regina Holliday. I am a patient artist advocate. I am
always painting about healthcare and also the patient view. I am so excited to
be able to present today.

I might have a suggestion as to why comments might be used more readily than
statistical data. That is one of the reasons why this PowerPoint is called
“Measure for Measure” from Shakespeare’s play. Throughout this
PowerPoint you will see quotes from Shakespeare because regular people do not
necessarily understand data in an analytical format, and they understand
narrative really well. One of the things to keep in mind is how we depict data.
If we depict it in a narrative format, we will oftentimes engage a patient
audience much better.

“The law hath not been dead, though it hath slept.” Daily, 25,000
patients are surveyed about their hospital experience. Each day more than 7,500
patients complete HCAHPS surveys. As of 2013, total performance score at
hospitals, the VBP, value-based purchasing, will have two components. Those are
clinical process domain. That is 70 percent of the total performance score.
Patient experience domain is 30 percent of the total performance score.

Let us look at alignment. I thought I should actually define alignment when
I looked at this slide. I really love the third definition: binocular lenses
that are out of alignment will yield a double image. I think we are getting two
different datasets that are not correlating well when we talk to patients the
way we are talking to patients and we talk to clinicians the way we are talking
to clinicians.

I love HCAHPS scores. I have painted about HCAHPS scores. I think they rock.
I thought when HospitalCompare came out, yes. Anyway, these are the HCAHPS
scores of two local hospitals compared. I de-identified the hospitals that were
picked. If this patient experience was depicted using a child’s report card,
these two facilities would get Cs, Ds, and Fs. These are the same two hospitals
based on clinical process scores. They got Bs and As.

Let us analyze the ramifications of this kind of divided view. The patient
view, as defined by the HCAHPS Handbook from the Studer Group: “patient
perception of care” is a lot more than making sure nurses and doctors are
friendly and smiling. It is about saving lives and delivering healthcare. It is
about quality in a very real, concrete way.

From the Nature of Suffering by Eric J. Cassel, the clinical view:
“Diseases are the ‘real’ things — the things that count. Symptoms
are a second-best access to the disease entity, the best being ‘direct’
views, such as X-rays, tissue examinations, electrocardiograms and so on. From
the same perspective sick persons, as persons, are a agglomeration of
‘soft data’ — feelings, emotions, values and beliefs — in these terms
they are not as real as their disease.”

I have a whole bunch of points to make right here. In the absence of time, I
make them again, but this is a nice bullet point to take home with you that you
can study in depth. We must share in the creation of datasets and compare like
to like. What is mine is yours, and what is yours is mine.

Condemn the fault, and not the actor of it? Currently, patients have limited
access to their medical record. In some cases no access is available until
discharge. In a best case scenario the access consists of labs, imaging
reports, discharge summaries, and medical administration records.

Rarely does a patient have access to a doctor’s progress notes, nurse’s
progress notes, or medical administration reports. Very few patients can add to
the medical record in real-time, adding either clinical patient-reported data
or soft subjective data of personal experience. The HCAHPS surveys are sent to
a sample of patients 48 hours to six weeks post-discharge, and patients are
expected to submit their responses based on their memory.

The clinical dataset is provided through Medicare claims data and additional
data submissions through the hospital EHR. This data is created in real time
using clinical decision support or computerized physician order entry.
Providers are not expected to remember events that unfolded up to six weeks
before and report upon them.

“The demi-god, Authority.” The patient portal and clinical
decision support for patients. In order to create patient measures that have
equal weight when compared to clinical measures, patients and caregivers must
have access to tools of data creation and capture. Only then can we attain
equal footing. Patients must be a part of the active process of data creation
from triage through the entire episode of care and culminating in a qualitative
discharge summary.

“The miserable have no other medicine, but only hope.” Many
patients are part of online patient communities that openly share data. Some
patients with rare disease want to share their data. Measures should include
input from rare disease patients whenever possible. Although concerns exist
that a small data pool can lead to identification of an individual patient,
that patient should be asked if they would like to be included in such datasets
rather than being subject to a default decision of exclusion.

We must also consider the views of those who have lack of access to
information due to mental state status. Their need for data access is as great,
and so is their need for appropriate survey questions. Too many patients cry in
solace with no recourse. Pain can be treated in many ways, but the journey
starts with the recognition of that individual voice.

Patients with disabilities may need accommodations to fully participate in a
qualitative and quantitative data capture. No one should feel left out of this
process. Every voice is important.

“Haste still pays haste, and leisure answers leisure; like doth quit
like, and Measure still for Measure.” Measures should encourage data
capture at the beginning of the patient journey. At the birth of a child many
parents are fully engaged in the healthcare process, and they should be
included by survey questions that apply to their unique experience.

Further, this window of time can create a parent who is empowered in their
healthcare choices, as well as teaching the young to be empowered as well. At
the beginning of a child’s healthcare journey they are fully open to and
embracing the ownership of their personal health. We must reach out to such
young ones and include them as well. If a child can write, they can be part of
building their patient story.

“They say, best men are molded out of faults, and, for the most, become
much better for being a little bad.” Patients want to partner with their
providers in the creation of the medical record. Such patient access allows
errors of miscommunication to be corrected in a timely fashion.

We all make mistakes within an episode of care, but it is a fool that
refuses to learn from their mistakes. Often the mistakes we make will sear a
cognitive path within our mind, and we will learn lifelong lessons if we
address our failings.

Patients who reported that their bathroom and room were always clean. How do
we define clean? How does that differ from disinfection? Should we be asking if
the staff always washed their hands? Has the patient been informed about the
dangers of hospital-acquired infections?

“Some rise by sin, and some by virtue fall.” Patients are not
compliant is the refrain echoed at medical practices and conferences. With one
stroke of the brush whole populations are painted with scarlet shame. Measures
should reflect both populations and personal choice. If patients are
non-compliant, we must discern the barriers that patients face in their journey
to have better healthcare and their journey to a peaceful death.

Measures cannot exist unless access is open and transparent, and this
transparency must continue until the end. In order to provide necessary data
for evidence-based medicine, we must reinstate a statistically significant
autopsy rate. In the United States hospital autopsy rates of 60 percent in the
1950s fell to 12 percent in the early ’90s and are less than 2 percent today.
Private autopsy services begin at $2,000 and range to the upward of $15,000 and
are out of the reach of many consumers. In addition, studies find that there is
a disagreement between pre- and post-mortem diagnoses in 30 percent of cases.
“Truth is truth until the end of reckoning.”

“We must not make a scarecrow of the law, setting it up to fear the
birds of prey, and let it keep one shape, till custom make it their perch and
not their terror.” We must consistently reevaluate the measures we are
using to determine that they are still assuring the quality and care in the
patient population, for all too often, the law is twisted and bent to serve a
new purpose. Legislation designed to allow for openness and access can become
the legislation of restraint. It is our duty as both patients and providers to
support the original intent of quality measures while constantly updating the
means.

“O! it is excellent to have a giant’s strength, but it is tyrannous to
use it like a giant.” As our database of measurement grows, we will be
able to apply the principles of evidence-based medicine to large patient
populations, ensuring more appropriate care. But it shall matter most to our
little ones as they reap the benefits of wisdom from those who have gone
before. The reward will be the greatest if whilst helping the multitudes, we
focus on the individual.

Let patients speak. We must encourage every committee, subcommittee, and
hospital board to actively recruit and include patients in every aspect of the
care process from design to implementation to resolution. Nothing about us
without us. From the exam room to the board room. Patients included. Thank you
for listening. It is that important.

MR. SIEGRIST: I am Rick Siegrist, pleased to be here, and I want to thank
Dale and Regina for setting up my presentation, since I am going to be talking
about a new way to look at HCAHPS and patient comments. What do patients really
care about?

Sentiment analysis may be something new to many of you. Susan and I are
going to be splitting the presentation, so we will go through that. As I am
going through this, I am going to be talking about some of the things we have
done at Press Ganey. For those of you that are not that familiar with Press
Ganey, about half the hospitals in the country use us for their surveying. We
receive back about 13 million surveys a year. Some of the results I am going to
be showing you are coming from that experience.

To really set this up, I think it is useful to think about how patients
really look at the healthcare process and how they evaluate things. I think
Dale Carnegie expresses it well. Remember that when dealing with people, you
are not dealing with creatures of logic; you are dealing with creatures of
emotion. That is what comes through in content, that emotion. Then I think the
author and poet Maya Angelou expresses it extremely well when she says,
“I’ve learned that people will forget what you said, people will forget
what you did, but people will never forget how you made them feel.”

This has already been covered, so I am going to go very quickly through this
just to emphasize that patient satisfaction surveys, HCAHPS, really focus on
the quantitative — average scores, relative ranks, trending over time. That is
important, but it is really not getting to the true emotions being expressed by
the patients.

What is the “but” we are talking about here? The “but”
is the 80 percent of the available information provided by patients that is
coming through in their comments really is unstructured data. Yes, you can read
through hundreds of comments, but your mind cannot process that information.
What do you do? You tend to focus on the anecdotes and not see the themes.

The comment databases, complaint databases, Dale was talking about website
reviews, social media sites — there is really a wealth of information that is
not being readily tapped into right now. Fortunately, there is a solution. That
is sentiment analysis, which allows you to gain insights from taking all this
unstructured textual data and turning it into structured data that you can link
with HCAHPS scores, you can link with outcomes, you can link with other things.

It is based in the science of natural language processing, where the
computer can actually interpret human language looking at adjectives, adverbs,
nouns, looking at the linguistic connections, looking at the context, and
really assign that information into meaningful themes and categories so you get
away from the anecdotes.

It can evaluate the strength of the sentiment that is being expressed by the
patient on a scale of minus-5 to plus-5. How strong does somebody feel about
what they are saying based on their word choice and the way they are phrasing
it. And just as you can do with HCAHPS, you can then look at regional and
national norms in terms of sentiment being expressed and look at trends of
sentiment over time.

This is really relatively new, even in private industry. Private industry
has been using this for competitive advantage. Walmart, American Express,
Wendy’s, Marriott Hotels, JetBlue have been seeing this as the holy grail of
how to understand what our customers are saying and how can we turn that to our
advantage. In fact, Wall Street is now analyzing tweets to predict whether
stocks are going to go up or down, and they are doing a pretty good job at
doing that. Isn’t there a better purpose that we can put it to in healthcare?
We think there is.

What we have done at Press Ganey is we have taken all our inpatient comments
for 2011. That is about 4.5 million comments. We are talking pretty big numbers
here. Not all comments have sentiment expressed. That is 3.7 million.
Interestingly, about half the consumers write comments, and they right almost
three comments per survey that they respond to. That is 5 million sentences.
That is our N that we are analyzing.

Dale was talking about some of the common misperceptions or myths. One is
that people only write negative comments. No, the net impact is that the
comments are positive that patients are putting on their surveys. It may be
different when you are looking at a hotel or you are looking at a restaurant,
but people care about healthcare, and they will give you a mixture of comments
based on what the experience really was, is what we have found.

How does this relate to HCAHPS? This is that same information looking at the
overall HCAHPS rating of 0-10 in terms of how you would rate the hospital. This
is comparing the sentiment score. You can see the top is showing sentiment
score higher down below. Pretty striking graph here. It is only the top box,
the nines and the tens, where the comments are positive. When you are looking
at the sixes, sevens, and eights, the comments showing up in the survey are
negative. Gaining insights from why you are getting the negative comments, as
Dale was talking about in terms of how you improve the scores, we think at
Press Ganey that is a way to have an important impact on the scores.

Same kind of picture here when you are looking at the likelihood to
recommend on HCAHPS. The positive come when somebody says I would always
recommend the hospital. Usually, sometimes, never, the comments are
increasingly negative in terms of the sentiment being expressed. I am going to
turn it over to Susan Madden now to talk about the “why” behind the
scores.

MS. MADDEN: Good afternoon. I am Susan Madden, also from Press Ganey. We
have spent the last year to year and a half working on sentiment analysis,
taking comments primarily from inpatient surveys, as Rick was saying, and
taking this natural language processing engine that does semantic analysis,
looks at words in text to put them into categories, but also the context of the
sentence, how words are used, whether there are negations, whether they are
emphasizing words, to really interpret how positive or negative that sentiment
is that is being expressed there.

As Rick was saying, most of this work has been done in retail, hospitality,
and travel industries. People use language very differently in healthcare. They
use different words. They mean different things when they use those words. So
we have spent a lot of time really tuning that engine to pick up the way people
use words in healthcare to extract real meaning from these comments.

We look at it as really giving us a way to get at the “why” behind
those quantitative scores. As Dale was saying, it is very hard to move those
HCAHPS scores. Hospitals are going to have a lot of money riding on that very
soon. As you all know, when you take a survey, do you take the time to write a
comment if you are not feeling particularly strongly about something? You sort
of whip through the survey and say I am finished.

It is only if you really feel strongly about something — you had a really
good experience, you had a terrible experience — that you take the time to
write about it. There is a wealth of experience in these comments, as well as
comments about things that the survey may not be asking you about.

My daughter recently had a baby, and she got a Press Ganey survey, which she
recognized because it came with a South Bend return address. She would never
have recognized it if I had not been working for Press Ganey. She said the
survey asked her about all kinds of things that she really had no interest in.
It was not until the comments that she was able to put in what really mattered
to her during her hospitalization.

What we found is that people with strong feelings do the comments. They
provide additional detail on their experiences and they identify issues not
covered in the survey. That is where I think their real strength is, as another
source of information about what patients care about. We have done all this
work so far in feeding it back to hospitals, not to patients, but it is a
direct conduit into what is important to patients. The only other really good
ways to get at that experience have been talked about earlier today with focus
groups and other kinds of surveying. There are, but this is information that
exists now that we can tap into to get at some of those same issues.

I want to just give you quickly some of the things that we have discovered
as we have worked with all of these millions of comments. When you look at our
pure overall databases of millions and millions of comments, the main thing
that people comment on in their hospital experience is about doctors, nurses,
and other staff at hospitals. They tend to be positive. People will call out
people by name that they have had a good experience with.

Process what happened to them in the hospital gets fewer comments, followed
by place. Then service alerts are things that went wrong, privacy issues, which
are negative. You can see the relationship of the scores.

This tool allows you to dig into the details of what people are talking
about. To show you an example of what kinds of issues people talk about around
place, which people tend to be more negative about, they talk about their room,
their furniture, their food and drink, noise and environmental issues, parking,
outside issues. None of that is surprising.

The most negative things are around furniture, instruments, equipment, which
tends to be around IVs, IV starts, and noise and environment. But with this
tool we can then dig into what exactly is it that is driving those negative
experiences. I will not read. You have these in your handouts, so they give you
some examples of real patient comments.

Around noise and environment the things tend to fall into certain
categories, things around noise in the hallway, people being across from
nurses’ stations and listening to conversations all night long. Problems with
roommates and TV noise and temperature and environmental noise.

One of the things that we are also able to look at is where are these things
happening. You can link this kind of unstructured data with structured data
elements and see what nursing unit it is coming from, what the demographics of
the patient are, what the disease process is, how long they have been in the
hospital, and link these kind of qualitative data elements to those
quantitative ones as well.

Looking at tests and equipment, it was uncomfortable chairs, not being able
to sleep, and painful IV insertions. A full 3 percent of comments are around
IVs, something that is taken absolutely for granted in hospitals. Everybody
gets an IV. It is not a big deal. To the patient it is a big deal. Probably all
of you have been stuck with painful IVs. It is terrible, and nobody pays any
attention to it. But it has a very big impact on people’s experience, and they
write about it, even though most surveys do not have a question around IVs, but
they will take the chance to comment on it.

We had a client that we did some research with who then went even deeper
with this to look at the correlations between various categories and sentiment
ratings. They found that for patients with pain, parking and transportation
issues were a concern, so when they commented about these kind of issues and it
was related to how they were scoring pain management on the HCAHPS survey, not
something that you would automatically come to. But if they had to walk a
distance or they could not get to where they needed to go and they were in
pain, that really impacted their experience.

This particular hospital has a very large sickle cell anemia population, so
they saw call light responsiveness and pain management as an equity issue. They
were not being taken seriously. They were not being treated fairly. Again,
blood draws and IV starts were a particular source of discomfort for many
people, and they impacted that pain score.

We have seen this over and over again. It is the overall rating of care is
impacted not just by nurses and doctors, which tend to be what we focus on, but
by people’s interactions with the food delivery people, with transportation,
with cleaning. Those are the people the patients many times see much more
often.

If a friendly face comes in who cares that they have gotten their food or
that they got what they ordered or just delivers it cheerfully or cleans their
room well and cheerfully, it makes a huge difference. I thought that was very
interesting. Also, scoring the room clean and quiet is also reflective of
temperature and lighting. It is not just clean and quiet that we are getting
from those answers.

A couple of things that we are able to do with the comments is look at how
the volume of comments varies as well as the sentiment ratings. In this case we
are comparing in the top graph comments from surveys that either got a 9/10 top
box quality of care rating versus 7/8, so just a little bit below that. In the
top graph it is looking at the difference in the sentiment rating of those
comments using that natural language processing, which is analyzing the
language being used and assessing how negative or positive it is.

Going from left to right, are comments around discharge, comments around
nurses, around cleanliness, and IV and blood draws. Just that little bit of
difference falling off the top box is a very big difference in what people’s
experience has been. That can provide a window to hospitals as to where to
focus in order to improve the experience for patients and for driving those
HCAHPS scores upwards as an indication of improved care.

The bottom graph just shows how it varies by volume of comments. This
software and technology allows you to look at trends over time. How does the
volume of comments and the rating, the sentiment, and feeling in the comment
change over time as you make interventions?

Just to leave you with a few thoughts that Rick touched on. Patient
satisfaction surveys, the comments there, are really the largest source of
patient experience that exists right now. It is readily available. It is more
than about just room. There are certainly lots of very specific comments about
very specific parts of the experience. But it is also about people’s total
experience in the hospital and what affects them. It is very strong and deep
emotions. These are critical times in people’s lives, so there is a lot of
information in that comment database. It is complex experiences.

We really have found this to be a treasure trove of information that exists
now and does not require new measures, new ways of measuring, new ways of
collecting data, but provides more detailed access into what a patient
experiences and what the important pieces of that are. That is it.

DR. MIDDLETON: Terrific. Thank you very much, one and all. Let us open it up
for discussion and questions.

DR. HORNBROOK: Just a clarification on the healthcare sentiment. Is that
proprietary by Press Ganey?

MR. SIEGRIST: The comments are proprietary to the hospitals. It is not
publicly available outside of the hospital. We share this back with the
hospitals.

DR. HORNBROOK: Do hospitals pay to get the data from you?

MR. SIEGRIST: This is a part of our regular service.

DR. HORNBROOK: Which is subscription to the hospital.

MS. MADDEN: We also analyze a lot of HCAHPS comments. One of the things that
we found on HCAHPS surveys is because there is not the opportunity for
commenting, people write novels. They write long paragraphs full of information
that can be hard to parse apart without some software to help do it.

DR. HORNBROOK: What you didn’t say, of course, is that we have written NLP
programs to process text data in medical records, and it is a big deal. The
intellectual work that went into sentiment is all the upfront work to take the
strings of text and put them into some sort of meaningful feedback and data.
Then once that is done, the marginal cost is low, but you put in a lot of work
up front. To get somebody else to repeat that is going to be a lot of money as
well. It is kind of interesting how NLP, as much as it seems simple, is not.

MR. SIEGRIST: We are actually using software from a third-party vendor, and
then, as Susan mentioned, translating it to be relevant. Healthcare is the
intellectual we have added. There are other companies that do the NLP, and it
has come a long way. It is very complicated to develop it, but it has come a
long way in the last decade or so.

MS. MADDEN: Once you have your rules developed, then you become very
efficient. There is, as you said, upfront work.

DR. TANG: Press Ganey also has quantitative surveys as well. Have you done
that comparison? I assume you have done that comparison between the sentiment
and your quantitative surveys.

MR. SIEGRIST: Yes. We find a very close correspondence. There are a couple
of examples in relation to the HCAHPS, but we have also done that with the
Press Ganey questions. We have done this not just in inpatient, but we have
even done it in terms of physician satisfaction with their organization and
staff satisfaction and also now with medical practice in the emergency
department. We are seeing the same kind of correspondence that sentiment gives
you a good relationship between the quantitative. But as Susan was describing,
we think the quantitative is driven by the sentiment rather than the other way
around.

DR. FITZMAURICE: I’m not sure I understand this, but it really sounds
unique. I write a comment based on something, say, a stay in the hospital. I am
wondering, how do you score a comment as a minus-four as opposed to a
plus-three? Do you take my choices of words and assign each word a score, and
then assign the combination of the words a score, and then it turns into a
number? Or is it that somewhere in the middle there is professional opinion
about what it means?

MR. SIEGRIST: It is the former, where words are quantified and words
expressed together and words expressed in a certain context and some of the
linguistic connections. It is a real science of natural language processing. It
has improved to the point where it is almost as good as if you had an
individual reading through that and scoring the comments.

DR. FITZMAURICE: Do you have to know what part of the country I am saying
this in?

MR. SIEGRIST: Actually, we found that has not been as big of an issue as we
thought.

DR. MIDDLETON: Just to follow up on Mike’s line of questioning, not to turn
this into an NLP technical seminar, certainly what NLP tools are out there —
and we have done a lot of work in this in my lab — really do have a variety of
different grammars and approaches and methods to the interpretation of strings
of texts.

My question is twofold. Simply can you just address the issue of how NLP is
different when it is only comments as opposed to more narrative prose, if that
is an issue at all? Secondly, how do you validate your assessments? Have you
done independent validations beyond the Press Ganey data looking at the
validation of the instrument itself on the comments?

MR. SIEGRIST: Let me deal with the second one first. About half of the
submissions for CAHPS are from Press Ganey, so we are basically representative
of that database. I would say that is pretty good validation. We are not using
a biased sample, if you will, of just our clients. What was the first part of
the question?

DR. MIDDLETON: Really whether or not NLP on sub-sentence fragments is the
same as NLP on full sentences or narrative prose.

MR. SIEGRIST: A lot of these comments are narratives. They will go on for
multiple sentences. The NLP allows it to be broken down into individual
sentences. Then the sentences are combined to get an overall score for that
comment, but also a sentence my go into multiple themes. A sentence may be
mentioning the nurses and mentioning the doctors. That goes into two separate
categories.

DR. MIDDLETON: Let me go back to the validation question for a second
because I do not think he quite got it, or maybe I did not say it well. Rather
than validating the sample of your surveys against all HCAHPS or CAHPS surveys,
what validation has been done on the NLP process itself on comments? Has there
been an independent assessment of simply how well does the NLP tool address
what people think they are saying their comments, regardless of what they said
in the survey data?

MR. SIEGRIST: One of the things we have done historically with our comments
is we have commenters who type them up and rate them as positive, negative,
neutral, or next. We have shown a close correspondence between that and what
the sentiment analysis is showing through the NLP.

MS. MADDEN: There are also methods that you use as you are tweaking the
rules in the natural language processing engine that will pull out a sample of
comments, and you go through and make sure that they are rated properly, that
it makes sense. Then you can tweak. There is a whole dictionary of about 1,100
words in particular. Then you add phrases to that that have certain ratings to
them. That is where a lot of upfront work went on because words may be positive
or negative if you are talking about going to buy a TV, and they are very
different. We have done a lot of crosschecking back and forth and not only
matching it to the human rating, but also internally in the processing.

DR. MIDDLETON: My only thought would be sort of a gold standard evaluation
might be to take a subset of people — it could even be a small number,
depending upon what kind of power you are really looking for — ask them to
give comments on something, really any subject, use your tool, and then ask
them does your sentiment analysis actually reflect what they were trying to
say.

DR. TANG: You made the comment that the comments drive the scores, not the
other way around. How do you know that?

MR. SIEGRIST: That is a hypothesis. It is really looking at how we all react
to things. Do you start by saying I am going to give this hospital a five, or
do you start by thinking about your experience, and then you give a five, and
the experience is what you put in the comments? The comment is really, in that
sense, driving the ultimate evaluation you are making.

DR. MIDDLETON: In the process are the comments filled out first or is the
survey filled out first?

MR. SIEGRIST: It is done together as the patient is filling out the survey.

DR. MIDDLETON: But I would never actually comment before answering some
questions. Is that right?

MS. MADDEN: Right. Rick is talking about mentally when you are thinking
about what your experience was, your feelings about the experience are what is
driving how you are — you have to then translate that into a quantitative
score, which can be a frustrating experience if you feel like —

DR. MIDDLETON: This is fun because the cognitive science literature might
suggest that scoring first could anchor a person in a particular orientation
that might then bias to a less effect their comments. It might be interesting
to actually experiment with some of that ordering of questions first, comments
second, and all that.

MR. MADDEN: I am not trying to push Press Ganey, but one of the design
features of their surveys is that at the end of each section they put a comment
section in there. They call it a mental speed bump to really slow people down
to think about what they just said quantitatively. So is there something else
about this experience that you want to comment on? It sort of slows people down
rather than just going (demonstrates motion and sound), threes. There is that.
That is something that they have explored.

MR. SHALLER: Can I just add a comment on this whole issue? Because where I
left off was the need to figure out better ways to take comments and figure out
how to report them along with quantitative scores. I do think we have room to
figure out what is the best way to elicit comments. The CAHPS team is actually
going to be thinking about that more systematically now. Where do you insert
the open-ended questions and how do you prompt them?

DR. MIDDLETON: If we were going to bet the farm on sentiment analysis, it
might be useful to go through that level of rigor and validation. I have a
question for Regina. Among all of Shakespeare’s characters, who was the best
patient?

MS. HOLLIDAY: That is a challenge. In my opinion, that would be in the
Taming of the Shrew, the shrew.

DR. TANG: Why?

MS. HOLLIDAY: Because she asks questions throughout. One of the things is
when you deal with patient communications, there is a tendency to bow to
authority and not ask questions. That can be really dangerous for individuals.
We have got to teach patients to ask questions. Another thing that I kept
hearing on the survey results is they filled them out, and I saw these kind of
motions. Are we getting most of our surveys through the mail still? That
concerns me. As we go deeper and deeper into HITECH and what is going on with
meaningful use, the fact that we have got a population there bored out of their
mind and wanting something to do, and we are not utilizing that time to do
surveys, when they actually would write full and in-depth comments.

DR. TANG: But that is before they see the doc.

MS. HOLLIDAY: — wouldn’t it make the most sense to give it at triage,
during care, and afterwards? You want it during every stage of the process if
you really want to get a true picture of care.

DR. MIDDLETON: You could almost imagine the patient tweeting their
experience the entire hospital stay, but you gather the tweets.

MS. WARREN: My husband blogged his to keep the family informed across the
country.

DR. MIDDLETON: It is probably therapeutic.

DR. QUINN: I was going to say that the comments are a huge source of data
today, but the way that the data is just proliferating in really unstructured
ways, the more that other industries are taking advantage of this data to
figure out what products to promote or which ones not to promote or what to do
and to understand experiences with their business. I tweeted something about a
certain delivery service, and they contacted me. It was too late, of course,
but it was being monitored in that way. I have to believe that they are taking
that information in aggregate and using it in structured ways.

I think the real balance here is how do we complement or how do we balance
investment in structured-type data surveys, traditional ones, along with the
new frontier and these new sources of data? How willing are customers, in this
case healthcare organizations, to use that data to make actual change? Is it
that right mix? What is your impression?

MS. MADDEN: Talking about hospitals and HCAHPS scores, in particular, there
is a lot of frustration with how difficult it is to move those scores. With
money riding on it, that becomes more of an issue. This really does help to
pinpoint where the problems are and what they are specifically and some of the
unusual combinations of things that patients are reacting to. I think there is
some specificity, some detail, in there that is difficult to get to through
straight quantitative scores.

But I want to talk to something that Dale had talked about, because we all
are going to Yelp and TripAdvisor when we are trying to plan something. You can
read 15 positive comments, and you get to that one negative and you think maybe
this place is not so great after all. How do you balance what is in those
comments and that this kind of engine allows you to put all of that together
and see where it falls and how it trends over time? The voice of the patient is
so critical, but it is hard to deal with a lot of voices of patients and know
what it is that they are concerned about. The quantitative gives you a certain
amount, and the qualitative gives you a complementary piece.

DR. COHEN: Again, how do we get this CAHPS information or the sentiment
scores back to folks so that they can use this information and evaluate it?

MS. SIEGRIST: That is a difficult question. I think that would be something
that if there were a mandate like the mandate for HCAHPS, that could encourage
the sharing of this information. But now this is proprietary to the hospitals
that are getting this in on their surveys. I will turn that back to you all.

DR. WARREN: I just wanted to comment on the notion that Bruce brought up. It
seems to me that we are assuming that if people had data, they would change
their behavior. There are no guarantees with data gets to behavior changes. It
is also the whole notion — and I have been trying to put it in my head and I
do not have it all there yet, but a discussion that I was part of recently at a
conference where they debated whether or not patient-generated data was any
good. The discussion was by providers with kind of the implicit understand of
if it was not gathered by me, it cannot be right and not useful for me to use
as I make diagnoses and decisions.

I sit here and I keep looking at your slides. The paintings are dramatic.
Somewhere there needs to be a change in the way we value the key partner in
healthcare, which is the patient and his/her family. Until some of that mindset
can be changed, we can measure everything perfectly, and if people do not want
to take that data or benchmark against it, then nothing happens.

The one painting you did that keeps disturbing me — and I will figure it
out in a minute — is the one with the brick wall. It is on page seven. If you
notice, this is painted on a brick wall, which has a lot of meanings. But there
is a little sliver of a person looking through. I have a feeling a lot of our
patients feel like this.

MS. HOLLIDAY: That is a picture of my eldest son who has autism while my
husband was hospitalized and how involved he felt in the care process.

DR. TANG: You said this was painted by your son?

MS. HOLLIDAY: No, it was a painting of my son. That is my eldest son and how
he felt about his father’s care.

DR. WARREN: We have a whole thing called art therapy. Wouldn’t it be
interesting for patients to draw their experiences for us when they are with
us?

MS. HOLLIDAY: It helps a great deal. I was at an event done by Avatar where
they had every provider paint their patient experiences. It was really amazing.
It was like your personal things that stuck in your head. The entire room
turned silent and holy while everybody was painting either themselves or their
mom or that one patient they will never forget and what happened that day. Some
people looked immature because they are not trained artist, but that did not
matter. The power was there. They were showing and emoting all that power.
There is no clinical spot to put that, but it is part of the conversation.

DR. MIDDLETON: In addition to Regina’s work, you may know the author Bernie
Siegel. He was a Yale oncologist who did a book. I cannot remember the title,
but it has to do with all of that kind of emotional processing. He used the
method of painting, drawing, and other art forms to allow patients to express
and providers. It is a profound piece of work.

We need to break, unfortunately. I am sure we can go on for quite a while.
This was a terrific panel. Thank you very much.

(Break)

DR. TANG: This final panel has to do with patient preference in making
selections related to health. We are delighted to have Sarah Thomas, David
Stumpf, and Rob Krughoff to talk to us. We are starting with Sarah.

Agenda Item: Use of Patient Preference Measures in
Selection of Insurance Coverage, Health Providers, and Treatment
Options

MS. THOMAS: Thank you for having me here. I saw looking through some of your
papers that you had my colleague Sarah Scholle here. She is the real gearhead
in our operation. I am going to actually focus us on a very policy-oriented
presentation. We have been doing a lot of work at NCQA thinking about
exchanges. It is a great opportunity to think about using consumers and
measures together. Our view is that this has the great potential to really move
forward a value agenda. I will give you a sense of what I mean by that.

My boss Peggy O’Kane, when she was tracking on the health reform
legislation, really saw exchanges which are marketplaces. It is a real change
in the way we think about the way that consumers and health insurance would
interact with each other. I think there was a lot of good discussion this
morning about how the conventional chooser of health insurance really is the
employer, and then consumers choose from an increasingly small number of
options from that.

Under exchanges, the vision is for multiple health insurance plans. We saw a
lot of examples, and there is a great book called “Nudge,” which
actually presented the problem of the Part D problem, where there were so many
choices and so much information put in front of people that they really became
paralyzed. The behavioral economists believe that the way to cut through that
is through organizing the architecture of choice.

Our view — and I think Robert is going to show you some examples of how he
sees this implemented — is that you put quality and cost, both the premium and
the cost-sharing, all three things together, and then you just filter the
information. That nudges people in the direction of the highest value options,
even if they have not had to fully grasp all the concepts and all the details
in the metrics and so on, so that you create a way of choosing that sort of
points people in the right direction in a good way.

One of my colleagues describes this idea of choice architecture in a very
helpful way, which I think is the way that a school cafeteria organizes the
food that is for sale, so that if you put the pizza and the chicken nuggets
first in line and the apples and the carrot sticks at the end or out of reach,
it is obvious the kids will go for the first food they see first. But if you
actually put the carrot sticks and the apples first in line, the kids will
actually buy more of it, and it is better for them. They do not have to
understand it, and lots of people in retail know this very well, the way that
they organize grocery stores for right or wrong. We should really use these
ideas to organize health insurance choices in exchanges.

The idea of philanthropical nudging also goes in the direction of the way
that we would hope that health plans would organize their benefits in
exchanges. Yes, there is the gold and silver, but we would love to see less in
the deductible and more that points people towards preventive services, care
that has an evidence base, and away from through higher cost-services that have
limited value overall. We would love to see patient activation tools like
shared decision-making tools integrated into the benefit design as well.

I know that the world that we live in is one where people want unlimited
choice, but we are beginning to see examples in Massachusetts where there are
carriers that are beginning to be interested in offering a more limited network
product. We would love to see that as an opportunity to actually choose
higher-value providers, where in the Massachusetts example the hospitals that
are actually good quality and low-cost into these insurance products offered in
exchanges.

Maybe we are cockeyed optimists, but we would really like to see health
plans not being the averagers, but really take on the role of change agents in
the markets. You do see some examples of that. It is not as widespread as we
would like, but we would like exchanges to point us in that direction.

There are three key areas of quality in the reform legislation. One is
around qualification of health plans to participate in exchange. There is a
series of requirements there. There is a mandate for exchanges themselves to
collect information on quality, including plans’ activities around improving
quality through what they do with benefit design and what they do with
incentives for providers. There is an opportunity for plans to reward high
performance of different providers through that initiative.

HHS is really partnering with states to try to move them along in all these
exchange functions. To be honest, we have presented our vision of how we want
exchanges to work, and I think a lot of states think it sounds great, but they
are still struggling with the very basic elements. A lot of insurance
departments, in particular, are used to evaluating quality through looking at
complaint data. The idea that you would actually collect quality measures and
create these web portals and point people in the right direction of good plans
feels a little scary to them.

Even states that have had a long history of report cards and quality
reporting are pretty overwhelmed, the ones that are really trying to pull
exchanges together. It is early days. I see this as an evolution. I am thinking
we have phase one where we built a framework and we built some baby steps. Then
we have phase two where we really see the full-blown realization of this
vision.

Here is the timeline for implementation. The bottom line is 2014. That is
when this is supposed to lift off. There are a lot of steps in between and a
lot of money going to the building.

I thought it would be useful to remind everybody who we expect to be in
exchanges. The Kaiser Family Foundation is a relatively older population, less
educated, lower income, more racially diverse. I think that the challenges of
health insurance literacy are going to be even greater than in an insured
population or in a Medicaid or Medicare population. Many of these people will
never have had insurance before, making these translation challenges even more
difficult.

I saw a very interesting presentation from one of the health plans in the
Massachusetts Connector that showed enormous amounts of unmet need in the very
first nine months of operation of the connector, just off-the-charts
utilization. Everybody has heard a lot about the stories of people having track
challenges, getting access to providers. For that reason, I think we have to be
very careful about measuring clinical quality as people are in this first year
of really sorting themselves out with healthcare.

We are recommending that states and the federal government start with a
national starter set, a fairly limited set, recognizing this is a startup
period. We think it would be optimal to have the same set of quality standards
or at least a basic package for everybody to use the same quality standards. We
can add to it over time, add state-level measures to it, but it would be really
useful to be able to compare the performance of exchanges themselves from state
to state to see whether the federally facilitated exchanges do a better job
than the state-based exchanges and so on.

Ideally, we want to align these with other initiatives so we can make the
comparisons, so we can reduce burden, so we can really get people to focus on
the same goals. We would like to start with measures that are in wide use. We
can come up with new and experimental measures over time.

Use the same measures for HMOs and PPOs, and that is a pretty important
point. There has been in some states not much of an experience of reporting for
PPOs. We know they can do it. They do it with Medicare Advantage, but the
markets in some states have just not required PPOs to report, so this is a new
area for them.

I think it makes sense for us to start with CAHPS. It may be that we need to
add measures over time. I thought the discussion about the comments was really
fascinating. I think that there is a lot of interest in creating these web
portals with a comment like a Travelocity kind of approach. Some of the
research that was discussed earlier gives me a little bit of pause.

Consumers and quality — what do we know? We have done work on report cards
with many states. We have a report card of our own on our website. I will just
sort of take you through some of the principles we have. The steps are
determining how to rate plans integrated into a decision support tool, and we
could not agree more with the need to actually put this in front of real
consumers, not consumer advocates, not researchers, not us as consumers, but
actual real-life consumers to refine and improve it.

There are a number of different ways of displaying rankings. You definitely
do not want to put detailed measure-by-measure results. Aggregating measures is
really important. Ranking — we have had a couple of years experience working
with Consumer Reports using rankings. It can be very easy to understand rank
order. Star rating can work, too, as well as grades, A, B, C, D. Symbols and
statements can also work, too, but definitely you do not want
measure-by-measure results.

Some examples of decision support. We sort of organized these into three
categories, with the basic being the report cards I think many of us have seen.
Advanced would be more of a personal worksheet that really allows people to
mechanically figure it out.

We also saw an example of a prototype from Wisconsin that was built before
the change in administration there where individuals could actually go into the
website and put their own weights associated with the different quality
attributes. That was really a very interesting approach to really getting
people to engage in the weighting of these different attributes. I do not think
it is up and running. It is for sale. That was a very exciting version.

It is important to define quality, explain its meaning, provide a framework
for it. You need to have short definitions. Filtering, how you sort the
information, is really important, allowing customers to customize the process
as well, keeping the information at a high level, and then allow those people
who are really engaged to drill down.

We have seen examples for some of the Medicaid brokers where at certain
points where people might get confused they can actually pick up the phone and
call somebody or pull in little dialogue boxes that allow greater explanation
of some of the concepts. Those seem like good ideas. Many people need different
ways of getting information, so we need to build them that way. The information
must be clear, not cluttered. These are just good common sense for any kind of
web presentation.

It needs to be able to be moved through quickly. People have limited
patience for using these tools. Even though you want to provide options to
drill down, I think it is really important that you realize people will lose
interest and abandon the choice if you put too much information.

It is very important to include human beings in this process, including not
only the navigators that were mentioned earlier but brokers and agents. In many
states they will still be very involved in the decision-making process. We
would love to see brokers and agents more interested in learning how to use
these quality metrics and help people integrate that into their decisions.

I will skip over Consumer Reports. I will just make one point from Lynn’s
presentation earlier. Most of the focus is on the cost issues, and we
understand that, but there is some interest in the quality providers. I am
holding onto that with a little bit of a thread of hope that there will be some
interest in quality measures.

There is some interest in states in measures specifically for providers, so
that perhaps there is a way for people to come into the web portal, find out
about the quality of a provider, and then pick a plan through that approach,
which I think is really interesting. I am not sure the state of the art would
support that yet. The problem with quality measurement for providers — and
Joyce mentioned this earlier — is small numbers. We often do not have enough
data on any individual provider to necessarily report on enough measures to be
valid. They will let you know that.

I thought I would just close with a couple of results from Judy Hibbard’s
most recent work where she is putting cost and quality information in front of
consumers to help support choice. These results are from an employed
population, so a little bit more literate probably than your exchange
population.

She took on this question of consumers thinking that more is better. The
idea here is if you put information on quality together with cost, are people
more comfortable choosing a lower-cost provider? She did a number of different
experiments with different displays of cost in different symbols.

She did find that how cost is portrayed makes a difference. Using the Zagat
rating approach of dollar signs is the least effective, which is interesting.
If you do pair a strong quality signal with cost information, people are more
comfortable choosing a low-cost provider and they are more comfortable with
their decision. Even if they do not entirely understand the detailed metrics
around the quality, if you put them together with cost, it can be more powerful
than leaving it out.

Another thing that I thought was especially interesting about this work is
that you do not necessarily need to provide detailed explanations of all these
different measures, that concepts like careful with your health care dollars,
appropriate MRI use, high-value hospital can be more powerful than the detailed
and specific measures that scientists would like to see. That concludes my
presentation. I look forward to the other presentations and discussion.

DR. STUMPF: I am Dave Stumpf. It is a pleasure to be here and be on a
panelist of people in organizations I have worked with in other contexts
before. The question that I was asked to address was sort of the general
statement about the roadmap here. I have kind of translated this statement,
which I will not read because it is in the handout, into several bullet points
that I want to address.

First of all, the fact that I think the committee is on the right track. I
want to suggest some mechanisms for putting the consumer at the center of
healthcare. I also was asked to use eMeasure as an example of how one might
incorporate some rules into mechanisms of putting the patient in the center.

I kind of changed the meaning of this other bullet, which was in the
original statement. Instead of what they are saying, I am going to try to
answer the question of how might we know what is meaningful to individual
patients. That is a twist on this, but I think it is an important one. We have
to figure this out.

To start with, the kudos part here. The committee has aligned itself with a
lot of other scenarios. I have been doing a lot of work with the NQF and their
framework around coordination of care. That really laid out several core
principles, among which were that we need a consumer healthcare home. That term
is used advisedly, not medical home.

They also would like to have a proactive plan of care. Among the central
principles there — and I will go into this later — is that one of the
accountable entities in that plan is the patient or the person or consumer
themselves. They also recognize that information technology is going to be very
important in juggling all of the aspects of this.

That NQF framework was picked up by the IHE who is still in a draft form
what they are calling Person-Center Coordination Plan. Of course, although not
as strongly as I would like to see it, it has been incorporated in Meaningful
Use 1 and 2. Then the National Priorities Partnership put patient centeredness
as one of its core principles.

This agenda is really pretty complex. What I would like to do in this is
address first some of the gaps that exist in really getting to
person-centeredness. I am working right now with the Illinois Health
Information Exchange, and in some other settings that I have worked, too, it is
very clear that even in the very best of systems, we get to about a 3-sigma
level of identifying individuals.

This is unacceptable, especially when you are starting to move information
between venues, that is of a personal nature. We cannot tolerate that level. It
is not an option to not have a unique patient identifier. HIPAA mandates one,
but we have not been able to implement it because Congress has not appropriated
and will not appropriate the funds. It really is up to the private sector to
take care of this. We are having some big discussions with this right now in
Illinois about addressing this issue, and I have a solution to suggest to you.

The diversity of people is really a wonderful aspect of the American
experience. We are blessed by the diversity, and we need not only to
accommodate it but also to promote and capitalize upon our diversity.

Diversity is very complex, as we have heard a lot of discussions about
today, but computers and computation — complexity is what they are designed to
do. I see great opportunity to expand our repertoire of applications and
promote diversity by using technology. I will discuss some of how this might
happen as we move through this.

The other aspect of diversity is that consumers use a lot of resources that
are not typically within the sphere of healthcare influence. It is going to be
very important going forward that our solutions be much more generic in nature
so they can accommodate and incorporate all of that diverse information, some
of which has been discussed today.

This also, by the way, has been part of the NQF deliberations on
coordination of care. Those of us on that workgroup call this Domain 6. There
were five domains that were actually outlined in there. Some of us wanted to
name six so that we can include things like going out to the houses of
asthmatic patients and cleaning their HVAC or buying vacuum cleaners for them
and things like this that make huge differences in healthcare.

Here is one potential solution for the patient identifier problem. VUHID is
an ASTM standard that has been favorably reviewed by the few high-quality
organizations. It is now being implemented in the Southern California/San Diego
area, HIE, by incorporating it into the initiate products. The VA has already
used this. It saved $8 billion in its implementation.

VUHID has a number of advantages. One of them is it allows you to actually
brand your card with your organization so that it puts your organization in
front. But it also supports classes, which can be used for privacy,
preferences. You can have a de-identifying class that will help you with
research and other things.

There are really two types of VUHIDs. One of those is called the open, or
OVID, and the other private classes. The concept here is much like the fact
that you may have more than one credit card in your wallet for different
purposes. You may have different identifiers used for different purposes.

The VUHID itself does not know whose identity is associated with a number.
That is the responsibility of an HIE or an aggregating entity. What VUHID does
is they know where they were issued, and they then know where they were used,
so that they can be used as the pointer to find the data, whereas the
organizations that are actually using them manage the other issues.

There is a lot of growth potential for these identifiers. They are going to
be absolutely essential for coordination of care. You cannot do it without this
across venues. It will allow us to do remote monitoring and body area networks.
It is also going to be — and this is where we focused today — on dealing with
patient preferences and also in tracking their educational components.

I am going to talk a bit about preferences later, but I do want to point out
that education is also a subject that is getting quite a bit of attention in
coding and getting to computable content. MedBiquitous is an organization that
adopted learning object metadata to the healthcare environment with HLOM. This
is now a part of continuing medical education for physicians. It can also be
used for consumer education as well.

The question comes up as how are we going to get measures that are
meaningful? I would submit that getting to this is going to involve not making
this paternalistic, but really to do define meaningfulness through the
preferences that patients express. If it is not meaningful to the patient, then
it is not what we want to be measuring, I would submit. How do we get to what
is meaningful to patients? That is really by them being able to express their
preferences.

Preferences can be registered and dynamically managed. In fact, they have to
be dynamically managed. They need to be computable, and they need to be
interoperable with other capabilities that we have. We also have to tie these
preferences to privacy and security issues because some preferences have very
sensitive information behind them. I prefer that you not share my HIV data, et
cetera, would be a good example of that.

Then we also have to be able to build systems which consume these
preferences and then allow us to measure their use. Those measurements have to
be actionable. What I am proposing here is that we have a service in the
service-oriented architecture sense that manages preferences. An example of
this would be a Hippocratic database. I will give you an illustration of what I
mean by that. The preferences really have to be stand-alone. They have to be
first-class. They have to be queryable by multiple applications.

The preferences need to incorporate concepts such as beliefs, desires, and
intentions. That sounds a bit soft and fuzzy, and it can be, but this is really
defining a school of artificial intelligence called BDI and the use of
intelligent agents that can consume beliefs, desires, and intentions to create
care plans and do other things.

One of the more interesting uses I am aware of is programming the Mars
Rover. We know how successful that was because it used this artificial
intelligence to navigate using this belief about the Mars system, its desires
to move from one crater to another and very specific intentions to get there.
This can be used in healthcare as well.

It also obviously has to incorporate a lot of the things that were discussed
today, the learning styles, the motivators, the social milieu that people exist
in. It needs to be linked to other key things like a knowledge library where we
can compute the semantic interoperability and to educational contents such as I
just described. We also have to have encoded policies so that they can talk to
these preferences. I will get back to that in just a minute.

I was asked to illustrate this maybe with an example of eMeasure. eMeausure
is an HL-7-structured document standard which allows one to encode rules, in
this case evidence-based medicine rules. There are a couple of strategies that
are used. One is there is intrinsic information inside the rules. This is
defined by standards, HL-7, reference, information model. I will not read all
those to you. Then there are other things that are incorporated into the rules
by reference so that we identify patients with numbers and doctors with numbers
and we link to taxonomies so that diagnoses would be defined by a list of
SNOMED codes, as an example.

At the NQF we are now looking at the future of eMeasures and looking at
other things that could be incorporated into the eMeasures. Most likely, these
are going to be done by reference. One of them has already been decided upon,
which is the first one on this list, which is educational material, that when a
rule triggers, it is very common that you are going to need educational
material to back up what you are showing to a provider or a patient.

The initial iteration of this was this is going to be patient education, but
it was also recognized that physicians needed education as well and nurses and
other providers. One of the problems we have today is the long time span
between discovery and moving stuff to the bedside.

Having educational material incorporated into rules will allow one to
trigger that educational tweak at the time of the clinical encounter. This is
how clinicians learn. They learn in context. They are not going to read the
article while they are with the patient, but they can put it in their provider
profile and go back and read it later. That is how I think we are going to get
to them and get them to learn. The patient education here obviously also needs
to consider a lot of the things talked about — literacy, learning styles, et
cetera.

One of the things that becomes apparent when you look at this schema is the
importance of workflows. My belief is that this is undervalued in our current
iterations of HIT. Another one was looking at this is quality is a journey; it
is not a single event. Patient-centered care is also a journey. There are some
ways to address this that I would like to review with you.

This is a conglomeration of the NQF coordination of care model and the IHE
framework for coordination of care. As I have listened to much of the
discussion today, if you listen carefully now, I think you are going to see
that many of the questions and issues that were brought up today can be
addressed by this model. It has several boxes that I would like to briefly
discuss.

One of them is the patient characteristics. This is the usual stuff of
healthcare problem lists, medication lists, and so on, but it also has the
other things that we have talked about today like literacy, beliefs and
desires, support systems, et cetera. I view this as being created in a
templated format which IHE has developed at the point of care with the patient
and a provider, most likely a nurse clinician initially, reviewed by a
physician, who put this together.

Having defined those patient characteristics, we can do a series of
analytics. If you have a new diagnosis of diabetes appearing on the problem
list, the rules should begin to population a list of proposed tasks — if we
are going to do hemoglobin A1c, foot exams, et cetera.

That brings us to the task. The tasks — this is part of the NQF framework
— have some defining content. They have what is the actual task going to be,
but they have an accountable entity. That entity could be a patient. It could
be a provider. It could be an electronic system that is going to track and
monitor something. It could be a lot of things. It also has an outcome. Those
are the three key elements.

It is also important, as this is trying to illustrate here, is that tasks
can be hierarchical and sequential. If the goal is to get a medication into a
patient, that involves an electronic cue, a formulary, a physician
prescription, pharmacy dispensing it, patient picking it up. It is a whole
journey of getting that medicine into the patient. This kind of rendition would
allow that to happen. I will also point out that the quality data model that
NQF is working now on version three is really a definition of task, so that
these tasks can be represented as QDM elements.

When you have these two ingredients, you can address a whole new payment
model — this is one of the other things I was asked to address — because you
now have a contemporary description of the patient and you have the specific
tasks that are being proposed for the care of that patient. Unlike our current
analytics, the predicting costs, which are usually retrospective-looking and
have about a 30 percent predictive value of what it is going to cost, this
gives you much better ways of monetizing the cost of that care.

I think if this is incentivized with a risk-adjusted payment, you will find
that physicians and nurses are now motivated to go out and find the difficult
patients, to document their complexity, and get paid more for it, instead of
what is happening today, which is to cherry-pick the easy patients. This
provides a real upfront incentive for providers to go out and document and
actually recruit those patients.

Once you have this, you then have to manage it. That gets into the task
management node here. From a computer standpoint it is a pretty simple
workflow. You notify the accountable entity. You provide them with the data
that is appropriate for the task that they are being asked to do. Then you ask
them to report back and deliver on a certain outcome. The report-back updates
the task, but it also updates the person-centered coordination plan. This is a
contemporary dynamic document.

The outcomes can also be rewarded, so that this is the backend incentive. We
have heard discussions today about how our outcomes are not really what the
patients want. This gets you to that issue of the outcomes being what the
patient wants. They also provide a mechanism, which I will talk about on the
next slide, about how payers can get to a different level than they are right
now, which is just a very limited set of outcomes.

There is another issue when you get to these workflows, which is the use of
time-driven activity-based costing, which I am not going to talk about. But I
would recommend to you an article to you in the Harvard Business Review back in
September which talked about time-driven activity-based costing. It is by
Kaplan and Michael Porter, some names you might have heard of. It also has the
potential of really changing the way we reimburse things.

Here is what I mean with a specific example of an engagement strategy. On
that last drawing you saw beliefs and desires on one side of the equation, the
patient characteristics. The problem with most disease management programs is
those do not get translated into really specific intentions, and there is a
breakdown there.

We need to have systems that accommodate where people are at in their
learning curve. If you do not believe that smoking causes cancer and all sorts
of other bad things, you are at a whole different place, and the intervention
for you is going to be different than the intervention for somebody who
believes smoking is bad, they have a desire to quit, but they do not have a
plan.

Those are different kinds of interventions. If you can document where people
are at with their beliefs, desires, and intentions, you have different goals.
The goal is to move the needle and not to get them to quit smoking. It may be
to change their worldview. Those things should be rewarded. There are, as I
mentioned, informatics methods to deal with that. The interventions in that
case are reflecting the person’s place where they are at. It should also
reflect their other characteristics, as I have listed here.

The other thing you may notice with this model is there are a lot of things
to measure. This becomes a situation where you can measure the workflow itself.
I realize there are limitations to process measures, but when you are trying to
do coordination of care, we really need to know the process measures. Was the
accountable entity communicated with? Did they receive the message? Did they
deliver the goods, et cetera.

But this model also does incorporate outcome metrics as well, so that you
get the best of both worlds in this particular situation. Because it is
workflow-oriented, you can see where the workflow defects exist. That is going
to be actionable information about where you need to be looking to try to solve
the problem.

Obviously, the patient here is an accountable entity. What they should be
giving is feedback. They should be telling us how the system worked for us.
They should be accomplishing certain tasks themselves like attending
educational sessions, et cetera. They are right in the center of this kind of a
model.

A lot of this revolves around being able to manage preferences. I introduced
the concept of a Hippocratic database before. Fundamentally, there are really
four things you have to juggle here. You first of all have to be able to really
accurately and uniquely identify a patient, as we talked about early on, and
you need to be able to manage classes of preferences. If this is sexual history
information or psychiatric history, et cetera, that may need to be managed with
a separate identifier.

You then need to have policies which are encoded. Without going into detail,
there are ways of doing this. The problem is there are several ways. Healthcare
has not decided on which one it is going to use yet, but we need to. We need to
get a method of encoding policies, whether it is XML or some legal XML, et
cetera.

Then the clinical data itself is there. PCAST recommended to us that we tag
the clinical data with preferences. That is a flawed model because data is
everywhere. You do not want to have to go back and find and retag every piece
of clinical data every time a patient changes their mind, which they are going
to. They do it all the time.

You have to manage the preferences and privacy concerns, whatever other
rules you want to bring to the table — the patients have to manage those in a
freestanding Hippocratic database kind of structure. That is the nature of how
I think you can incorporate preferences. These will be preferences that are
meaningful to the patient.

This is a bit of a disclosure. I am working with the top three organizations
on this list. VUHID is an organization that can dispense VUHID IDs. There are
companies that have solutions for what I have described. This is not just
pulling it out of the sky for you. By the way, the NQF did not invent that
model off the top of their head either. The NQF models are always developed
based on best practices, so they are good examples of how people have used
accountable entities and service agreements and so on to change healthcare.
That is it. Thank you.

MR. KRUGHOFF: Thanks for the opportunity to talk here. I had a couple of
general reflections on things that were said earlier today, if that is all
right, and then I had a little presentation I was going through. But then some
people suggested I not do that presentation, so this will be kind of a mix and
match, just a couple of things that I wanted to mention that just came to my
mind today that I did not think were fully aired out.

One is we are talking very much about trying to reach patients directly with
information to get them to use quality information. I think it is very
important to stress how important it is to reach doctors with this information,
make it credible to doctors, because, in fact, doctors are making most of the
decisions for patients. I do not think we give quite enough attention to what
the basis is for doctors making their referral decisions, their hospitalization
decisions, whatever. I think that is a big deal.

First of all, it causes them to refer well. Of course, it is not a bad idea
to have doctors see the quality information and believe in it and trust it in
terms of their own quality improvement efforts as well. It is quality
information about them and about all the others in the system.

The second thing I wanted to mention is that we have not talked much about
physician compare here. I was going to have my whole talk here be about
physicians. I think it is quite striking how disappointing the physician
compare is so far. Law calls for it, et cetera. CMS has an awful lot of work to
do, so I forgive it, but I do think we need to focus on it. I was a member of
the MAP Clinician Workgroup, and not much talk about physician compare there.

The other thing is I just want to underline what Lynn and several others
have said about testing, that in all of these measurement programs and
databases and reporting programs we have to test them with consumers and see if
they understand them, if they mean anything to them, if they are compelling to
them, et cetera. Those are just some general observations.

There are two things that I diverted my talk to. One is I want to talk more
extensively about patient experience, getting patient experience data on
doctors. The other is I want to sort of give an example of our health plan
comparison tool, which jumps off from Lynn Quincy’s wisdom and picks up on your
thoughts and wisdom as well as to what is needed in a comparison tool. I will
sort of show you what we have developed as a tool for people.

Having said that, I am going to go through a few parts of my basic
presentation. The first thing I want to do is just sort of make us not feel so
guilty about the fact that nobody is paying attention to measures, because we
have certainly found over the years that consumers often do not use measures as
a basis for decisions that we think they should.

I do not know whether you know what I do, but one of the things I do is
publish CHECKBOOK Magazine, which rates everything from plumbers to auto repair
shops to doctors to whatever. Our biggest competitor for the consumer’s mind is
suggestion from a friend or family member, however ill found, just businesses
they pass by or a doctor’s office they walk by, something like that, signs they
see, all kinds of things that we would not wish they would use as their basis
for decisions. This is true even of our own subscribers who have paid for our
information and really have something much better to use. That is just a little
bit of context.

I also thought I would just give you a little information on what we are
actually seeing consumers are interested in, because I think that is something
that is kind of pertinent here. We are out to set our priorities, to some
extent, in what kinds of things we try to measure and report on.

You will notice here that this is just the health-related topics that we
cover at checkbook.org. It is just for a limited period of time, but just where
people went, what topics they went to get ratings on. You can see that doctors
beat the band here. During that particular time 31,000 unique people went to
look at doctor ratings. That beat auto repair shops at 18,000 and plumbers at
18,000, and it beat dentists, although if you include the specialists, they
were maybe at about 18,000. Then eyeglass and contact dispensers were about
12,000. How about hospitals? 1,465 during that particular period, dramatically
lower interest in that kind of information. We talk a little about some of the
reasons that might be.

I just wanted to give you one other sort of perspective that we have gotten
in related fields here. If somebody goes to our website and says I want ratings
of auto repair shops, they click on auto repair shops, takes them right to a
table, basically, that lists 500-600 auto repairs which they can sort and
filter or whatever to find the best shops with all of our ratings on that.

We tried a little experiment with doctors. We said, okay, you want doctors.
We are going to take you to an intermediate page first. We said which kind of
ratings do you want? There were six different possibilities. They were put in
random order. It is totally randomized as to which came first. They were all
written in about equally long paragraphs just so that there would not be sort
of disincentive to work your way through it.

What do we see here? Aetna excelled, United Healthcare Premium Physicians.
This third item, how doctors rate when claims data are used to measure doctors’
performance against nationally defined evidence-based medical guidelines for
quality and efficiency of care — that is pretty much the language right off
their website. This is the thing we are trying to sell people to be interested
in. 7.6 percent of people clicked first to that. How surveyed patients rate
doctors on listening and explaining things, giving helpful advice by phone,
arranging appointments quickly, that was 23.5 percent, almost 25 percent.

Then almost 50 percent was something that we would never dream of talking
about in such an august group as this, which is how surveyed doctors rate other
doctors when asked which doctors they would consider most desirable for care of
a loved one. I kind of hope that causes people to reconsider what we are doing
here, to some extent. Why is that?

Some things are quite helpful in general. That is a simple, quick, what they
regard as a trustworthy answer. That is something that we are trying to achieve
with all of our measures, a simple, quick summary answer from somebody who
seems to know something. I think that is one thing. It also ought to get us
back to the question of how can we get doctors to know what they are talking
about. At any rate, I thought you may be interested in that.

To get to what we want in terms of people using measures, there is some
variety of obstacles to overcome. One is a constrained choice where the
patient’s doctor has chosen the hospital or chosen the specialist for a
referral by the time it even comes up for the patient. In many cases the
hospital choices are constrained by the health plan that the patient chose.
That is a reason that was alluded to earlier, that we might want to have in a
model comparison tool an opportunity to look at the providers who are in the
different plans so that at the moment you are choosing a plan you are at least
not ruling out certain types of providers.

Other things that I think are important to overcome the obstacles to use of
quality comparison information. I actually phrased it here in the negative. The
problem is that what is measured is often not important or distinctions are
just too small. Information about providers is hard for consumers to
understand. It is not clear to consumers why what is measured matters or how
much it matters. The measures are too difficult to digest. The information is
not pushed out, so we are not aggressively enough marketing it, and in some
cases it is just plain hard to find.

Addressing some of those first things about how important the measures are
— we have a whole list of things. We talked about those earlier today, so I
will skip over them. Clinical outcomes. I think there are a lot of things that
could be done on this front that would make the measures a lot more salient,
but they have to relate to outcomes very heavily, whether it is
patient-reported, clinical, death rates, or whatever.

We also have to really help consumers understand why it matters. We have to
explain and document and quantify the relationship between whatever measures we
have, particularly if it is the case of process measures, the relationship
between that and outcomes because patients can relate to outcomes.

We have to provide compelling examples of bad and good results and really
make it hit home. I will give you an example here. We started out one of our
articles on hospitals and choosing hospitals by pointing out that when they go
to the table a few pages back, they will see that some of these hospitals had a
12 percent death rate for the selection of cases we were looking at and some
had an 8 percent death rate.

We try to say that is a difference of four deaths per hundred people walking
into that place. How would you feel about that if you found out that about a
hotel? It is pretty striking. That would be worldwide news if you found that
out, and yet it just rolls past patients. At any rate, you have to make it
compelling.

You also have to make it not too difficult to use or digest. I think the
federal government’s HospitalCompare website is a striking case of failure on
this front in the sense that when you go in there, if you want to compare
hospitals at all, you have to choose three hospitals. Then you go in and you
look at the hospitals on their process-of-care measures. Then you come out and
say now I will go in and look at those same three hospitals on the outcome
measures, the death rate measures, and sort of work your way down. What could
be more painful than that?

You have to wonder actually why it is set up that way. Is it that everybody
is afraid to say this is a bad place? That is another thing that we have to
think about here in general. Who can be doing this who is not afraid? I have an
illustration of where we just use checkmarks to summarize three or four
different things, but I will not go into that. There is nothing brilliant about
that, but it does try to get people to an answer.

I am going to skip on to pushing the ratings out. Some of these things I
will talk about later when I talk about our health plan comparison tool. I
think we need to think creatively about this. One can imagine in one of your
very sophisticated systems that when some information goes in that suggests
that there is going to have to be a hospital chosen or a hospital gone to or a
specialist used, that the personal health record triggers an alert to the
patient that says here is some information that you could use to compare
hospitals. How far distant that is, I don’t know, but we have to catch people
at the moment of truth. Actually, the personal health record might be a way to
do that.

We certainly need to aggressively work toward search optimization. When we
try to even get people to our doctor ratings, you put in a doctor’s name, I do
not care whether we have them or not, we take you to our website. You want to
see how this doctor did. At least we have captured people who are not looking
for a doctor. They may be looking for the phone number, but we got them. And
certainly regular publicity and so on.

Now I am going to talk about the two things that I was not going to talk
about much. I will talk about them a little more here. One is getting patient
experience measures into the hands of consumers. I am focusing here on patient
experience ratings of doctor quality. I am talking about it at the individual
physician level.

I was glad to hear several speakers earlier today, Dave Lansky, for
instance, talking about the importance of going down to the individual
clinician level, although Dale Shaller was talking more about the practice site
and the group level. I happen to be a strong believer that we ought to go down
to the physician level.

We can get adequate sample sizes there. Sample size is an excuse for some
clinical measures, but I see no reason not to go down and aggregate up. You can
aggregate to the ACO or the medical home or whatever, but you can go down to
the individual practice level. But one reason for thinking they cannot do that
is that it is too costly to be feasible. One of the things I want to address
here is that it is really not that costly. It does not have to be that costly.

We did four pilot projects in Denver, Kansas City, Memphis, and in Manhattan
in New York City, where we surveyed patients about individual doctors. We got
the sampling frame data from the health plans — United Healthcare, Aetna, and
Cigna, and in the case of Memphis, Blue Cross of Tennessee. They gave us a list
of doctors, a list of patients, and the list of all the claims for those
patients. We merged all those lists across doctors, and thus were able to
identify a sampling frame of patients for each doctor, regardless of plan, and
then to do a sample for each of those doctors.

We used the C/G CAHPS survey that Dale and others were talking about, the
questions and protocol. We got an average of about 50 completed surveys per
doctor, a little more than that, actually, on average About 40 percent of those
doctors had scores that were statistically significantly different from the
community-wide average. That depending on the question in some cases. Some it
was 33 percent, some if was 48 percent or something, but a lot of statistically
significant difference.

Also, they were substantively significant, practically significant. For
instance, how often did the doctor listen carefully to you, the ones that were
statistically above average on that, 95 percent confidence level, about 90
percent of those doctors’ patients said the doctor always listens carefully to
me. The ones who were significantly below average, about 55 or so percent of
their patients said the doctor always listens carefully to me.

If you believe listening is important, as I do — and this would be true in
explaining things, et cetera, those kinds of big differences — that makes a
difference. Hey, I really do not have a very good chance that that doctor — at
least a good enough chance to be listened to.

The big question here, of course, is this is just prohibitively expensive,
but we were able to do it at a cost of about $120 per doctor. The business
model for this thing was that we actually, as a nonprofit organization, were
the sponsor of the survey. But health plans, as I say, provide the sampling
frame data, and then they agreed to pay to license the data to have back for
themselves to use in their provider directories. That worked, except that we
cannot get them all united, but the others have not decided yet to roll this
out.

The cost, $120 per doctor, does not seem like a staggering amount of money
to me, although when you multiply it out times 200,000 doctors is still $24
million, so I do not want to trivialize it too much. But a lot of things are
$24 million pretty quickly in the healthcare system. We do not believe it needs
to be done every year, paid for by the health plans or by the government, but
maybe every three years. Board certification is every seven or ten years. It
does not have to be every year.

More important, if doctors say I really want to have this done every year, I
want to have it done every six months, I want to have it done every three
months, knock themselves out. We can constantly have the sampling frame data
available. They can use any vendor who follows the right protocol to do it as
often as they want to do it, and they can have it substituted in for their
otherwise public report if they say in advance that is what we want to have
done.

We believe this model could be easily replicated, but the federal government
is going to need to participate because it is going to have to agree to buy
those data for physician compare or whatever. I believe this is a very easy
model to replicate, different from a model trying to impose the HCAHPS model,
the Hospital CAHPS model, onto doctors, where the doctors are supposed to be
incentivized to do the survey because they are going to get paid enough to make
it worthwhile, and then paid enough for public reporting afterwards. There are
some reasons you might not be that incentivized if your scores are low. It is
just much more difficult to do that in terms of getting the samples, et cetera.

I want to say that those data could be used not only for reporting on
individual doctors, but they can be reported for medical homes. We can ask
enough questions that they can be aggregated enough to the medical home level,
I hope, and for ACOs, for maintenance and certification. There are a lot of
other users of data like this.

I will try to show you this website quickly to you. This is a health plan
comparison tool website. If you go to checkbook.org/exchange, then one of the
resources there would be an actual audio over a video of this website. The only
reason I was interested in this is it sort of illustrates some points about
being able to summarize and filter, which I think applied to all of our other
measures.

If you are asking for a concluding statement, I do think it is very
important that we have ways that show that people are able to weight the
different dimensions of quality by pulling sliders, which thing matters most.
You may have your own quality measure based on a lot of data that you could
choose or a lot of measures you could choose among.

You are able to sort so you can put things in order for you, and you are
able to filter out certain types — I only want a plan that has three stars for
quality or better, or I do not want to pay more than X. All those things that
affect health plans also are very important for all of our other measures to
give the person control over choices.

DR. TANG: Good. Thank you. Questions?

DR. FITZMAURICE: I have a question for David Stumpf. On one of your slides
you have what goes into eMeasures. I am thinking we have now moved into
meaningful use stage two. Many of the qualities measures in the Federal
Registry Notice point right to NQF and their quality measures. Are the quality
measures that are in meaningful use stage two and generally in NQF electronic
measures, that is, people who use them know exactly the data elements that
their electronic health records have to report to the physician in order for
him/her to report to CMS to get the incentive payments?

DR. STUMPF: Yes. That is the design. They have very specific value sets in
there. These are the diagnosis codes you are supposed to look for. Then there
is an aligned standard called QRDA, quality reporting document architecture,
which CMS supports now, which will report out the specific thing that was found
in the electronic health record that addresses the measure.

DR. FITZMAURICE: Consumers, I assume — I hope it is a good assumption —
will then be able to see these measures eventually.

DR. STUMPF: Yes. Eventually, we hope they will. Really, there are a lot of
uses for eMeasure. This is one, public reporting. The reporting can be done at
the individual level. There is a QRDA level one report, which is the individual
patient. They should be able to look at their own individual result. Plus, they
should be able to look at a population result.

You can also use them for clinical decision support because the electronic
system can look and see we are measuring this rule on them. The patient is in
the room right now. We can see that they have not adhered to this rule, so let
us get it done right now. There are a lot of uses for these.

DR. TANG: The goal of the NQF QDM is to try to standardize the definition.
As you know, part of the problem is we have too many measures. That can
contribute to confusion as well, and that it is so hard to calculate. The goal
is to have the data element definition standardized, the calculation
standardized, and that it is almost spit out of the EHR. It is going to be
years before that is true.

DR. STUMPF: I don’t think it will be. I would disagree. We are getting
fairly close now to plug-and-play with eMeasures. That is because we
standardize the internal structure, so you have got patterns. Also, people who
are really using good subsumption analytics now — and there are some examples
of this I quoted where you can parse the eMeasures, identify the content, link
that through to your data itself, and pull those together and do the analytics.
There are companies that have automated this almost to the point of
plug-and-play. I think the limitation right now is there is still a little bit
too much variation in the patterns in eMeasure, but that is what the group is
working on right now, is constraining those so that we can get to a
plug-and-play mode.

DR. COHEN: The person characteristics that you described in your diagram on
page 11 around literacy, beliefs, support, functional status, you have got
operational definitions for those that are incorporated into the EHR?

DR. STUMPF: I don’t. This is the model. What I am trying to do is create an
overarching vision and a roadmap. These kinds of things can all be services.
There are people, as you know, who are working on those models of literacy.
There is a whole area of artificial intelligence dealing with beliefs, desires,
and intentions and how you define that.

What I am trying to do here and what NQF is trying to do is create an
overarching model so we know that these things can fit in this context. The one
area that I think fundamentally I disagree with around a lot of the discussion
today is that this is going to become very regimented and very precise. I think
it is going to go exactly the opposite. I think you are going to see hundreds
of applications that are going to be dealing with these various issues, and
that they will plug into this environment because they are interoperable.

I think the level of complexity here is going to reflect the complexity of
our diverse population. That includes the practice community as well as the
patients. If you look at what is happening with the Internet, there are things
that can be configured in many different ways because of the interoperability
of the system.

I think measurement is going to become much different. I do not know if it
is going to be just surveys and things like that. It is going to be things like
crowd sourcing. There are a whole lot of interesting technologies out there,
like computing serendipitous results and things like this. I think we are going
to see some really innovative stuff coming out.

DR. QUINN: The reference to GPII, this is Global Public Inclusive
Infrastructure. This is an approach for auto-personalization that has been —
it is a consortium led by a guy named Greg Vanderheiden, who is one of the
founders of computer-based accessibility. This is the idea that people can
establish preferences with regard to how they want to view information or
whether they use accessible assistive technology or things like that. It will
be automatically applied wherever they go.

The idea of having to enter your preferences for every piece of software
that you use or every piece of hardware becomes sort of a moot point. The
example is the settings that you use and the assistive technology you use on
your personal computer are reflected automatically, automatically personalized,
when you go the train station and you use a kiosk.

DR. HORNBROOK: Could you imagine Rush Limbaugh reading you the train
schedule? This is poisonous.

MR. QUINN: Even more poisonous to Rush Limbaugh, there is a $10 million
project going on in the EU right now. Extending this beyond just assistive
technology and the way that you view information and expanding it to security
preferences or role-based security is basically the way that Verizon does it in
their cloud for their Virginia HIE.

Then expanding it to the preferences that you have about getting
information, I think it is going from sort of one-size-fits-all preferences for
the measures that you see or the way that you query a system like the health
insurance exchange to sort of the personalized approach. One of the paradigms
for the Part D was that a lot of times people wanted to know how much will this
plan cost to cover the drugs that I am on or the drugs that I anticipate being
on. Am I going to have to change? Do you anticipate that being one of the
personalization issues that is going to have to be appropriate for the health
insurance exchanges, for example, a plan that covers pregnancy?

MS. THOMAS: There is a calculator that the Pacific Business Group on Health
has that is very sophisticated, and you can plug in all kinds of information
about yourself and your preferences. I think the tension is if you ask people
too much, they will just say forget it, this is too hard, because picking
health insurance is excruciatingly painful. I think you need to think about
making it fun or making it easy in some way because asking people to put too
much information in, you just lose them. I think you have to strike a balance.

MR. KRUGHOFF: I think that particular example you have, Medicare Part D,
that is a very complex and quite amazing system and very hard to maintain
because they have to have all of the reimbursement rates from all the
pharmacies for all the health plans. It is a very complicated system. I do not
see this happening in the near term in state health insurance exchanges. I
think it would be desirable to have, but it does take a little time to put the
data in.

One thing we have found, if I show this thing later on to anybody, is we try
to get people to an answer within less than five minutes. We know that if you
do not get them there within less than five minutes, they say okay, I will take
the lowest deductible or I will take the lowest premium. In many cases that is
absolutely the wrong answer.

DR. TANG: Any other final questions? Then I want to thank the panelists for
an enlightening discussion. Thank you very much. That concludes this very long
day. Tomorrow we will start at nine o’clock. I suspect we will end before noon.
We are working on a summary of what we heard today. We will try to begin really
honing in on the findings and recommendations for our letter. I really would
like to leave tomorrow with that.

(Whereupon, the meeting adjourned at 5:30 PM)