[This Transcript is Unedited]
DEPARTMENT OF HEALTH AND HUMAN SERVICES
NATIONAL COMMITTEE ON VITAL AND HEALTH STATISTICS
SUBCOMMITTEE ON STANDARDS AND SECURITY
July 26, 2005
Hubert H. Humphrey Building
200 Independence Avenue, SW
Washington, D.C. 20201
CASET Associates, Ltd.
10201 Lee Highway, Suite 180
Fairfax, Virginia 22030
TABLE OF CONTENTS
- Call to Order, Welcome and Introductions — HARRY REYNOLDS, JEFF BLAIR, CO-CHAIRS
- NCPDP Update — LYNNE GILBERTSON, NCPDP
- Update on E-Prescribing Issues:
- Codified SIG Workgroup — LAURA TOPOR
- Prior Authorization Workgroup — TONY SCHUETH
- Update on E-Prescribing Issues HL7/NCPDP Script Harmonization — DR. ROSS MARTIN, LYNNE GILBERTSON
- Introduction to Secondary Use of Clinical Care Data — DR. STANLEY M. HUFF
- Issues in Secondary Use of Clinical Care Data:
- Indiana University Perspective — DR. CLEMENT J. McDONALD
- NLM Standards Related Activities — VIVIAN A. AULD, NLM
- Issues in Secondary Use of Clinical Cara Data:
- Secondary Use of Clinical Care Data to Support Billing — SNOMED CT and ICD-9-CM — DR. JAMES R. CAMPBELL, University of Nebraska
- Auto Assisted Coding — VALERIE J.M. WATZLAF, AHIMA
- MARY STANFILL, AHIMA
- MS. WATZLAF (concludes)
P R O C E E D I N G S [9:04 a.m.]
AGENDA ITEM: Call to Order, Welcome and Introductions –HARRY REYNOLDS, CO-CHAIR
MR. REYNOLDS: Good morning. My name is Harry Reynolds, and I’m
Vice President, Blue Cross and Blue Shield of North Carolina, and Co-Chair,
along with Jeff Blair, of the Subcommittee on Standards and Security of the
National Committee on Vital and Health Statistics.
The NCVHS is a Federal advisory committee consisting of private
citizens that makes recommendations to the Secretary of HHS on health
On behalf of the Subcommittee and staff, I want to welcome you to today’s
hearing on e-prescribing and the secondary use of clinical data. We are being
broadcast live over the Internet, and I want to welcome our Internet listeners
As is our custom, we will begin with introductions of the members of the
Subcommittee, staff, witnesses and guests. I would invite Subcommittee members
to disclose any conflicts of interest. Staff, witnesses and guests need not
disclose conflicts. I will begin by noting that I have no conflicts of
I would request that witnesses and guests turn off their cell phones. Also,
during the hearing, if we will all speak clearly and into the microphones,
those listening on the Internet will be most appreciative, so, Jeff, you can
start the introductions.
MS. GREENBERG: Harry?
MR. REYNOLDS: Yes, Marjorie?
MS. GREENBERG: I don’t know if you received the email from Judy Warren?
MR. REYNOLDS: Yes.
MS. GREENBERG: You’re aware of this, okay.
MR. REYNOLDS: Thank you.
MS. GREENBERG: Her plane was cancelled, and she’ll be here later.
MR. REYNOLDS: We’ll ask her for sure if she actually missed it. See, that’s
the party line, that her plane was cancelled.
MS. GREENBERG: The dog ate my homework?
MR. REYNOLDS: Okay. Our first presentation today is an update from Lynne
Gilbertson on the work that NCPDP has done since our last hearings.
AGENDA ITEM: Presentation — LYNNE
MS. GILBERTSON: Thank you. It’s really great to be back; it’s like old home
week. But I love the energy of the room. Let’s get things done. So I’m here to
the NCPDP status of the NCVHS recommendations to HHS on electronic
prescribing for the MMA, the Medicare Modernization Act.
The testimony will go through the different observations that NCVHS
recommended, of which NCPDP had a part, and give you a status on those items.
Observation 4 dealt with prescription messages, and the recommendation in
particular was to include the fill status notification function of the NCPDP
SCRIPT standard in the 2006 pilot tests to assess the business value and
clinical utility of the fill status notification function as well as evaluate
privacy issues and possible mitigation strategies.
The status on this item is that NCPDP Work Group 11, E-Prescribing and
Related Transactions, RXFILL Task Group, created implementation and operational
guidance to pharmacy and prescriber system participants for the consistent
utilization for the fill status notification transactions.
This guidance includes operational challenges such as automatic triggering
of fill status notifications, triggering on return to stock, inferring pick-up,
privacy, liability, coordination with medication history functions, a patient
changing physicians and other items.
This guidance was added to the SCRIPT Standard
Implementation Guide Version 8.1. They were not balloted items, but occurred
in that publication of the version of the imp guide. The estimated publication
of SCRIPT Version 8.1, because there were balloted items, will be in the
October/November time frame.
Observation 4, coordination of prescription message standards — this was
related to the NCPDP HL7 e-prescribing coordination, the mapping project, and
that status will be given by Ross Martin later this morning.
Observation 5, formulary messages — HHS should actively participate in and
support the rapid development of an NCPDP standard for formulary and benefit
file transfer, using the RxHub protocol as a basis.
The status is that the protocol was brought forward. It had been vetted in
the industry beforehand. There were numerous meetings held with a task group to
vet it even further and the formulary and benefit standard Version 1.0 was
brought forward to NCPDP at the March, 2005, work group meetings.
It was balloted in April of 2005 as part of the ANSI process. Ballots that
receive any negative with comment are adjudicated. They were adjudicated in the
May work group meeting. The comments were about clarification, ways to state
things a little clearer, and they were very positive comments.
As part of the ANSI process, the recirculation ballot is going on right now
in July. The final review of any comments that come in will occur in the August
work group meeting at NCPDP. After the mandatory appeal time frame, which is
September, the NCPDP Board of Trustees will be asked to approve the formulary
and benefit standard implementation guide Version 1.0, and we anticipate that
taking place in October.
The ANSI process is going on in tandem with the NCPDP process, so, again,
we expect that probably by the time all the little bits and pieces — I can’t
promise an exact date, but we know the time frames that hit each of the
different things. We should have an expected publication by the November, at
the latest December, time frame.
The standard includes the sharing of formulary status lists, codes to
explain how to treat non-listed brand, generic, over-the-counter; whether the
drug is on formulary or preferred status; its relative value limit.
It includes formulary alternative lists, conditions under which the
patient’s pharmacy benefit covers a medication.
The benefit co-pay lists, the extent to which the patient is responsible
for the cost of the prescription — the specification supports multiple ways to
state this cost, including a flat dollar, a percentage, and the tiers.
It also contains a cross-reference file of user-recognized health plan
product names to the identifiers that are used in the formulary, alternative,
coverage and co-pay.
It’s quite a proud industry undertaking to see that it got done. It’s usual
in a lot of our work group functions, but it is also amazing to see the amount
of people that came together with various interests and sometimes competition,
but recognized that there needed to be something moved forward that they could
all agree to, and to watch this actually taking place and as quickly of a time
frame — I mean, if there were no negative comments on the ballot, then the
ballot would have flown through faster, but I would have been skeptical that
nobody read it, so it’s always good to have some clarifications added and get
it as tidied up as possible.
Observation 6, eligibility and benefits message
— the item for the NCPDP status on this was a guidance document to map the
pharmacy information that’s on the Medicare Part D Pharmacy ID Card to the
appropriate fields that you would use on the ASC X12N 270/217.
That has been completed. The notification went out with the 7-15 NCPDP
newsletter and the information is posted on the — I believe it’s a public area
of the website.
Observation 7, prior authorization messages — this is to develop prior
authorization workflow scenarios to contribute to contribute to the design of
the 2006 pilot tests and to automate the communication, and we’ll have a great
update from Tony Schueth later this morning.
Observation 8, medication history messages, the recommendation was the HHS
should actively participate in and support the rapid development of an NCPDP
standard for a medication history message for communication from a payer/PBM to
a prescriber, using the RxHub protocol as a basis.
Once again, RxHub fits, submitted what we call a “Data Element Request
Form.” It’s a request to add functionality to a given standard. They
brought it forward in November of last year. It was incorporated into the
SCRIPT Standard Implementation Guide Version 8.0. The standard has been
balloted and approved. At this moment, the NCPDP Board of Trustees is approving
the ballot through the procedures. ANSI approval of SCRIPT 8.0 is also
underway, and I expect notices back from the Board and from ANSI by September.
Observation 9, clinical drug terminology — HHS should include in the 2006
pilot tests the RxNorm terminology in the NCPDP Script Standard for new
prescriptions, renewals and changes.
In June, NCPDP members met via conference call with John Kilbourne from the
National Library of Medicine. Stuart Nelson mentioned in his late update that
he had a new employee who would be taking over this role in coordination with
standards development organizations, and that is John.
We held an initial call to introduce John to the different standards that
we thought there might be some fits for RxNorm, and right now the group is
working on a Q&A document based on the different standards, where there are
gaps, what kinds of problems have been seen with not having a standard set of
code, values to go back and forth in different messages, and John will be
attending the NCPDP August session and during Work Group 11 we will be
discussing that document and starting to form a framework for what the next
work product should be, to try to figure out how we get from the 50,000-foot
level down to the 5,000 and what are we really talking about? What in a
formulary message is really needed? What level of drug discussion goes on in
those business functions? And is there really a fit for some level of RxNorm at
In the new prescription messages that go back and forth, the refills, are
there levels of business discussion that go back and forth with the discussion
of the drug that
RxNorm could fit? At this point, we don’t know. We think, we hope, but we
need to continue that exploratory. And now that John has joined NLM, we have a
go-to person to really start digging into the technical or the clinical side
that the pharmacies and the prescribers understand from our perspective and
John to understand from the RxNorm perspective, so hopefully we’ll be able to
pull everybody together.
Observation 10, structured and codified SIG — HHS should support NCPDP,
HL7 and others, especially including the prescriber community, in addressing
SIG components in their standards. And we’ll have an update in a little bit
from Laura Topor, who’s the task group leader of this activity.
Observation 13, pilot test objectives — NCPDP didn’t have any particular
work item at this point. We are waiting on information from CMS.
One other item I wanted to bring to your attention is a lot of work is
going on in NCPDP with Work Group 14, which is long-term care. They are
spending a lot of time with a lot of industry expertise. They formed task
groups to work on the needs of this sector, especially in light of the MMA.
They’re working on billing needs. They’re examining electronic prescribing
needs, and they will work with NCPDP’s Work Group 11, e-prescribing and related
transactions, to develop enhancements to the SCRIPT Standard that meet the
needs of the long-term care.
They have done a lot of work showing the process flows in long-term care.
They’re working on conformance criteria with the HL7 group for the LTC EHR —
electronic health record — minimum functions. They have pulled expertise from
across the long-term care industry, from organizations, from standard bodies in
these various efforts. They have a lot of challenges, but they have been a
really strong foundation for getting the work done.
And that concludes the status. Thank you.
MR. REYNOLDS: Lynne, thank you. I’d like to congratulate you and everybody
that worked on it. You guys are a guiding light for what the word
“collaboration” really means.
MS. GILBERTSON: Thank you. I get the good job of giving you the status, but
there’s an awful lot of people around the room —
MR. REYNOLDS: No, that’s why —
MS. GILBERTSON: — that could take a lot of credit.
MR. REYNOLDS: — I think as a group they really stepped up.
Jeffrey, you led this, a lot of this e-prescribing, so why don’t let you
open the questions, if you have any. If not, I’ll open it to the rest of the
MR. BLAIR: Yes. I’ll probably save my congratulations, but this is a
preview, because I think the entire industry really — RxHub, NCPDP, HL7,
Express Scripts — I could go down the list of all of the entities that have
worked together to make this happen. This is unprecedented in my mind. I don’t
know that anything like this has ever happened in this time frame before.
Lynne, what is your expectations of the role that NCPDP SCRIPT will be
playing during the pilot tests?
MS. GILBERTSON: I would expect that as the different groups who I know —
some folks are working together to form coordination, expecting what might be
happening in the pilots — I would expect that they would be coming forward
with draft requests of “we think this is what we need to perform this
particular function that we haven’t thought of yet but is part of what we’re
going to test in the pilot so can we put together, you know, a draft copy of
the document that shows how to implement this particular function?”
You know, I could see that we’re going to have things crop up that people
have said, well, now that we’re in the midst of it, we realize could we add a
field here or there to help this function better, knowing that that would then
be brought forward after the dust settles and make it actually part of the
standard. I could see that.
One of our functions I would see could be as some kind of assistance, maybe
a conduit, maybe a way of sharing knowledge during our working group sessions
to discuss who’s doing what in the pilot, what they’re finding, if they need
someone to test something — you know, putting out a call to arms for
“come join us; we’d like to get a sector from long-term care, we’d like to
get a sector from ambulatory,” whatever kind of setting.
Let’s see. I think we’re going to be somewhat reactive because, as you will
see from Laura and Tony’s presentation, there are things that the dust hasn’t
settled on yet and so people are going to be finding things in some of the
draft standards that haven’t been vetted yet that we’ll be modifying as we
learn, typical lessons learned. That’s just find; they’re draft documents to
Obviously, we’ll be very glad to notify the membership when we have the
proposed prescribing pilot information available so that people can react to
the request for proposal or however it’ll be handled.
MS. FRIEDMAN: I have to say a word about that. Just a quick update. That is
in process. We’d hoped to have it on the street by now; obviously it’s not. So,
stay tuned, and I will let people know as soon as it’s published. The work is
going to be done collaboratively between AHRQ and CMS.
MR. REYNOLDS: Jeff, any other question?
MR. BLAIR: Not at this time. Thank you.
MR. REYNOLDS: Okay, seeing no other hands, I have a few — or Stan has one.
So give it a shot.
DR. HUFF: Lynne, could you say more about what the issues and questions are
related to the use of RxNorm?
MS. GILBERTSON: Some of it is my own. I’m just not quite sure what exactly
— from a standards perspective, I can put in a qualifier that says RxNorm and
I can have a field that fits whatever codes you want to throw in there. Okay,
so that part could be very easy.
What I’m trying to help the task groups and the work groups get through is
when they see RxNorm, they’re presented usually with a chart of about eight or
ten tables, a model, and they say, what do I use? When I want to transmit the
drug being prescribed, well, where do I go, what do I use in RxNorm? Or is it
something that my drug knowledge base vendor will provide me some kind of
cross-reference file to RxNorm based on what I have in my system?
So it’s not necessarily the standard supporting; it’s how you would support
it to the best in that particular business case. I realize it’s not a great
answer, but that’s the kind of thing we’re stumbling with.
If on a new prescription, current use is to send the name, because it’s the
most current, the most up to date, the most usable item we have right now, and
you turn that into an RxNorm code, how do you get it? Where does it come from?
At what level do you pull it off?
The other is there have been some questions as well in making sure that
does the RxNorm code that you would pull map exactly what that drug name was
trying to tell you, and is there a confidence factor? And so we’re looking at
what that mapping really is, and does it reflect the right level of what the
prescriber had pulled off when they pulled text, and making sure that
confidence is high as well.
So those are some of the things we’ll be working through in formulary and
Does it make more sense to send a representative NDC in the business case of
sending some kind of formulary list? Does it make sense to send the RxNorm
code? Formulary and benefit has a place-holder for RxNorm, but we have no
guidance in the document for what that means. It’s just a value place-holder
right now that says when we know more, we’ll add it to the implementation
So those are some of the things we’re trying to work through.
MR. REYNOLDS: Michael?
DR. FITZMAURICE: I guess Stan asked one of my major questions. But first,
Lynne, I want to join everybody here — this is just remarkable, the level of
industry cooperation and the output of having concrete products, and you gave
us dates on which they were going to be published. So you’re reflecting the
commitment of a lot of the work of the people who are on these committees, and
traveling and being on the phone, giving up a lot of their time to make
something work. And all I can say is: More!
DR. FITZMAURICE: That’s not what you were expecting but —
MS. GILBERTSON: Yes, sir, we’ll get right on it!
DR. FITZMAURICE: — you always want more.
So, I gather from the RxNorm that to be ready for prime time, there needs to
be more information about the business case incorporated by the developers of
RxNorm and more knowledge of how it might work in the business by NCPDP and the
pharmacies and the prescribers, and that has yet to be worked out yet. Just the
fit of it in the workflow and what the advantages are and the benefits and the
costs of it, is that another way of saying it?
MS. GILBERTSON: Right. That would be a good way of saying it.
And one of the things for the pilot, for example, would be the just — and I
don’t know the answers to this, but one of the questions I would ask that we
nail down is when, if someone’s going to participate in the pilot with it,
where do they get the RxNorm codes from? Are they up to date? You know, those
kind of things, so that we understand — is a production ready, is it able to
be loaded in, do they require or need a drug knowledge base behind it with some
kind of interface as well of some type?
And getting all those — you know, there’s a point where NCPDP will step
back because we’re not in the business of sending code sets around, that kind
of thing. But to help facilitate the people in the pilot, we’d like to be able
to at least point them in the right direction of how you get what you need to
participate in the pilot.
DR. FITZMAURICE: So you’re looking for a commitment by the developers, let’s
say by NLM, to maintain the code set and have a place for it on a website, a
downloadable place where you can say, “Here’s where it is and here’s a
version of it and this is ready to be put into a production.” Is that it?
MS. GILBERTSON: And if the users have questions, where do they call? Who do
they contact? You know, a help desk —
DR. FITZMAURICE: Both maintenance and support.
MS. GILBERTSON: Right. Of a production product, yes. Yes, and just making
sure. Do they call their drug knowledge base company first? You know, those are
just kind of things that — because it’s not out there and really being used in
this environment, those are all just general questions of how do you get it
limping for a pilot?
DR. FITZMAURICE: And so this is some of the information that John Kilbourne
and NLM are going to be understanding and looking to see how they can meet the
MS. GILBERTSON: I’m hoping. That’s on my list of questions as we go down
them, that’s right.
DR. FITZMAURICE: Okay. One more question. That is, on the Observation 6,
eligibility and benefits messages, you talked about the task group has
completed a mapping document between the two different standards. Do there need
to be made changes to implementation guides for those standards to reflect the
mapping, or in the normal course of standards you just say, “I need them
like this into this and so I’ll just grab the map and use it?” I’m looking
at the top of Page 3.
MS. GILBERTSON: Right. Okay — I guess I should apologize. It’s kind of
misuse of the word “map.” It is a map, but not quite in the mapping
functions we’ve been talking about.
Basically, you have an ID card implementation guide that says this is what
your pharmacy health care ID card looks like. And it has, you know, where your
name should be, where the routing information should be, things like that.
This mapping that they built was saying, if you are presented one of these
cards as part of the Medicare program, for example, where do you put the
information that you see on that card into the 270/271 message?
So it’s not a functional/technical mapping; it’s more a guidance for the
vendors that say “the field called RX Bin on the card goes in this field
on the 270 transaction.”
DR. FITZMAURICE: Does that mean a change to the implementation guide for the
270 or —
MS. GILBERTSON: No. No changes were necessary.
DR. FITZMAURICE: Okay. Thank you.
MR. REYNOLDS: Okay, I’ve got a couple questions, Lynne.
On Observation 3, privacy issues was a point that we brought up, and
possible mitigation strategies. And as I listened to the testimony, it mentions
guidance on privacy. But could you tell us a little more about whether or not
you saw any mitigation and what types of recommendations? Do you feel that
you’ve dealt with the subject to the point that you made recommendations or
that it needs further review?
MS. GILBERTSON: To the point that NCPDP, as a standards organization, there
is guidance about privacy, but as a standards organization, we do not take it
that extra step. So it only went as far as notifying the reader about the types
of issues and the things to be concerned about but did not go that extra step.
We don’t give that kind of legal or that kind of guidance in our environment.
MR. REYNOLDS: Okay. On Observation 5, and Laura, I would appreciate it if
you would maybe touch on this as you get to your presentation, as we listen to
e-prescribing and you think of the actual implementation of it, especially in
the individual doctor’s office, when you look at the device they may have,
whether it’s hand-held or it’s a notebook or what it is, as I look at what came
out of the formulary, it talks about a lot of things.
As I reviewed Laura’s presentation last night, there are a lot of pieces to
the SIG. Can you give us any sense of whether or not the actual implementation
was considered as you were putting it together and whether or not you see any
kind of issues — with the way the standard’s set up, it wouldn’t necessarily
translate into when we get into the pilot, we might find that it’s going to be
more cumbersome than originally thought? Or just any comments you might have on
MS. GILBERTSON: Considerations were given, and I know that during the
working sessions that RxHub had prior to bringing the formulary and benefit
standard forward and then after the standard was brought forward, there was a
lot of discussions about the amount of information. I mean, for the first time
available to some participants, you know, to some prescribers, you can’t plot
all this information on a screen somewhere and expect it to be usable.
One of the things that did come across my mind as a lot of these discussions
were taking place about how mystical and magical I think the e-prescribing
vendors are, because they have to take a lot of information and figure out what
is the prescriber’s workflow. When do you want this information to show up? Do
you want it to be a button you click? Do you want it to be a hard versus soft
message? Things like that.
So there was consideration, and SIG we’ve talked a lot about, where SIG
could go. I mean, there were people who started SIG thinking that we would have
a six-byte code and then millions of possible combinations to describe each one
of those six-byte codes, that that would be the easiest to do. And that’s very
true, but it’s usefulness quickly became — you know, it was not at all useful.
And we tried to then look at other examples, as SIG has been worked on for
years, of how far to the other side you could go. And then there were a few of
the doctors who kept the mantra of “keep it simple, keep it succinct.
We’re doing the 80/20. We’re not doing all 100 percent.”
So it may look a little verbose still, but there’s a lot of information that
has to be shared.
As far as presentation and what the vendors — maybe some of the audience,
Terri or others would have a lot, you know, to give you some real good examples
of how that’s been translated into usable, but I don’t have a lot of working
knowledge of that other than we did make sure it was on our shoulders when we
MR. REYNOLDS: Yes, I don’t need to go into that much detail. What I
appreciate is that you considered it, you’ve had the discussions, because in
the end, we all, as you’ve heard from the hearings, having it be able to work
in the end is also one of the driving ends.
Jeff, you had another question?
MR. BLAIR: We don’t have the reg yet in terms of how the pilot test will be
constructed, so I don’t know if this question is going to be good that it
happens now or not good.
MR. BLAIR: So I’m beginning, as you could tell — I sort of feel like a lot
of the work has been done to identify the gaps and limitations in standards and
to address them and to get them ready before the pilot tests.
And so my thinking now is in terms of getting ready to make sure that when
the pilot tests occur that the mechanisms or processes or structures are in
place, to gather the information that is needed to feed back to the SDOs and
the industry so that either corrections can be made or issues can be mitigated,
and so on one part that’s a CMS activity, on the other part it would be on the
part of the SDOs to have a mechanism to, as you indicated, answer questions,
refine, maybe a help line, clarifications.
But the other piece might some kind of an internal adjudication process
during the pilot tests to try to resolve issues that are being identified. Or
even to give information back to CMS to say “here’s the kind of data we
need to have captured” during the pilot tests so that we can resolve
So I guess my question is: Have those discussions already taken place
between NCPDP and CMS, or is that something that you’re planning on doing, or
is that something that, you know, you sort of feel like you can’t do until
after the rule is released? Or what is the status of addressing those types of
MS. GILBERTSON: Do you want to go first, Maria, or should I?
MS. FRIEDMAN: I really can’t say anything to address that because things
are in process and I really can’t say anything till it’s on the street, you
MR. BLAIR: Okay.
MS. GILBERTSON: But the other side, Jeff, would be that I would expect —
and if this is a volunteer, then it’s on the record — that as part of the
process, depending on what items are put forth for actual testing, that the
different sectors, the people, the organizations that come forward — and
actually I don’t know if you have to sign on a dotted line or exactly how you
sign up to participate — that we have something that’s very clear to them that
if they’re testing something having to do with an NCPDP standard that they can
use, you know, me as the contact person and we will get items identified and we
will work through them.
If it’s, you know, a list of items I need to bring back to the work group,
we’ll do that. I mean, we have process for bringing things forward anyway. It’s
just making sure that those who participate in the pilot know they have some
place they can go.
MS. FRIEDMAN: I’d just like to draw on a couple
of things we can say at this point, and one is it will be competitive
process, and people will have to submit proposals.
MS. GILBERTSON: It’s competitive?
MS. FRIEDMAN: Will be a competitive process.
MS. GILBERTSON: I was hoping it would be open to anyone who wanted to
MS. FRIEDMAN: There will be guidance in the document we will be issuing
soon, I hope. It will all be tested in how you apply.
MR. BLAIR: Well, in a sense that’s good because it does give CMS the
opportunity to raise questions to make sure these structures are in place to
address the issues that I’ve just raised, so I’ll just leave it at that. P>
MR. REYNOLDS: Okay. Lynne, thank you very much, and we’ll move on to our
next presentation with Laura Topor and Tony Schueth who’ll be presenting, and
we’ll have both of you go ahead and do your presentation and then we’ll open it
for questions. Laura?
AGENDA ITEM: Presentation — LAURA
MS. TOPOR: Thank you for having me here today and I apologize in
advance if my voice goes.
Just to walk through, I want to give you an overview of some of the history
of the SIG standard, what we’ve done, show you the structure that we’ve
developed, the impact we think it’ll have, and then talk through some of the
As many of you know, we’ve been working on this off and on throughout the
industry. It resurfaces annually. It’s been going on for over ten years.
The stakeholders have changed where now long-term care is a big player at
the table on this. We’re trying to involve the different hubs such as RxHub and
SureScripts to make sure that we’ve got everybody at the table; the previous
efforts with NCPDP, with HL7, with the Continuity of Care Record, as well as
the work being done by Julie James in the U.K. trying to incorporate that.
We went into this with a couple of operating assumptions. One, the need to
be flexible. It’s interesting that Lynne brought up about the physicians in the
80/20 rule because then I got an email from one of the physicians saying,
“Okay, but remember, we got rid of 80/20 and we’re at 99 percent where we
think what we’ve mapped out really will work for any form of a SIG based on
what we know from prescribing today.” And again, the different industry
segments with inpatient and outpatient and with products that have transitioned
that used to be delivered or administered only in an inpatient setting now
being administered in an outpatient setting.
There’s over a hundred people signed up on this
task group, and I would say out of that there’s a core group of
approximately 20 who have been very, very active in this, and pretty much
everybody’s at the table. We have the pharmacy providers, we have physicians,
we have the knowledge vendors, payers, the e-solution organizations, people
from the academic centers, other SDOs.
I’ll take a moment to recognize Peter Kaufman of Continuity of Care Record,
Alan Zuckerman at Georgetown, Rick Peters from AAFP, Rob McClure for a lot of
the work that they’ve done because they’re bringing that clinical practice
piece into it, which is an area where NCPDP has not always had strong
When we set out, we wanted to be sure that what we were doing would conform
with existing e-prescribing standards but not duplicate any of their content,
and I’ll get into that a little bit more later; obviously take advantage of all
of the industry experiences that we know of to date, and the existing work
products so as to not reinvent the wheel.
And again, a key focus on developing something that was flexible, that
would support interoperability with different systems — again, inpatient,
outpatient in a retail environment.
We really got going on this a little less than a year ago. NCPDP met
mid-August last year in San Francisco,
and it was probably the end of September when we had our first conference
call. Since then, we’ve had calls pretty much every other week. We’ve met four
times face to face and we’ll be meeting again in August in Philadelphia.
We’ve been working with organizations, other SDOs. We do have a format that
is pretty solid right now, and I’ll show you that, but we’ve mapped
approximately 30 SIGs to this format to make sure that what we’ve come up with
will work out in the real world.
We have a draft of an implementation guide and I did get confirmation about
a week or so ago that what we are doing is in conformance with the current ASTM
Continuity of Care Record.
Again, in terms of structure, making sure that this will fit with whatever
work is being done with NCPDP SCRIPT, to make sure that it fits and that it’s
something HL7 can incorporate as well as the Continuity of Care Record.
Right now, we have this structured in segments, and the segments include
dose, dose calculation, dose restriction, the vehicle, route, site, frequency,
interval, administration time, duration, stop, indication and free text.
So, what I’ll do is walk you through all of those segments in a little bit
more detail. I won’t go through
all of it — you don’t want to hear all of it.
MS. TOPOR: Lynne can attest to that. She’s been on all the calls!
Within the dose segment, which will define a fixed dose, is a repeating
segment. It will support something as a range.
At our discussions last month, we identified the need for a dose indicator
which, from an implementation and programming perspective, it was felt as
valuable to say, okay, do I even need to look at this field or can I just skip
it because there’s nothing in here and it’s all somewhere else? So we put the
We have a dose delivery method, so how is it delivered? Is it take, is it
applied, is it injected? For every code field that we have, we have a code
system and a code system version field so that we can identify, again from a
programming and implementation perspective, what we’re using.
We have a dose delivery method modifier. Again, working with the data that
we had available to us of what’s happening out there in the real world today,
you know, apply repeatedly, apply to the affected area sparingly. So we pulled
all of those.
We get into the dose unit’s text, which is,
again, the milligram, the numbers. We have codes for that.
And sequence position allows if it’s take one to two every four hours. The
sequence position supports that, as does the range modifier.
The dose calculation segment has been a topic of much discussion. One of, I
think, the struggles for this group has been focusing on the broader picture
and looking at not just what happens when I walk out of the doctor’s office and
I go across the street to just, you know, neighborhood pharmacy XYZ but what
happens in the long-term care setting? What happens if it’s something that’s,
you know, administered or mixed at school, trying to incorporate all of those
as well as inpatient? So, a lot of concern about whether or not the calculation
segment was truly needed. I think we finally all agreed that there is a place
for it, probably much more so on some of the more acute care settings than your
basic “take two and call me in the morning” type of prescription.
So we have a lot of the same fields that you saw in the dose segment. The
range modifiers — we will have the ability to actually put in the calculation
— so, in the example I think it’s 125 milligrams. There are 40 milligrams per
kilogram per day and three doses will be able to do that as well as have a code
to indicate milligrams per kilogram.
Moving along. The dose restriction segment — again, this was a patient
safety issue as well as what’s commonly seen in prescribing today. You know,
“don’t take more than ten tablets in 24 hours” or “not to exceed
this.” So we did incorporate that, again, with the code, with text, with
variables, to allow for the incorporation of that including a calculation
The vehicle segment, another challenging one. Is it part of the SIG? Is it
part of patient instructions or pharmacist instruction?
We did go back and forth on this one a fair amount. We had some pretty
solid examples of “mix with applesauce” where applesauce would be
your vehicle as opposed to “take with,” which would be a different
segment and part of the patient instructions.
So it’s important to keep in mind that within the segments, pretty much
everything is optional. We are working through defining the situations of when
it would be used.
The route segment, hopefully self-explanatory. It’s the route of
administration and it is a repeating segment, so text code, sequence, all of
Site segment, similar to route in structure, simply, you know, “insert
in left ear” or we did get pretty granular on some things, down to the
vein. Really just went away from that because we figured the physician
wouldn’t know which vein the IV was in anyway, so —
MS. TOPOR: The administration timing segment incorporates a number of
things. It’s the actual timing, so if a date is specified, a time is specified,
if it’s morning or before meals, those components.
We’ve also incorporated in here the rate of administration, so if it — I
think my example is five milligrams per kilograms over two minutes, one gram IV
push over five minutes, things like that. So we incorporated all of those in
there to support, again, more of an inpatient acute care setting.
Frequency is events per unit of time. It’s pretty straightforward.
Interval is time between events, so “take one every four hours.”
The duration segment allows us to support when the prescriber specifies
exactly how long you’re supposed to take the product for — so, “take for
ten days,” “take until gone,” as well as a stop segment, which
is just “take it and then stop.” So we’ve incorporated those.
Indication segment is really the indication for use of the medication.
Again, a lot of debate on some of
the items that you don’t see in here, and I’ll talk to those, but really we
know that there are prescribers out there who are going to write something and
it’s going to say, “Take as needed,” or, “Use as directed,”
and that’s the only thing that we’re going to get. So we needed a way to
support that, or “Take as needed for insomnia,” things like that, and
we put all of that in.
And then specifically we maintained a free text segment, and we looked at
when this would be used. There was a lot of discussion about existing free text
fields available within SCRIPT or within HL7, so what we wanted to do was
really narrow this down to say that this is the SIG free text.
If it’s being used, here are the six situations where, you know, you tell us
why you’re using it, what’s in it. Is it because the system cannot generate a
structured SIG? Is it to capture what the prescriber ordered, which is
important in a number of states based on existing state laws? Is it completely
pulled together from the structured SIG? Is it pure free text?
And then the last two values that we’re recommending is fulfillment
instructions. Those are the instructions from the prescriber to the pharmacist
but they aren’t part of the SIG as it’s viewed by the group. And then, the
patient instructions. So, again, not necessarily part of the SIG of how to
address or consume the medication and then how often, but by the way, do this.
So those are the pieces. And then the entire segment does repeat.
And then we picked the Prednisone example because everybody likes the
Prednisone example, which is Prednisone 10 milligrams, and you’ve got it to
take four tablets a day for three days, then three tablets a day for three
days, then two, then one, and then stop.
And so what we’ve mapped out here shows you, again, we’re only using the
fields that are needed to transmit the SIG, so there’s a dose indicator saying
“yes, there’s information in the segment; keep going.”
The delivery method text, what the dose is, the unit — in this particular
case, we’re using a SNOMED code set.
The sequence position, which tells you the first thing you’re going to do is
take four, then three, then two, then one.
The route, we used an HL7 code set as an example.
The interval is one day, and we’ve got that mapped out, the interval
modifier to say “take one, then it’ll be three a day, then it’ll be two a
day, then it’ll be one a day.”
The free text string is just the indicator and then what the actual script
is, what was written, and then the repeating segment to show you the sequence
So that’s one of the 30-some examples that we’ve managed to map out so far.
In terms of what’s next on our plate, it’s finalizing the format. We think
we’re about 95 percent there on the format in the examples.
Our struggle right now is code set validation. What code sets are we going
to select? How are they maintained? How are they distributed? And trying to
come to some agreement on that and keeping focused on the work that’s being
done with RxNorm and a number of the other initiatives.
Once we can get through that last hurdle, finalize the implementation guide.
There’s been discussion among members of the group about doing a pilot and then
sending this through the necessary balloting processes via HL7 and NCPDP SCRIPT
with Work Group 11.
And then to figure out how we’re going to launch this. At this point, it’s
still going to be voluntary within the industry. I know we’ve got some
opportunities again with AAFP and some of the other trade associations that are
out there to really get this out in front of the prescribers and their
e-prescribing system centers as well as the pharmacies.
That’s where we’re at.
MR. REYNOLDS: Okay. When I first saw your presentation last night, I don’t
need to hear from Tony about Friday morning. You did a great job, both of you.
MR. REYNOLDS: Really, in all respect here, I didn’t know how I was going to
break that news to him.
MR. REYNOLDS: Tony, if you’d go ahead and proceed. Thank you very much.
AGENDA ITEM: Presentation — TONY
MR. SCHUETH: Thank you. My name’s Tony Schueth, and I’m the Managing
Partner for Point-of-Care Partners and the task group leader for Prior
Authorization Workflow to Standards Task Group.
I think the name of the organization, or the task group, is important, and
it sends a message that we’re not just looking at a single transaction. We’re
looking at prior authorization, from soup to nuts, to the point where plan
determines status of a drug through to when the claim is adjudicated at the
pharmacy. And so it’s, you know, the entire process, and we’ve mapped that out
and we’ve got different organizations working on it that bring different
perspectives to the process.
My first slide that I want to present is a quote from one of our task group
members recently, and I think this can’t be stated enough, and that’s why I put
this as the first slide in this presentation. I’ve said this before to this
group, to NCVHS, I’ve said it to the work group, and we’re going to say it
again: “This is not an attempt to usurp the coverage decisions of the
plan, but it’s an effort to streamline and standardize the mechanism for
So everyone understands that that’s the philosophy, that that’s why we’re
working on this project, and we’re all sort of in agreement. And every time,
you know, we might get stuck, you know, we consistently bring that up, so it’s
a philosophy that permeates the whole process.
Again, the task group name is Workflow to Transactions. It was formed at the
November 18th NCPDP work group meeting. I’m the task group leader. A gentleman
by the name of Ajit Dhavle, who’s here today in the audience, has been also
very active in leading some of the latter sessions, and I’m going to talk to
you guys exactly about what he’s been doing in a minute.
Our objectives are to promote standardization, the standardized automated
adjudication of prior authorization; coordinate the further development and
alignment of standards, and identify additional needed standards.
We have about 40 people that are task group members, about half of which are
active. We’ve got about 20 active folks.
And I think what’s pertinent about this particular slide is how
representative this group is of the industry. First of all, it’s not just an
NCPDP task group. It’s a joint effort between NCPDP, HL7 and X12. So we have
task group members that come from each of those organizations, in fact that are
leaders of groups in the other organizations.
We have physicians, nurses, pharmacists. We have folks from managed care,
from PBMs, pharmaceutical manufacturers. We’ve got retail pharmacies that are
represented. We’ve got providers. We have about as representative a group that
you could ever hope for, you know, with only 40 task group members, and even
among the 20, it’s a very, very representative group, and we’re proud that
those folks are very active.
We’ve had several meetings. I’m not going to go into a ton of detail about
these meetings, but the point is that we’ve had, you know, basically one call a
week for the last almost two months, two and a half to three months.
Where we are in the process I’m getting to in a minute. But the first thing
I want to do is talk through the workflow.
Prior authorization really begins with the payer, the health plan and the
PBM, who determines the prior authorization status, the criteria and the rules.
And I think it’s important that I sort of define what we mean by these because
I think there’s been, in the early days at least and maybe even a little bit
further on, there was some confusion within the task group.
The criteria — when we talk about criteria, we’re talking about the
questions that a plan asks relative to the request for prior authorization.
When we talk about rules, what we’re talking about really is the logic. It’s
the “if, than, and” kind of thing. And so the payer determines
status, the criteria, and the rules.
Then what happens is that drugs can be flagged as requiring prior auth, and
some very simple rules can be applied using the NCPDP formulary and benefit
standard. Things like age restrictions or quantity restrictions can be
accommodated in the formulary and benefit standard, which, you know, as Lynne
just mentioned, is something that is going to be ready by the pilot.
And so, you know, one of the things that we might think about is that, you
know, without even piloting — I said is going to be ready by the pilots — but
also a standard that’s been named or proposed as one of the foundation
standards, so, you know, there is a piece of prior authorization that could
actually be, you know, put out there today that doesn’t even need to be
Lynne went into a lot of detail on what’s in the formulary and benefit
standard but there’s a couple things related to prior auth that she might not
have mentioned and I’d add. Things, for example, like I just said — we’ve got
status of prior authorization, we’ve got quantity limits, age limits. There’s
also the ability to tie information directly to the drug and to put, you know,
a link, a hyperlink, to information.
So one of the things that many plans do is to have a website that has the
prior authorization form. So what you could do is today, without even piloting
this, is, if the regulations are written as such, it could go out and they
could launch and they could request, you know, this form and fill out this
information or they could get certain prior authorization information.
Sorry for the diversion but I thought that was an important point to bring
up at this point.
Now, it happens when the patient visits the prescriber, the prescriber
writes the prescription, and what happens is that they see that a drug requires
prior authorization. And what they’re going to do today, based on our last task
group, is that they’re probably going to request prior authorization from the
plan, and the information that they’re going to send in the request is going to
be some basic information that they already have about the patient and about
Then what’s going to happen — and that turns into a 278, an X12 278
transaction, which is a HIPAA-named transaction.
That’s transmitted to the payer, and based on this last task group, where
we’re at is the payer would then respond with the criteria, with the questions
that they wanted answered relative to that request.
So there’s a back and forth.
And then when it gets back to the physician side, they respond to those
questions and then transmit this back to the payer, at which point the payer
can operate whatever process that they have within their four walls, whether
it’s, you know, going to go to a committee, or whether, you know, there’s an
individual that can make a decision. They probably have rules; there’s all
kinds of different ways, and every payer is different.
But then they’re going to respond, and they’re going to do one of three
things, really. They’re going to ask for more information. They might deny the
request if they have enough information to make that determination. Or they may
request more information.
So this is sort of a back and forth.
MR. BLAIR: Tony, requesting more information you gave twice; I think there
was a third thing that you wanted to give — deny or approve.
MR. SCHUETH: Or approve. Thanks, Jeff. Thanks for the clarification.
And this back and forth comes in the form of an X12 275 with an HL7 PA
attachment. So now we’ve identified, and we have standards within three
standards development organizations, NCPDP, X12 and HL7.
So the next phase is let’s assume now that it’s been authorized, that
they’ve approved the drug to be prescribed. What they’ll do is they’ll send an
authorization number to the prescriber.
That number is transmitted then with the prescription electronically to the
pharmacy in the form of an NCPDP SCRIPT. The pharmacy then includes that number
with the claim that they submit to the payer, and the prescription can then be
Now, there’s one other piece of all this that I should mention, and that’s
that we already have a process where if a prior authorization is originated in
the pharmacy, there can be a request and response transaction between the
pharmacy and the payer. And that already exists today within telecommunications
standards, so it already exists today and it’s out there being in use today.
Okay, so with that as sort of the framework of what we’re doing, so what
decisions has the task group made?
Well, in the very beginning what we did, and in fact what Lynne had
presented to NCVHS in the past — I’m sorry if I switched some slides around a
little bit — what Lynne had presented in the past is that we have looked at
six therapeutic categories and seven health plans and done some analysis. And
what she showed you was sort of a spreadsheet, and on that spreadsheet she
showed you that not each plan had the same criteria, the same questions for
each drug, that there are differences between plans.
And we only looked at, like I just said, six different drugs and seven
different plans. So the first decision that the task group made was that we
needed to be more comprehensive. There was no way we could build a standard
based on, you know, that small of a sample set.
So what we did was we decided that we would do more analysis. And we
actually made a request, and at this point I’d like to acknowledge AHRQ, who
helped us fund the consultant to do this analysis.
I’d also like to acknowledge MediMedia. What they did was they sort of mined
their database and determined, you know, which plans had prior authorization in
forms and then they went out and they gathered about 300 different forms from
health plans and PBMs around the country.
We also put out a request for forms to be submitted to us, and organizations
like BlueCross BlueShield of Tennessee and Caremark, they provided their forms
So we ended up with about 350 forms, and Ajit led this process. And what we
did was we analyzed this, and we decided that we wanted to complete this
analysis as close to the plan intention as possible. We wanted to look at it by
drug and therapeutic category, depending on the way that the plan did it. We
wanted to record the decision tree, or the rules if the plan had put the rules
on the document, and that happens. Sometimes those rules are kept within the
organization and sometimes they’re put on the form.
And we wanted to log information that was outside of the drug criteria
questions that were on these forms as well.
Now, when we did the first analysis, we had a managed care consultant,
someone who had been in managed care for 25-30 years, who was sort of between
jobs. We had her do the analysis. And with her experience, she was able to sort
of normalize. That was when we did the six therapeutic categories-seven plan
And now that we were doing the full analysis of the industry, we didn’t feel
like a consultant should normalize that data. We felt like we should do that as
a task group. And so we decided to do that, but what we wanted to do was we
wanted to make sure we had broad representation of sort of some of the right
people, and so we’ve made a concerted effort to make sure that there were
physicians, pharmacists, plans, folks from HL7 and folks from X12 that have
been on each of these sort of task group calls where we’ve been normalizing
We also decided that we would just have one PA attachment versus using, you
know, laboratory and different attachments that are already in the works, so we
would have just one PA attachment.
And we decided the drug or therapeutic level criteria would be transmitted
in response to the initial PA request as I sort of just described in the flow a
What have we accomplished?
We’ve drafted a PA attachment. We’ve drafted an HL7 attachment. But we’re
now waiting until we’re through the normalization process just to sort of tweak
it a little bit. We drafted it based on the first work that was done; now, you
know, we’ll update it based on the normalization we’ve been doing.
AHRQ has not only, you know, helped us fund the initial process but they’re
also helping us fund the normalization. It’s just an awful lot of work, and
what it’s been able to do is just sort of keep the momentum going. AHRQ’s help
— you know, we appreciate it so much. And Michael also has been on a couple of
calls and sort of helped rally the troops a couple of times, so we really
appreciate that as well.
MR. SCHUETH: It’s a lot of work, and any time that somebody gets on there
and says you’re doing a great job, that helps.
As I said, we analyzed 350 forms, about 1700 questions from 53 PBMs or
plans, and we’ve normalized data now in the following therapeutic categories:
ED, anti-fungals, antihistamines, Cox-2s and PPIs.
So what’s going to be ready for the pilots?
Well, first of all, as I mentioned before, pharmacy PAs can be submitted via
NCPDP telecommunications, so that piece is alive and operational today.
And, of course as Lynne mentioned this morning, formulary and benefits is
able to go today.
The 278 is in the process of being updated. In fact, comments close at the
end of this month and there’s a meeting next month, and so the 278 will be
ready for pilot as well.
Now, the whole process of writing a prescription, transmitting it via NCPDP
SCRIPT to the pharmacy — I’m sorry; I dropped some text, but that being
submitted as a claim to the pair, that’s all existing and operational today as
The piece that will not be ready by January 1st but nevertheless, you know,
could be added to a pilot process is the 275 — well, the 275 would be
available, but the HL7 attachment, we’re probably on track for that not to be
available until February of next year.
And so that’s sort where we’re at. And I’m going to get a timeline in a
little bit that’ll make that whole thing a little bit clearer.
Now, at this point what I’d like to mention is another thing that that the
task group has brought up recently, and this is a new slide — my apologies.
We’re going to redistribute. I added this slide last night.
This is the whole notion of — actually, let me go back a step — when the
information is on the prescriber’s desktop, you know, can we ask the questions
and provide the criteria right then and there? Right after the doctor sees that
a drug requires prior authorization, can they go and can they fill out a series
of questions? I mean, can they do that in a structured, machine computable, and
And the task group has struggled a little bit with that, and some folks that
are part of the task group that are also active in HL7 are working with the HL7
clinical decision support group on an effort called GELLO. And there are some
folks in the audience that if we get into a lot of detail about this, they can
come up and sort of help me on this.
But GELLO is basically Guideline Expression Language, and then they added
the “LO” to make it jiggle or something.
MR. SCHUETH: I’ve got to have a joke about that.
What GELLO does at a very high level is it does two things that are really
powerful to prior authorization.
The first thing is it allows you to ask or present this criteria in a
structured, computable way, okay?
But the second thing that it does is it allows queries within the electronic
medical record to sort of answer this information so that the physician or his
or her staff doesn’t have to type in this information, you know, from scratch.
Each of these questions type in the answer to it.
It allows some of this information to be pre-filled, and it also allows them
to pull out things like labs and things like that, so pull that out of the
electronic medical record and include that in the response, or the request for
So it allows this whole thing to be more streamlined. It’s really an
exciting project. We haven’t spent a lot of time in the task group talking
about this. There’s a separate effort within HL7, and I’m going to talk a
little bit more about that within the timeline here.
So the timeline. The PA attachment, as I mentioned, because this is within
HL7, that it’s expected to be, you know, adjudicated and balloted by January
8th, by the January 8th to January 11th meeting. So that wouldn’t be ready by
January 1st, but it certainly could be included as part of the pilot.
GELLO’s further out. It is, it’s further out. But now is the time to be
thinking about it and now’s the time to be talking about it and working on it.
The first thing that really needs to be done with GELLO is that we need to
do some analysis, and specifically we need to look at the syntax within HL7 and
if there are any gaps in the HL7 rim, you know, to see how that fits with prior
And before we can do that, we need to finish the state of normalization, so
that whole process hasn’t exactly started yet. We’ve sort of asked for funding
for that, and that funding request, I think it’s in the process of being
drafted at this point, but there has been some, you know, sort of discussions
about the possibility of that and it does seem to be possible.
The interfaces piece of it is further out, and it’s going to take a lot more
effort, because what the second part of this is is the part that I talked about
where GELLO would go out and it would grab information from within the
electronic medical record and it would pull that into their quest. That’s a
much more complex process and just a little bit further out.
X12, as I said, we’re really making good progress with X12. The 278 will be
ready. The 275, it’s not going to be voted on until February of next year, but
nevertheless it can be piloted, so they’re comfortable with that piece of it.
And of course formulary and benefits, Lynne’s already talked about.
So what are the next steps?
We need to complete the data normalization for therapeutic categories. We
need to put the data into the format that’s required by HL7 for the attachment.
We need to complete harmonization of NCPDP and the Medicaid group.
They are actively involved in this as well.
We need to update the 278, 275 work groups and move 275 to comment and
ballot. Within HL7, we need to create a booklet and ballot that booklet.
Long-term care, I’m sorry I neglected to mention earlier, has been involved
in this project, but they need to sort of determine the impact of prior
authorization on them, on long-term care, and then sort of streamline those
And we’ve been doing all this over the phone, unlike, I think, Laura’s group
who’s met, I think she said, four times. We’ve done this all over the phone,
and we may need a face-to-face meeting. It’s something that we’ve talked about.
So one of the things that I think you guys wanted from us was sort of a
problem list — you know, what are our challenges right now?
We’re having some challenges with drug allergy code sets — you know, just
which ones do we use, those kinds of things. There’s lack of code sets for
outcomes of previously failed therapy.
There’s an inconsistent classification system for prior authorization. Some
plans use therapeutic categories, others use drugs, still others use some sort
of a generic form, and so we need to settle on something like that, and that’s
something that the task group is working through.
There needs to be consensus to encourage drug specific criteria versus
general forms. So what happens right now is we’re just not super excited about
the generic forms because it can be very time consuming and not that efficient,
so we’re going to try to encourage more drug specific forms to be used.
There doesn’t seem to be any industry consensus on therapeutic categories,
so if you look at this from a therapeutic category standpoint, that’s great,
but (?) uses one set of therapeutic categories, (?) uses yet another. You know,
there’s no consensus on which one to use.
And there’s insufficient standardization, structured way to present the
criteria and the rules. That’s what I was talking about relative to GELLO, so
that’s another sort of problem that we have.
Some of the issues that we need to resolve is, you know, what’s going to be
the home of the PA questions/criteria superset? And we need to put together a
documentation implementation guide.
What processes will be used to keep the criteria updated? How will new
questions and criteria be added?
Some plans might be comfortable with rules being presented in the clinical
system, others might not. How do we facilitate this?
So these are all issues that the task group is working on.
What can HHS do to help?
Well, you know, one of the things — and this may be being worked on; I
think I’ve heard that it has been, to a degree — but there needs to be some
sort of central information code set repository. So as we’re going through this
normalization and we’re saying, you know, gosh, where do we get code sets for
this or where do we get code sets for that? If we could just go to that
Because we don’t want to recreate the wheel. We don’t want to create code
sets when they’re already out there. But we’re having a heck of a time even
with folks that are actively involved in HL7 and X12. We’re having a heck of a
time sort of figuring out where these are.
And we think that the support of the GELLO development effort will also be
And that’s sort of my presentation.
MR. REYNOLDS: Okay. Thanks to both of you. Excellent job again, and it’s
amazing what all of you are pulling off. So, open floor to questions. Simon?
DR. COHN: Well, Harry, I actually want to sort of second your comments about
the progress being made and I want to extend my appreciation of the efforts.
Tony, actually I had a couple of questions for you. And obviously I think
one is — I guess I shouldn’t be surprised, I know the Subcommittee has talked
about decision support now at some time and we’ve looked at it from time to
time. I guess we’d always sort of thought it would be relating to critical
clinical drug interactions and things like that.
But I guess I shouldn’t be surprised that it should maybe become more
advanced in things that have to do with payment or prior authorization or
whatever and that may be one of the first major sort of standardized
interoperability use cases.
I do see you sort of glommed on GELLO and clearly, you know, I think we’ve
all been sort of watching it. I mean, is your assessment that that is really
the way to go, whether that’s ready enough for prime time?
MR. SCHUETH: No, I wouldn’t say it’s ready for prime time.
What we would say is — I guess we used the analogy, if you’re going to
build the bridge, you need to start with the girders, you know? So we think
that, you know, now is the time to be working on it.
And when I talked about it, when I showed the timeline for it, you know, I
showed the first piece that needs to be done and can be done yet this year is
analysis of, you know, where it is and how prior authorization would fit into
And then the second piece would be a little bit longer term, and that’s
relative to the timeline for developing it and its interfaces, and that would
extend on into the next year.
I wouldn’t suggest that that should be part of pilots, but, you know, we do
think, and the task group has talked about that as being, you know, one of the
long-term solutions and a use case that works.
I would like to invite — particularly when we get into some of these more
detailed questions, and I’m sorry; I meant to acknowledge many of the task
group members are actually here in the room today. We’ve got Ross Martin from
Pfizer, who actually, you know, could go into a lot of detail about GELLO;
Ajid, some others that are in the audience that we could bring up.
DR. COHN: I guess in the interest of time I’m not sure we want to go into
great detail on GELLO. It may be a subject that the Subcommittee wants to delve
into further in subsequent hearings.
I guess I was just trying to get a sense from you since you spent a lot of
time at the end sort of talking about it. I wondered how ready it was, based on
your timeline and all of this. And it sounds like you actually have an
optimistic timeline, based on your last comments.
I guess the other piece I would is you identified a set of problems that had
to do with inconsistent classification systems and things like that. I was
uncertain in my own mind how much to relate that to the use of decision support
versus just the actual existence of these attachments and use of doing prior
authorization. Can you reflect on that? I mean, are these all barriers to even
doing sort of non-decision support prior authorization or are those things that
just would be nice if we had decision support as part of it?
MR. SCHUETH: That’s a very good question. First, let me say that the notion
of GELLO, of decision support, has not been widely vetted within the task
group. It came up on the last task group conference call. It was presented as
an option to help us address this challenge of displaying and presenting
criteria in the workflow and in responding — you know, it was presented at
that time, and the task group got very excited about it but would agree with
your assessment, Simon, that it’s further off.
And I would comment also that the timeline — it is optimistic, but it would
be based on if it were possible to receive funding, okay?
DR. COHN: Okay.
MR. SCHUETH: Now, to your question about the code sets and does that apply,
so the answer to that question about the code sets and does that supply to
decision supporters, we had identified those as issues long before we started
talking about decision support.
Now, certainly, you know, you could do an attachment, you know, with free
text fields, but it would be optimal if we could have, you know, codes instead
of free text fields.
DR. COHN: Thank you.
MR. REYNOLDS: Okay, Suzie?
MS. BURKE-BEBEE: You answered a couple of my questions, but one that I want
to ask about your problem list, I see allergies is on it. Was there any
discussion in the work group about contraindications or indications?
MR. SCHUETH: Yes, that was another challenge that we have.
MR. REYNOLDS: Jeffrey?
MR. BLAIR: Thank you. In a sense, my question is similar to Simon’s because
if there’s any possibility of being able to include prior authorization or SIGs
in the pilot test for next year, you know, it’d just be wonderful to be able to
add that in.
In listening to your testimony, you know, it sounds like neither of you will
be ready by January the 1st.
And then there’s aspects and pieces that I’m hearing that are just a month
away, two months away, or like GELLO, a year away.
You know, all of these things are attractive. Maybe we can’t get GELLO in
the pilot test, but I’m wondering if each of you are able to articulate some
type of a vision where there’s some aspect of SIGs, some aspects of prior
authorization, that might be ready for implementation and testing two months,
three months, six months after the July 1st, 2006, beginning of the pilot test.
I’m not sure that this is realistic, and I guess Maria maybe can’t comment
on it, but my thought is that if there’s a whole year, then maybe there’s a
chance for some things to get phased in after some of the testing begins, so
maybe we shouldn’t preclude something from being part of the pilot test just
because it’s not ready on January 1st.
So, Tony and Laura, could you each comment as to whether you’re able to
visualize a date somewhat later in 2006 where it might be possible for the SIGs
or the prior auths to be part of a pilot test?
MR. REYNOLDS: Could we hold your question? Maria has a comment on this and
then I’ll let you talk.
MS. FRIEDMAN: Just two quick comments. First of all, we would certainly
invite — these are the kinds of things people need to be thinking about, and
if people apply would include them in their proposals. And of course there’d
have to be specifics about how those might be addressed.
But the second thing is that we’ve had a lot of discussion here in the
Subcommittee about a lot of things need to be pilot tested that couldn’t be
done in time for the MMA pilots in January, 2006, and there certainly needs to
be a research agenda where these things need to be tried out, both in public
and private sectors. But that’s just a thought, because there are so many
things that are emerging now, we can’t do them all in January, 2006 — there’s
But they certainly do need to be tested before they are ready for prime time
and implementation, so I encourage people to think about those kinds of things
and how we might address them and what other opportunities there might be for
MR. REYNOLDS: Okay. Peter, do you have any comments?
MS. FRIEDMAN: I think Mike wanted to go next.
MR. BLAIR: Could I ask —
MR. REYNOLDS: Wait a minute — I held up there the answer to your question,
MR. BLAIR: Yes.
MR. REYNOLDS: — so Maria could comment. Let me let them answer and then —
MS. TOPOR: I think, based on, you know, Maria’s comment about just what the
industry is ready to absorb for January 1st, has been, you know, one of the key
issues we’ve come across in — you know, this is great, we think we can do it,
but I can’t do it by January 1st unless somebody tells me I have to do it by
I would anticipate that if we can resolve some of the code set issues that
we’re struggling with and that Tony also clearly is struggling with, if we can
get those resolved over the next few months, I would expect that, you know,
towards the end of first quarter next year, possibly the beginning of second
quarter, from the standard SIG perspective, we should be in good enough shape
that we’re ready to go to include in some of those pilot testings.
That’s what I’m hoping for. I was actually hoping we’d be done a little
sooner but as we delved into it, it grew, and there were too many questions and
we didn’t want to rush through and not deliver something that was going to be
MR. SCHUETH: And my perspective is that there are things that can be pilot
tested out the gate and then things that would have to be phased in over time.
Because it’s workflow to transactions, we’re talking about several pieces to
this puzzle, and I think, you know, as I tried to represent, several of them
are already, you know, operational, balloted, even considered as foundation
standards — the telecommunications piece, prior authorization between the
pharmacy and the payer, formulary and benefits, SCRIPT, it’s in
telecommunications, and others.
There’s a lot of this that’s live and operational today. The HL7 PA
attachment will not be ready by January 1st, but that could be phased in. And
the whole idea of clinical decision support and criteria in the doctor’s office
won’t be ready. It simply won’t be ready. Analysis could be done, and it could
be, you know, a subsequent pilot.
MR. REYNOLDS: Jeff, did you have a follow-up on the comment?
MR. BLAIR: No. Thank you. You’ve both answered my question. Thank you.
MR. REYNOLDS: Okay. Michael, and then Stan.
DR. FITZMAURICE: I guess I’ll preface some of it here and then ask a
I agree with what Maria said, and if we look at how the pilot testing is
worth — we look in the world for things that have sharp edges and time
barriers and timelines. I guess in an ideal world we would want to have some
way to work with people who would propose in this competitive process that was
mentioned how to do the pilot testing.
Imagine that you have to pull together an electronic health record or a
prescribing software and get the links to prescribers, pharmacies, PBMs or
health plans, the same kind of mapping you did, Tony, and then as new stuff
comes along, you have to have a bank of programmers ready to insert modules
into their software, and it’s hard to do that on the fly.
When you have only a year to do pilot testing, it’s really hard. You have to
make the opportunity to do this, and that’s more not so much a contract and pin
down but being flexible, being open and willing to try to make this work and
have as much value to the pilots as is possible.
So there’s some responsibility and foresight needed on the part of the
government, on the part of the industry, and some flexibility needed by the
people who eventually will be doing the pilots and the evaluating of the
So I just wanted to say that the more flexible we can have that process and
the resources available, if they would permit such a process, then I think we
can put a lot into testing. It’s not going to be easy.
It’s not easy getting here, and I applaud the hard work of the committees
who really tried to make these things fit together. But we’ve done hard work
before, and so we can do hard work next year to get these pilots testing as
much as we can.
And the MMA pilots end at the end of the year, but it may create an
opportunity for the industry to continue testing things they think will work
and will help save time and money. So it may be the start of a process,
although we envision it as here’s a year’s window that we have to do the
pilots. It may be something that can continue.
My question. Laura, when I looked through everything that you did on all of
your pieces of paper, I’m looking at the code set for this, code set for that,
code set for this, and I’m thinking, my gosh, that’s a lot of code sets. I’m
used to dealing with like their CPT 480s, but a lot of code sets and answers to
the questions — well, what runs through my mind is: Who should develop those
The answer probably would be many of them are already developed that have
many standards, so many code sets. Which one should be chosen? Who should
maintain the codes and try to harmonize all of this? Who should pay for the
maintenance and the help desk needed to answer questions when people come about
trying to use the codes? Or when they have a new thing that would work well
with the code set, you’ve got to go to some editorial board or some place that
is working with the code set.
There is just a large number of code sets. And I saw code sets in what you
have, too, Tony. Is there any thinking about should there be an organization
and institute, an existing SDO, should it be one of the major code set
maintainers out there and give that person or that organization more work to
It’s got to be more a matter of what does the industry want and what does
the industry trust rather than the government solves the problem. But with all
those code sets, I don’t know what the solution is. Can you help me out on
MS. TOPOR: No, because I was hoping you were going to tell us what to do.
MS. TOPOR: And that has become the biggest challenge that we’re facing right
The struggle is there are a number of code sets that exist. For SIG, SNOMED,
probably we’ll get 95 percent of the way of where we need to be, and they’ve
indicated their willingness to go ahead and just create any new codes that we
might need. So their support has been invaluable.
Where we’re struggling is, is that the path that we think we should go down,
is that the path we want to go down, of designating one code set or code system
for every field that we have out there with a code? Do we want to look at
maintaining the flexibility to say, you know, NCPDP has codes that’ll work in
this field and HL7 has codes and SNOMED has codes, so here’s your code
qualifier. And if you’re going to pick SNOMED, you, you know, code it this way;
if you’re going to — and that’s what I think our current struggle is, and
then, again, trying to incorporate, you know, a lot of the discussion that
Lynne talked about with RxNorm.
You know, I was a part of that call, but because product isn’t part of SIG,
you know, that piece was pulled out. So right now I think that is our biggest
struggle, to find something that works domestically and internationally that
everybody can accept and can implement, and we don’t know yet if it’s one
system or if we’re going to need to incorporate the flexibility to support
multiple code systems.
DR. FITZMAURICE: So it still remains a problem, yes?
MS. TOPOR: Mmh-hmm. I’ve got a call tomorrow afternoon, if anybody wants to
join in and help solve.
DR. FITZMAURICE: I also want to make a comment
on GELLO. I’m used to seeing (?) evolved into GELLO to put this, then this,
statement for clinical decision support, and I agree with what Tony said.
It looks like a natural pathway for GELLO, and I’ve also talked to Ross
Martin about this, that if you can have these statements put into a common
framework and if the industry takes it and uses it for prior authorization,
then you’ve got a framework that everyone understands, and it could make the
pathway easier for clinical decision support because the framework is already
I don’t think we know at this point. I don’t see GELLO in production
anywhere, as Simon brought up, but I think it’s worth trying to see if it can
fit and to see what molding has to be done, just like we’re molding with some
of these other — I think it’s an opportunity to support testing this out.
MR. SCHUETH: And, Harry, I want to clarify one thing that I said a second
ago. I said that what will be ready and what won’t be ready.
What won’t be — GELLO won’t be ready to be pilot tested by January 1st. But
that doesn’t mean that presentation of criteria as part of the process couldn’t
be tested. That could possibly be tested.
So it’s GELLO that won’t be tested, and the task group, you know, thought
that GELLO might be a solution.
But again, it wasn’t fully vetted through the entire — on the call was, you
know, a subset of the task group; it wasn’t fully vetted through all this.
So, you know, Maria, you may receive a proposal where somebody says, look,
you know, I don’t need to do GELLO where I can, you know, actually put these
criteria in the doctor’s office so that at least they can, you know, take a
step to making that request.
So there may be something — you know, I just wanted to clarify that. GELLO
won’t be ready, but it doesn’t mean that the criteria couldn’t be presented.
MR. REYNOLDS: Stan, you have a question?
DR. HUFF: So, I wanted to ask you for a clarification on one statement, make
sure I understood, and then make a comment.
I think you said sometimes the rules are not placed on the form. My
understanding of that basically was that the benefit providers are not making
public their rule for this pre-authorization. Did I understand that correctly?
MR. SCHUETH: Based on our analysis of 350 forms, that’s correct. Some of
them do and some of them don’t.
DR. HUFF: I guess the thing that came to mind — I mean, that, from the
provider’s perspective, is maybe where some of the frustration is, and it’s the
opposite side of the coin.
So, recognize in your first statement, you know, that this isn’t an attempt
to usurp the coverage of the decision makers. I think, you know, the provider
side of that would say, yes, but we need transparency. I mean, this can’t be
the sort of thing where somebody in the back room is watching this month’s
receipts and says, oh, we need to change our rule — or if they do need to
change the rule, they need to do that transparently so that it’s obvious to the
provider because, I mean, that’s the heart of the suspicion the providers have
about this process from the start, that what we’re going to do is make you jump
through hoops just to prevent you from knowing how this can be done and, you
know, and so this kind of obscurity of what the real rule is doesn’t help or
play to really clarify that issue, so — but that’s interesting.
MR. REYNOLDS: Simon?
DR. COHN: Yes, and Stan, I think you do bring up a good issue. I’m actually
sort of reminded that of course we don’t have Clem McDonald here in yet — he
won’t be here till this afternoon — because I think early on he made some
Certainly, I have no idea whether it’s intentional or unintentional, but
obviously the question will get called as we sort of move into these areas.
I guess there was really sort of one comment, and I guess I may have
forgotten my question at this point — or actually really I think it’s two
One is that, just as a sort of a comment to the Subcommittee itself, Mike, I
think, made allusions to the issues, the fact that MMA is one pilot in one
specific set of time, and it may be worthy for us to consider whether there’s
any advice that we may need to give the Secretary or the Administrator of CMS
about the recognition that there may need to be some sort of ongoing pilots in
The other piece, of course, is that what you’re doing may be of enough value
to the industry that the industry may just in and of itself decide to do
pilots, because clearly automation of the things we’re talking about may have
such major business cases that people will just go out and start doing them
anyway, and those are things that we need to consider.
Now, the other piece, and I’ve just been hearing this now from both Laura
and Tony, and there’s a bullet here, I think, Tony, in your slide where it
says: What can HHS do to help? And it says “central information code set
repository.” And I think I’ve heard that from both of you. I’m actually
sorry that NLM isn’t here today, this morning, to in any way address that
because it’s something we should talk to them about. Certainly, I think our —
MR. BLAIR: Vivian Auld will be here later.
DR. COHN: What?
MR. BLAIR: Vivian Auld from NLM will be here later.
DR. COHN: Well, that’s right. There may be something we really want to talk
to her about. Certainly, I think all of our views are that people should not be
at this point going out and making it up on their own, if at all possible.
Certainly, I think we would all observe that there are things like the
consolidated health care informatics data code sets and all of that. There have
been pronouncements by the Secretary of various code sets that are sort of
national standard code sets.
And so, you know, on the one hand I would hate to see in every field that
you have you say “SNOMED” and that’s all, you know, knowing that
SNOMED has hundreds of thousands of terms. One would hope you’d get a little
more specific than that. But I would say certainly it would be helpful if one
started looking at code sets that had been out there and identified as national
standard and then only if those don’t meet the needs, then you need to extend.
Just a comment.
MR. REYNOLDS: Lynne?
MS. GILBERTSON: One of the things that has come up in multiple task group
calls is the SDOs don’t want to be code set maintainers. And I hope I’m not
misrepresenting anyone, but we’ve had representation from X12, from HL7 on
different calls and they said that’s really not the business they’re in, and
especially when you get into the distribution and things like that. It’s one
thing to have a list in your data dictionary, but it’s another to be a code set
And part of the problem we’ve had, too, is, you know, scanning the Internet,
Googling, trying to find who might have code sets of things, just trying to
figure out: Has anybody touched this area before, you know? And asking the
representation on the task group list: Does anybody know of anyone who’s
playing in this space? Because we don’t want to go out and reinvent the wheel,
but to get something out to press, we may have to, and that’s kind of a shame,
that, you know, are we the first group to have thought about that or are we the
only ones who have to get something done, you know?
MS. GILBERTSON: I mean, I don’t mean that negatively, but, you know, if
you’re going to try to get something out there, where is it coming from? You
know, we’ve gone ISO, we’ve gone ANSI, we’ve gone different places like that
looking as well.
MR. REYNOLDS: Jeff had a comment on code sets.
MR. BLAIR: Laura, you had indicated that you have a lot of frustration with
the code sets, not having an answer for that? But then you also wound up saying
— I thought I heard you say — that SNOMED has agreed to address 95 percent of
the code set needs that you’ve raised. And I’m trying to reconcile the two
MS. TOPOR: Let me see if I can clarify. What SNOMED has will — for the
fields that we have identified where we want codes, SNOMED has a code for
almost everything, or will create the codes or the variances that we’re looking
What the struggle is — to Simon’s point — do we want to put something out
there where the only code system option for implementation of the SIG standard
is SNOMED? And until this is named as a mandated standard, I’m concerned about
barriers to adoption where, you know, when you look at the players on this from
a prescriber perspective, depending on the size of the group practice, a lot of
them already are using SNOMED; it’s already embedded in our legacy systems or
in the systems that we’re implementing.
From the pharmacy perspective and, you know, as the recipient, they’re not
using SNOMED. They don’t see it now. They probably never heard of it, which I
know I hadn’t till I started this project.
So it’s really just trying to find the balance to say: What can we do to
make the voluntary implementation of this the most successful and widespread?
MR. BLAIR: Let me ask for a clarification.
Since the Secretary of Health and Human Services announced I guess it was in
May, 2004, the CHI standards that Simon just referred to, consolidated health
informatics initiative for DOD and VA and HHS, kind of as an example to the
rest of the industry and SNOMED CT was identified as one of the core
terminologies, that may be as far as the Federal government may be willing to
go, because mandates might not be what you need.
Did you really mean mandates, or did you just mean Federal government
recognition? Because there is Federal government recognition for SNOMED and
LOINC and RxNorm — well, I’m not sure whether RxNorm is in there yet; I think
it is. Because if mandates is what’s needed, then that may not be on the
MS. TOPOR: Quite honestly, I don’t know if we need a mandate or if we need
more widespread acknowledgment of that recognition. And again, the potential
barrier is from the pharmacy community because, I mean, they don’t really deal
with a lot of code sets today from the, quote/unquote, “medical
perspective.” They’re not doing a whole lot with CPT codes. They’re not
dealing with some, they don’t see those things.
So that’s where I think there’s still work to be done and whether it’s an
educational campaign or I’m not quite sure what we need to do to say: You know
what, pharmacies? There is just not going to be an NCPDP code set for this —
which is what you know and love and are used to, but here’s the code set,
here’s the source — you know.
I think the costs associated with access to the SNOMED database are
relatively reasonable but I’m not a solo practitioner in a rural pharmacy on
the Iron Range in Minnesota so I don’t know if that cost is cost prohibitive or
But those are just a couple of the issues we’re still struggling through.
MR. REYNOLDS: We have a comment from Stan and then one from Maria and then
we’ll take a break.
DR. HUFF: So I need to preface this with I have a potential conflict with
HL7 because I am a co-chair of the vocabulary technical committee of HL7, but
just a couple of things.
One is SNOMED has the codes, and one of the common issues has been, though,
that you have the codes but they’re not a recognizable subset so they don’t
have a name collection that corresponds exactly to this set of — so you get
the idea, for instance, if you were talking about methods of birth control, you
know, you could have some of those things that are behavior, some that are
medications, some that are barrier methods, abstinence, and those things are
scattered different places in SNOMED and there may not be a name collection
that meets the need of this specific field.
And so it’s more than just saying there’s a code that exists for this. You
have to know the exact subset. And there’s been a lot of work that goes on
The second thing. SNOMED has been recognized, and we have the license within
the U.S. The organizations that are dealing with this oftentimes are
international, and the SNOMED approach for funding is not clear for Canada, for
Australia, for other countries, and so when you start talking about these
things within the SDOs, it becomes problematic that SNOMED in fact is not free
for use worldwide, and that becomes a barrier to adopting that as a solution
And the U.S. can be separated from that, but the international issues come
into it because HL7 is international, ASTM is international, and so that, you
know, presents some problems there.
Third is I agree we should talk very specifically with the National Library
of Medicine but again, from my position as one of the co-chairs of HL7
vocabulary technical committee, we currently have a contract, HL7 has a
contract, with NLM. One of the parts of that contract is in fact to investigate
whether the NLM could in fact be the distributor of these code sets, value
One of the reasons for that is in fact that we recognize that it’s
ultimately going to be a combination of LOINC codes and SNOMED codes and other
things, and what you’d like to do is in fact have a sort of a one-stop-shopping
thing where I could go to this place and get everything that I need to
implement, you know, my EMR or my NCPDP interfaces or whatever.
And so I think it needs to be bigger than SNOMED or LOINC or something else,
and you want something in fact that you can bank on in the long run and is not
susceptible to company or even a volunteer organization’s viability. These are
now becoming important enough for the infrastructure that I think we need some
really permanent, well-funded — it may be private, but if it is, it needs to
be well understood exactly how that happens and how that’s going to be viable
for not just the next five years or ten years but for the next 50 or a hundred
And so I really think there’s an argument about why this should become a
government institutional place, but certainly we want to understand how that’s
funded and how it’s regulated or governed.
MR. REYNOLDS: And Maria passes on her comment, but thanks, Stan.
We purposely let this questioning go over a little bit because with this
much data in this important a subject, let’s go on a break and then come back
and try to pick it back up.
So, thanks to all of you — excellent job. And we’ll take a 15-minute break.
MR. REYNOLDS: Could we go ahead and get started please?
Okay, let’s get started on the second part of our morning session, and
we’re going to get an update on HL7 and NCPDP SCRIPT harmonization from Ross
Martin, a familiar face to the Committee, and Lynne Gilbertson. So, Ross, if
you’d — thanks very much.
AGENDA ITEM: Presentation — DR. ROSS D. MARTIN
DR. MARTIN: And Jim McCain was just here who’s representing HL7, and he
stepped out for a second, but if you see him wander back in, please tell him to
pop over this way, too, because he may have some comment.
MS. GILBERTSON: Come to the front of the room!
DR. MARTIN: Jim, if you could join us for a second, that’d be great. Good.
My name is Ross Martin. I’m a Director of Business Technology at Pfizer and
also a Member of the Board of Trustees of the National Council for Prescription
Drug Programs, and I’m going to be presenting a third update to the Committee
on the NCPDP-HL7 Electronic Prescribing Coordination Project.
I’m happy to be doing this. I’m grateful, again, for your real impetus in
making this happen. You were the ones who said, well, by golly, you guys should
get together and do this, and it sort of put the fire under us to get working
on this in a more concerted effort. So, thank you again for your encouragement
and continued support.
Just a comment about the slide, the opening slide, here. Maybe you’re
wondering why this thing is in purple. Well, blue is the color of NCPDP and red
is the color of HL7, and so we figured for the Coordination Project our, quote,
“logo” would be the combination, which is purple.
MR. BLAIR: How politically correct!
DR. MARTIN: Yes. So Barney had nothing to do with it.
DR. MARTIN: So the summary of the prior update that I gave to you back — I
think it was in December of 2004 — was that we began this Coordination Project
last year, last summer. We had long recognized this need, but really hadn’t
developed it as a process yet, and met back in I think July, just before one of
the NCVHS hearings, because we were all in the area.
From there, we started with just 16 participants and grew to a group of
about 54 that are currently participating at the Yahoo! group’s site with a,
you know, subset of that that are actively participating in the calls, in the
process. And I’ll get into that in much greater detail.
I think in our last hearing, at the last hearing where we testified, we were
saying that we were going to be doing a demo, a demonstration of the mapping
process, and since that time we’ve completed that, and I’ll get into that in a
bit, and then just talking a little bit about where we’re trying to go from
So, you may recall this slide from our original presentation. These were the
16 people from the different organizations that participated, including three
of us who were designated liaisons between NCPDP and HL7. That would include
Jim McCain, myself and Karen Eckert. And so we had fairly good distribution of
folk from NCPDP and HL7 at that first meeting.
From there, we had pretty active involvement, as of December, of a larger
group of folk, and then, again from there, we now, as I mentioned, have 54
people who have subscribed at some level to the Yahoo! groups, maybe just to
monitor it to make sure that they can see what the documentation is going
But we established a regular call schedule that actually involved two calls
per week for many months. One call was for an hour, one for two hours. And that
involved — what we found was, in order to really make the project work, we had
to have a certain subset of those people who were from both HL7 and NCPDP who
were considered essential mappers, if you will, and if they weren’t on the
call, if we didn’t have a subset from both HL7 and NCPDP represented, we had to
not have the call.
We went through a couple of personnel changes in terms of the overall
administration and project management of this, eventually settling on someone
from now Accenture, previously from — they came on behalf of Café Rx
and now from Accenture, and that’s Kevin Deysenroth, and I will get into it a
little bit more about what his role and the support that he provided for us.
But you can see the asterisks next to the names were indications of people
who were very critical to the mapping, the day-to-day mapping, of the
And, by golly, I forgot to empty that box out, and if I could ask Shelley
Spiro, just over to your left, that’s a box against the wall that has copies of
the actual mapping document that I would like to distribute to the people
around this table. If you’re a NCPDP or HL7 member, you may have a copy of it
as well. If you are neither, we ask you not to take a copy at this time because
this is a draft document that is considered at this point a non-published
proprietary document. But we did want to make it available to the Subcommittee
and staff because it’s important, I think, just for you to kind of get a sense
of what we’re talking about here in terms of a work product.
MR. REYNOLDS: Process check. Marjorie or Simon?
If it’s a non-published proprietary document that’s about to be handed to
us, that I believe —
DR. MARTIN: Are you obligated to post it?
MR. REYNOLDS: I believe — does that change it to a — you might want to
hold passing them out for a second. I don’t want to — in other words, if
you’ve got members of the Committee, they’re a group that may not see it. If we
see it, I’m not sure — I need some help.
MS. GREENBERG: I’d probably prefer not to — I mean, when you said
proprietary, is it because it’s pre-decisional?
DR. MARTIN: It’s because it hasn’t been balloted, it hasn’t gone through
that process. And also, because of the nature of how we publish standards, they
are technically owned by the standards — they’re copyright by the standards
organization. And in the case of both NCPDP and HL7, one of the ways that they
make money as an organization, as a nonprofit organization to sustain
themselves, is through the publication of their standards.
So it’s a benefit of membership. We make them available to participants of
the standards process, of this mapping process, whether they were members or
not, if they were considered to be an expert that needed to be involved. I
think in every instance pretty much everybody was a member of one or the other.
So if that’s an issue — if you could just —
MS. GREENBERG: If this is something that would be helpful to the
Subcommittee to see to understand what your process is — I mean, I think in
that case, it can be given to the members and the staff, and we would not
distribute it elsewhere.
But I guess ultimately someone could make a Freedom of Information request,
probably, because there are exemptions to those requests because of proprietary
materials — I mean, I think rather than saying no, you can’t look at it,
because I think you may need to to advance your work, then I guess we’ll
proceed that way.
DR. MARTIN: I just want to be clear. This is not like these are trade
secrets or something or, you know, nobody — three people in the world can even
read this thing, but —
DR. MARTIN: I think the point in even showing it to you is this is a
double-sided document of 130 pages — well, 65 pages, double-sided, so it’s 130
pages of text — and it didn’t exist before this mapping process began, and now
this is, again, in draft form, but every word in that pretty much had to be
written or drawn from existing documentation modified to talk about how these
two different standards talk to each other.
MR. REYNOLDS: Ross, I just didn’t want you to get caught! [Laughs.]
MS. GREENBERG: If anyone has insomnia —
MR. REYNOLDS: I wanted to make sure — we didn’t want to put you on a
billboard on the highway. [Laughs.]
DR. MARTIN: This is more being central to the SDOs that, you know, own this
product and so — thank you so much for working through that with me.
We did feel like it was important for the Subcommittee to understand a
couple of things about this. We’ve already heard about the volunteer efforts of
the participants in these projects. I think this one is
particularly special because nobody’s going to make money off this one.
There is no revenue stream. Maybe, I guess maybe SureScripts, because it’s a
transaction, they can make a quarter off of, you know, every time somebody
sends one of these, but even if it’s massively implemented, this will be a very
small subset of the overall e-prescribing traffic.
But it was recognized as a critical part of the safety path, of
miscommunication that happens today. And so, for example, Kevin Deysenroth from
Accenture, he was mentioning on a call the other day — and this is just one
example of many — where — he’s a consultant, you know, a big consulting firm;
he earns his keep by billable hours. And this project, which he was part of
project managing, he was not a subject matter expert. He was facilitating the
process. He was the one manning the Web-x and the phone calls and making sure
that all the Minutes got pulled out and all that stuff.
And he basically had to kind of put this one under the radar. And all of us
were in that situation where this is not a project for anyone to do for a
business reason directly. It’s more it was the right thing to do.
So I think especially the people who dedicated an inordinate amounts of time
— I also want to point out in particular the woman to my right, Lynne
Gilbertson, because when we didn’t have funding for a documentarian, someone to
write the guideline, the guidance document, Lynne stepped in and served as
primary author. And I know that while that’s part of her function as staff, we
all know that if you start adding up the hours of calls that she’s on, and if
you’ll notice that her voice is hoarse, I don’t think it’s because she’s been
screaming at a football game or something. It’s because she’s working
tirelessly. And if there is some, you know, medal of honor that can go to the
unsung heroes of the world that you guys can recommend, I would suggest that we
So, getting back to the slides, we did do two demonstration projects in
2005, in February and March, one at HIMSS, the Health Information and
Management Systems Society, conference in Dallas, Texas. It was part of the HL7
demonstration group. And one at the NCPDP annual conference in Phoenix that was
actually sponsored by Pfizer and supported by both HL7 and NCPDP.
The demo participants, these were the ones that actually, you know, paid the
fees to show their stuff at the conference or were part of the collaboration to
do this. The Cleveland Clinic and Epic systems, HealthSoft Applications,
InterSystems Corporation, NDCHealth, NextGen, Healthcare Information Systems,
RxHub and SureScripts all participated in those booths in a very active way,
had to spend time demonstrating this to anybody who walked by.
The scenarios that we primarily focused on were ones involving the scenario
where medication orders were being created for a discharged patient from a
hospital setting or an emergency room, for example. And that prescription,
instead of being sent to the inpatient pharmacy where it would normally be sent
through their computerized physician order entry system, was being sent to a
And that’s the fundamental use case of this mapping project. You have to be
able to talk HL7 in your own internal context and then transfer that to a
retail or community pharmacy environment where SCRIPT is the dominant player,
if you will.
But this could also be related to an electronic medical record product that
uses HL7 for their e-prescribing tool and wants to do the same thing. It also
would be in the ambulatory space where an electronic prescribing tool that
normally speaks SCRIPT has to send something to a specialty pharmacy, for
example, at a hospital, maybe for an oncology drug or something like that, but
they would have to be able to talk in the other direction.
We also did medication histories that went from the pharmacy benefit
management claims history database using SCRIPT, and that’s actually no longer
soon to be balloted SCRIPT, but it’s been balloted, and translated into an HL7
One of the vendors presented the use of a smart card to be able to deliver
that medication history to an e-prescribing tool, to a physician in an
So this next graphic just shows the overall demo scenario, and it shows the
players and the transactions that were demonstrated using these different
So, one could go around the booth and kind of watch this information move.
Normally, this is the kind of stuff that is so behind the scenes that you
really never get to see it in action because it’s very transparent to users; in
fact, most of them would never know that there’s any translation going on, no
concept of that.
This is just a picture of the HL7 demonstration booth, and again it was
part of a larger — there were many other sponsors and participants in the
demonstration booth there at HIMSS for the HL7 booth, but this was sort of the
primary role in that and we also had live theater events at that where, you
know, different individuals would get up and present at that.
And here’s a photograph of the actual HL7/NCPDP booth at NCPDP, at the
annual conference, which is a much smaller exhibit hall. HIMSS is a vast thing.
So this got actually I would say probably more focused traffic at NCPDP than at
So in terms of a timeline of where we’re going with this, we completed
these two demonstration projects. We’ve now completed our work on the mapping
document that you have in your hands now. This has been distributed to
stakeholders at HL7 and at NCPDP. They’re currently reviewing it, and any
comments we would receive back, we will, quote, adjudicate at the task group
level in August.
And then, if things go well and there aren’t a lot of things that we have
to fix and there’s not a lot of controversy there — I’m imagining that most
things will be process oriented or typos and that sort of thing — we will
release this in September, hopefully, September 1st.
And then it’ll go through the publishing process of the individual
standards organizations, and I’m assuming that the general thing will be if you
access the HL7 pharmacy stuff, you can get a copy of the mapping document along
with that. If you access the NCPDP SCRIPT standard, you can get a copy of the
mapping document from that direction. So you don’t have to join both
organizations or buy it from both organizations to license this tool, or this
And another just point is that it is guidance; it’s not a standard, because
it’s mapping to things that are implemented in different ways. Especially on
the HL7 Version 2 side, there’s a thousand ways to implement that, as this
Subcommittee has heard on many different occasions about the general delivery
of standards in the marketplace.
As you know, as you know well, pilots begin in 2006, so we’re anxious to
find out what CMS does have to say about the role of this mapping project in
those pilots. We think that it’s ready for that environment. We think, in fact,
that’s an essential part of this process because in theory we have the theory
on paper now, and this is based on prior work from a number of stakeholders,
including Cleveland Clinic and Epic and NDC and RxHub.
There’s a lot more to prove. There’s a lot more to prove that this can work
in many multiple settings, and I’ll get into that in a bit.
I’ve already mentioned that there were lots of individuals that contributed
to this. We tried to do some sense of how big of a bread basket was this, and
with the conference calls, consistently we had about a dozen people on those
calls. You know, thousands of hours that happened off the books when we weren’t
in a call, when people were doing the homework that they were given between
that call and the next call.
The demonstrations, everybody had to contribute, you know, had to sign up
to participate in those demonstrations and pay registration fees and, you know,
just the participation of that involved many, many hours.
So a very conservative estimate of the billable hours, and I really think
that this doesn’t account for the true capturing of the costs, is about
$300,000 so far that corporations have basically donated to this process, and
So we wanted to share with you some of the lessons that we’ve learned in
this process and things that not only apply to this project but perhaps other
standards development — especially standards harmonization processes — and I
think we shared this already in past testimony, that as hard as it is to get
volunteers for single standard organization-directed projects, when you’re
trying to harmonize two things where they don’t talk each other’s languages too
well, the reason that we didn’t go to Version 3 in HL7 in the first place was
because that was a whole another paradigm to understand and grasp and we
weren’t prepared to go there, so we really started with a Version 2 which I
think will help in future efforts should we go down that next path.
There is a need for ongoing support for project coordination. We could not
do this without somebody that’s not a subject matter expert but just make sure
that the calls get made, that the schedules get done, the Minutes get put out,
the process continues. You know, you make sure that the right people are going
to be there, all those — I’m grateful that there are people in this world who
really love to do that kind of work.
DR. MARTIN: It’s like I’m so glad that my accountant loves when numbers come
out right because I could never stand having to do that for a living. There are
things that I do that I’m sure other people, it would drive them mad as well.
But, thankfully, there are people who like to do this. They need to be paid
to do that, because there is no business rationale for somebody to do that work
other than to make this thing get done.
Use cases are helping to drive this process, and so it was very helpful to
have a stated goal about what thing in the marketplace were we trying to do?
And market readiness, like this is the next thing that we need to accomplish —
okay, we’ve already gotten these basic things done, the 80/20 rule; now let’s
go to 85, let’s go to 90 percent, let’s capture those last pieces of challenge.
So that helps bring resources to the table because the more the market is
ready, the more individuals and corporations and willing, you know,
organizations, entities, are willing to bring those resources to bear on these
I think the pilots, as I mentioned before, are critical for confirming the
real world utility of the mapping documentation. We don’t know what we don’t
know until we test it in non-controlled settings where things — you know, real
life happens, and new problems are encountered. And that helps us articulate
the refinements that have to go into future versions of this.
Again, I mentioned this a little bit earlier, but the document that you have
is only guidance. Every pathway to success will be different for every
implementation because there is so much variability in how these things are
As it’s been mentioned already earlier today on the two prior projects that
we’ve received testimony about, vocabulary issues remain a challenge. We
literally dodged some of the issues related to vocabulary in this project
because there wasn’t a way to resolve some of these things.
And I think some of the suggestions that you’ve already received are good
ones about the need for a real coordinated process where the work of the
standards developers is made easier because there’s a place that they can go,
and expertise like a librarian that knows how to guide them through the process
of finding the right vocabulary — a mechanism for people who don’t own a
vocabulary to make additions and modifications to that vocabulary or code set
so that if the existing code set accommodates 90 percent of their needs but a
couple things need to be added to accommodate 100 percent, that can happen
without having to own it, having to maintain the process for changing it, but
being able to inform that change will be very important.
This is critical for semantic interoperability, and I think I’m preaching to
the choir on this issue, so I won’t go further, but we welcome that
opportunity. And I think the RFPs that have come out from ONCHIT in these last
couple of months, early June, about standards harmonization, I think — at
least my personal read on those is that could be a real forum for this to
happen and a mechanism for us to make this a normal process where anybody who
owns the vocabulary, wherever it lives, there’s a standard developer and a
standard developer process for working with those together.
The fact that we don’t have an unambiguous patient identifier — we’ve seen
the Cleveland Clinic Foundation commented on the call when we were discussing
what we wanted to present — there are challenges with maintaining accurate
communications between prescribers and prescribing entities and dispensing
entities, pharmacies, because there’s not a necessarily natural place to keep
the medical record identifiers that are native to the prescriber’s environment
where they’re not native to the pharmacy environment.
They identify those patients in different ways, and because the payer
information, their member information, changes, a lot of the identifiers
change, those are some challenges with that, and so there’s a need — they
talked about the need to have a retain and return policy, so if you get an
identifier from an entity that’s specific to that entity, like a medical record
number and even a prescription number, that that be maintained with the
original prescription in the pharmacy system so that if they ask for a refill
or they ask a question, they can always send that information back.
And there are similar challenges for the provider identification and perhaps
a national provider identifier and the enumeration process with that will help
with that, but because these providers can change locations and contexts and
things, that remains a bit of a challenge.
Some of the things that we’re considering for future efforts include having
a common process for reporting — and again, these are more global lessons
learned about what can be gained from this project for other harmonization
projects. How are we going to report this? And the RFP around standards
harmonization, maybe that’s a place where we would do this regular reporting,
kind of like we do it at a certain level at ANSI HISB today — the Health
Informatics Standards Board. Perhaps there’s a place for this in this new
entity that may emerge from those RFPs.
Places for shared work spaces — I think we have this common process that we
see over and over, and that relates back to vocabulary and code sets. And some
of the tools that we’re seeing emerge from projects like from the National
Cancer Institute that actually have been very helpful in finding and
identifying potential solutions, the more that those things can be fleshed out,
those tools can be fleshed out and made available to standards developers will
be very helpful.
Again, the project management that I’ve mentioned.
Meeting support — that there’s maybe a notion that said if there’s an
SDO-to-SDO harmonization process, there’s a natural need for live meetings for
coordinated, you know, web conferences and that sort of thing. Everyone agreed
on our lessons learned call that having more live meetings would have made this
whole thing go a whole lot faster, and if there’s truly a sense of urgency
being built around these things, the support for live meetings, to get people
there, to get the experts that maybe don’t have the natural support from a
corporate or other organization sponsor to be able to show up at these would be
I mentioned already the potential role for the future recipient of the
standards harmonization RFP grant.
Then there’s this whole question — maybe, Jim, you could articulate or
speak on this just a little bit more — about this notion of a common model, a
common information model, for the standards harmonization process in the
One example would be using HL7-RIM as the thing that we all kind of build
toward and then look at that and make sure that the RIM is accommodating all of
the standards development organization efforts without having to be subsumed
necessarily by HL7 itself. I don’t know if you want to make any additional
comments right now about that or anything else so far.
I wanted to spend just a couple minutes talking about just the
recommendations for pilots for 2006, and I know we’re expecting these at any
point, but we do hope that we can test this mapping guidance document in
various settings, not just in large hospitals and emergency rooms but also in
small practices that have EHRs that use HL7 as their back end to demonstrate
the value of the message.
And this is an important thing — to isolate the impact of the transaction.
So, it doesn’t necessarily make sense to say, let’s pilot e-prescribing, and,
oh, by the way, we’re going to include the mapping.
We would really like to see this tested in places where e-prescribing
exists, and what you’re doing differently is, instead of having this thing go
to print or to fax, it goes to an e-transaction, a message that goes to the
pharmacy, so that you can isolate the impact of that thing. Does it make it
happen faster and more effectively? Can you delineate the cost/benefit analysis
of who has to do more work in that setting than not and who benefits from the
existence of that versus not? And so kind of get some ideas about where the ROI
lit exists and how we can accommodate for discrepancies between the return on
investment of the efforts involved versus the efforts required.
Then, you know, are there decreased call-backs? Are there increased
call-backs? Is there reduced staff time or prescriber time? Does it have an
impact on fill/refill rates?
There’s an opportunity to show either that there’s a decreased compliance
because the patient no longer has the paper reminder. I’m sure they’re going to
have some form or something that would say this is what you’re supposed to go
do, pick up your prescription, but there’s also now a hand-off to the pharmacy,
to the retail pharmacy, who has an incentive to contact that patient and say,
hey, your prescription’s ready, come get it, your refill’s ready, come get it,
which can perhaps impact refill rates.
Then just something I sort of made up, the notion of a semantic loss ratio,
or SLR. You remember the old game of “Telephone” that you used to
play as a kid where you took the people in the circle and you’d said something
at one end and everybody whispered it to the next person and then you found out
that it sounded something completely at the other end?
What percentage of this stuff, because we don’t have a completely
standardized code set because some of this stuff, we have truncation issues
where some of the fields in NCPDP SCRIPT have a certain length and they don’t
go over that, and there aren’t the same limits necessarily on the HL7 side,
what happens when you lose that information? Do you lose the semantic meaning
of it, behind it?
There are many issues like that that you want to be able to truly identify
and figure out, and one assumes that, because it goes into electronic format,
you have some ability to read it a little more accurately than what the
handwritten script might be, but what happens? You know, what are the issues
Some of the things that we’re considering for our future efforts — you
know, we’re taking a little bit of a hiatus and a much needed breath from this
project while we’re awaiting comments from the two standards organizations and
from you. Please do comment if you have anything to say.
But then we will need to look at this and try to maintain in our quarterly
basis, and Jim is very focused on how we’re going to do the ongoing maintenance
of this. Also, then, the next question is: Do we take the next step and map it
to Version 3? And there’s some, again, real advantages to that potentially,
even though it’s not in widespread use, because everybody’s looking at this
move toward Version 3. Is this an opportunity to have the one size truly fits
all because as you’re mapping the Version 3 from your old HL7 2X version, you
can have one way to get to NCPDP SCRIPT.
But as much as this required each person getting to know the other side of
how HL7 works versus NCPDP, a whole lot more education needs to be built in to
this in order to accomplish that task, so that’s a clear opportunity for
The next slide just shows the email address, how to subscribe to Yahoo!
Group, or you can contact any of the project coordinators for assistance on it
Again, thank you for your attention. Thank you for your support of this.
This has been a really — I’m just anxious to see the day [voice catches with
emotion] — hmm, finish like that, excuse me — when a life is saved.
I’ll stop there.
MR. REYNOLDS: Thank you. I think what’s exciting about what you just covered
is that there may be a life saved is one thing. But obviously, I think as you
look at it from strictly a business standpoint, the work that all of you have
done will allow people to take what they already have and continue to use it
and not have to reinvent everything that’s going on out there.
I think that was one of the key things that was discussed early on about
this whole mapping, letting the hospitals and others stay as they were and then
move forward. So I think that’s a great step.
I’ll open the floor now for any questions. Jeff?
MR. BLAIR: It’s going to get like a broken record, but, Ross, thanks for
tremendous leadership on this. I know that you didn’t do it by yourself and
you’ve given recognition to all of the folks that contribute.
In terms of the next step, and you were mentioning mapping to HL7 Version 3,
has any thought been given to whether that is a two-way mapping or a three-way
mapping? Is it just going to be between HL7 Version 2 and Version 3 or is it
going to have to be in addition to that, mapping between NCPDP SCRIPT and
Version 3? That’s one part of my question, so let me let you answer that first.
DR. MARTIN: Jim, feel free to chime in on this as well — inasmuch as there
is some level of mapping already between Version 2 and Version 3, we’d
certainly build on what exists there and hope that there’s very little to add
to that necessarily.
You know, as we dig into it, there may be some clarification of specific
needs that may be identified and may be unique to has to translate into NCPDP,
but the intent would be to focus on the Version 3 to SCRIPT and just, you know,
check the boxes to make sure that the 2 to 3 existing work is adequate, is
Is that fair to say, Jim?
MR. BLAIR: Okay. I’m sorry —
DR. MARTIN: I think Jim McCain wants to comment, too.
MR. McCAIN: I would just say that in the process that we’re going through,
the aspect of doing Version 3 mapping and so forth, back a year ago and so
forth there was basically maybe only one way that we could accomplish the
mapping to Version 3 and there were reasons that we did not do that mapping, as
Ross alluded to.
In conjunction to what he alluded to was the fact that the HL7 Version 3
medication, model artifacts, and the pharmacy message model artifacts were not
sufficiently stable at that point to warrant us going forward with a mapping to
those particular things.
That has now changed, and in conjunction with that, there are now multiple
processes that are being considered within HL7 for how to map from HL7 Version
2 standards to HL7 Version 3 standards.
So, the bottom line is there are multiple ways now that we can possibly
achieve the NCPDP SCRIPT mapping to the HL7 Version 3 mapping, and we are in
the process of having some experts provide us consultation on which method
would be the most efficient and probably the best, cost-effective way to
MR. BLAIR: Okay. The other aspect — I just made this three aspects — so
the second one. As you went through this process, did you find that the
predominance of the use cases were that you would be translating from HL7
Version 2 to NCPDP SCRIPT and that that was likely to be the bulk of the way
that the translations had to occur?
DR. MARTIN: As opposed to the other direction or as opposed to —
MR. BLAIR: As opposed to the other direction.
DR. MARTIN: Yes, I think that that was the more common scenario, because
there are many more settings where, you know, almost every patient discharged
from the hospital is going to have some medications to be on, many patients
discharged from emergency rooms, large institutions’ systems.
You have a relatively smaller number of pharmacies that would exclusively
work on the HL7 side where there would be a need to send information from the
I don’t know that anybody’s done any kind of objective measuring of the
transaction potential there; I’ve not seen that.
MR. BLAIR: Did this wind up resulting in areas that were identified to NCPDP
SCRIPT that needed to be enhanced because there was specificity from the HL7
pharmacy order that wasn’t quite there in NCPDP SCRIPT, so in a sense just this
exercise alone led to strengthening NCPDP SCRIPT?
MS. GILBERTSON: From what I recall, no. In fact, it went the other
Most of the needs were recognized to be taken to HL7 to include, for
example, some NCPDP code sets as part of their vernacular.
MR. BLAIR: Okay.
MS. GILBERTSON: Code sets were the biggest ones. I’m trying to think if
there were — I don’t remember data elements. Well, no, there were some data
elements because there were some address fields that were found to be missing,
and we shifted back and forth.
Originally, we were going to start with just like Version 2.3 was it, way
MR. McCAIN: Yes.
MS. GILBERTSON: And we found that we needed some fields that had been added
all the way up to HL7 Version 2.6 to get the functionality needed for some of
the transaction exchanges. So that was the good thing we found.
MR. BLAIR: So we almost got two for one out of this —
MR. BLAIR: — in that we not only had a mapping process, which is beneficial
for interoperability, but we also kind of set a new higher level of standards
that whichever of the two message format standards had greater specificity, the
other rose to that level. Is that correct?
DR. MARTIN: I think that’s a fair statement, and I think it’s consistent
with the overall observation that one of the benefits of what happens often at
example, being an international organization, even though you’re dealing
with very specific, realm specific needs — you know, pharmacy in the U.S. is
very different from pharmacy in Germany, very different from pharmacy in
Australia — but it forces you to abstract out to a level that accommodates
them all, and you see these common things that, oh, here’s the way to look at
that, that kind of gets at the pure truth of it all, if you will. And I think
that we observed some of that.
I guess the other side of the comment that Lynn made about more of it going
in the other direction, it’s not that HL7 doesn’t have a lot of detail in it;
in fact, a lot more detail in some areas. But it’s not relevant necessarily to
the outpatient prescribe, the ambulatory care setting, for what you need for a
Well, so much of the pharmacy model for HL7 deals with the administration of
drugs and other, you know, timing issues with that that just aren’t dealt with
in a prescribe notice in the ambulatory setting.
MR. REYNOLDS: Michael?
MR. McCAIN: Jeff, if I can just add to the comment about where you’re
talking what transactions you’d say — maybe more HL7 to NCPDP or vice versa
and to the notion — again, our use case is we’re primarily for the
institutional setting physician going outside.
But in the context of this particular Committee and the particular testimony
that we’re doing now on MMA, one side benefit of this is going to be — as you
know, there’s been recent public release of the Office VistA software, EHR
software, and so forth out to the general community. Well, that particular
product has fairly robust support of HL7. It has less robust support of NCPDP
Therefore, if you do get widespread implementation of the Office VistA
product out there, you now with this map and so forth, you already have in
place now at least a technical specification or document that the people out
there trying to implement Office VistA or the MMA pieces can reference to
MR. BLAIR: So it’ll kill three birds with one stone.
MR. REYNOLDS: Marjorie, you had a question?
MS. GILBERTSON: There is the other side of the equation, the third set,
whatever, of transactions, which is the medication history, which are the PBM,
the drug benefit program that speaks NCPDP sending information into the
hospital setting or clinic setting for the medication history information.
MR. REYNOLDS: Michael?
DR. FITZMAURICE: Well, I don’t want the Audubon Society to get after Jeff
and me —
DR. FITZMAURICE: — because we’re always looking for birds to kill —
DR. FITZMAURICE: — but as these mappings take place, does the RIM get
changed as a result of additional variables and additional findings as you map
to other standards? Is the RIM also a growing, living thing that rises to meet
MR. McCAIN: That’s kind of a complicated question.
DR. FITZMAURICE: Sounds like “no.”
MR. McCAIN: No, the answer is actually yes. But what you find is that it
depends at what level of the RIM that you’re talking about.
If you talked at the high-level RIM aspects, you’ll find there’s very little
that needs to be added. It’s when we dive down into creating the lower-level
model artifacts and so forth that we will bring into the harmonization process.
DR. FITZMAURICE: Well, like Jeff, I also want to give strong kudos to Ross
Martin and Lynne, Jim, all the people who worked on your committees, to bring
this about. And yet you said there’s not much money involved in this.
It’s mapping back and forth. And yet it’s going to make the system work.
It’s going to make sure that someone in the hospital gets the right drug when
they leave the hospital and go home.
Likewise, it can have the same effect on the hospital. It can add to the
medication history. There’s just a lot of things that can be done that isn’t in
the mainstream. This is really a critical linchpin for bringing together the
SDOs to work together and so that we can make the system work across our
Many congratulations for making that happen. It’s rare that somebody bursts
on the scene like Ross has and just has such an impact. He’s able to leverage
the good work of the people who are already working hard to get them focused
and go for additional efforts.
While I have you up here, Ross —
DR. FITZMAURICE: — you’re a physician and you make decisions, and I’m sure
you like to have decision support.
I don’t know if you looked at GELLO and R10(?) syntax, but in an earlier
discussion we had this morning, there was a possibility of GELLO being used for
prior authorization that is taking rules: If this, then this. What is your
opinion about that? Do you have a sense on whether GELLO can fit or not or
whether it is worthwhile to look at?
DR. MARTIN: Well, I would say — I guess it’s an unfair question because I
was the one who has been pushing for that within NCPDP and sort of instigated
yet another full activity.
I think it’s where we have to explore. Just for further clarification, while
GELLO has not really been implemented much of anywhere, it is now a balloted
ANSI-accredited standard, and that just happened in the last couple of months.
So I think there’s a real opportunity to test it. It may actually turn out
that it’s overkill. It’s possible that GELLO has so much potential robustness
that we don’t need all of that; we need a very small subset of it to do what
amount to fairly straightforward clinical criteria.
But there are a couple of pieces of that that, in order to get to this next
step of using GELLO within prior authorization, we need a compiler, we need
other tools so that you can express these things in HL7-speak, and that
I don’t know what the opportunity is in terms of piloting, as Tony
mentioned. I think we could do pilots in 2006. They just wouldn’t be the pilots
that lead to final CMS regulations. There are the pilots that get us further
down the road to having a finished product that would do that. So I hope
that we can put some real effort on this part of it.
The reason I thought it was so important to bring this particular use case
to GELLO in particular is because clinical decision support requires something
like GELLO, a way to express clinical concepts and criteria in a computable
And it’s not just exclusive to prior authorizations, for evidence-based
medicine, for any of these types of things. But the nice thing about prior
authorization is it simplifies this whole question of clinical criteria because
it’s a point in time, whereas clinical guidelines involve a process that could
span a year, two years, and things change in that. You’re asking essentially
yes/no, logical questions at a point in time, and if they’re all true, or if a
set of them are true, then you can get to the final yes, and that’s the real
And so that’s why the clinical decision support group at HL7 adopted this
project, because they saw that it was a great way to bring GELLO into the
clinical setting and make it work and then go on to these more complicated,
much more involved processes.
Does that address the question?
DR. FITZMAURICE: Yes. Thank you, Ross. Thank you.
MR. REYNOLDS: Okay.
DR. MARTIN: You’re welcome.
MR. REYNOLDS: I’d like to thank everybody. This ends our first part of the
presentation today on e-prescribing. We’ll start this afternoon on clinical
I’d have to say that the wow factor probably entered this discussion this
morning a lot more than I thought it would. You all, everybody that’s been
involved, has done an incredible job.
A lot of people think of standards organizations as having a general
direction. You have taken a very focused, laser-like look at the future and our
making a major, major contribution, so thanks to all of you.
We will resume at 1 o’clock. We’ll give you the hour for lunch. And I’d like
everybody to be here promptly. We will start at 1.
So, thank you very much to everyone, and, Michael, thanks for what you’re
doing on helping lead AHRQ help some of these people out, so thank you very
much for that.
MR. REYNOLDS: Thanks.
(Whereupon, the meeting recessed for lunch at 12:05 p.m.)
A F T E R N O O N S E S S I O N (1:00 p.m.)
MR. REYNOLDS: Take your seat, please, and get started. We’re going to start
our afternoon session, and Jeff and I have talked about this a little bit and
we’re going to change our mode of operation a little bit.
Usually when someone presents, we wait to the end to ask them questions.
But since this is a new subject to us and I looked through the charts last
night and struggled on a few of the acronyms and some other things –[laughs]
— as we try to learn this, staff and Committee, please put your hand up if
you don’t understand what an acronym stands for.
And then we’re not going to ask questions during the presentations, but we
would like to at least make sure that we’re following along, because about the
time you get six or seven acronyms that you didn’t understand, you’ve lost the
subject. So this afternoon we are going to change that process just a little
bit to make sure that we can kind of stay with it.
Before we get started, I’d like to thank Dr. Stan Huff for being willing to
lead this subject. Jeff did an excellent job with e-prescribing; Stan’s now got
secondary uses of clinical data, and we’ve got Judy on deck for some other
stuff here shortly. So we appreciate that help.
So without any further ado, I’ll go ahead and let Stan take over. Get the
AGENDA ITEM: Presentation — DR. STANLEY M. HUFF
DR. HUFF: Am I on now? All right.
The first thing I’d like to point out is that this idea isn’t original with
me. The NHII Roadmap, there are a few sentences that talk about this subject.
It talks about — actually I guess it was this Committee said, you know, a
comprehensive set of PMRI information standards can move the nation closer to a
health care environment where clinically specific data can be captured once at
the point of care with derivatives of this data available for meeting the needs
of payers, health care administrators, clinical research, and public health.
This environment could significantly reduce the administrative and data
capture burden on clinicians, dramatically shorten the time for clinical data
to be available for public health emergencies and for traditional public health
purposes; profoundly reduce the cost for communicating, duplicating and
processing health care information, and last but not least, greatly improve the
quality of care and safety for all patients.
So I think that this is a continuation of that thought. And I think that
thought also — I don’t think it was original with the NHII Roadmap. I think
it’s sort of an unstated sort of understanding and motivation for a lot of the
electronic health care records systems for the last
30 years. I know that’s been one of the goals, for instance, of the help
system that’s in use at Intermountain Health Care.
So I just wanted to point out this isn’t something that’s original with me
but something certainly I have a great interest in.
So, just a couple of definitions then, at least in the way that I use the
If there’s secondary use of data, then there has to be something that was
primary use of data. So primary use of data is the collection, processing,
display of data for purposes of taking care of a patient. And the way I use the
term, that’s care of the patient whether I collected the data in my institution
and I’m caring for the patient in this institution or whether I transfer the
patient to another institution and I send that data, using HL7 transactions, to
another institution, take care of the patient there.
That’s still primary use of the data, the primary use being the care of this
patient, this individual patient.
Secondary uses of the data really then fall to, you know, when I want to use
that same information to automatically drive billing, if I want to derive
statistics from it, I want to do quality assurance from it, I want to do any of
these other things. All of those are the secondary use of direct patient care
So, just to illustrate, this is an example that I used for Cancer Registry
data, and recognize this is just one example of 20 or 30 or 50 kinds of
But in the way that things happen now, if you look at clinical data flow
within Intermountain Health Care, we have data that’s coming from the
laboratory, we have data from radiology, we have data from clinicians, we have
pharmacy data. We have lots of ancillary systems that are contributing data to
an interface engine. We normalize the data and then we store it out into our
clinical data repository. And to the extent possible, we try and use standard
interfaces to do that.
That’s primary use of the data, because what we’re doing is it’s clinicians,
physicians, nurses that are looking at that data to care for the patient, to
make decisions about this patient’s care — what medications they should be
given, what diagnostic tests should be done, all that sort of stuff.
So that’s primary use of data.
If you compare that then to the Cancer Registry data flow, and I think this
is pretty typical right now, what you have for Cancer Registries in general is
that all of that electronic data becomes part of the patient’s chart
and then there’s some person who is manually extracting data from the charts
and in a lot of cases re-entering that data back into computer systems, and
then that data can go out. It becomes part of the hospital Registry. You can
use standard interfaces for that and it becomes part of the regional or
But the point is, that is secondary use of data now, because now we’re
populating the Cancer Registry for the purposes of understanding population
statistics relative to particular kinds of cancers and outcomes from cancer
treatments et cetera.
But that’s largely a manual process, and what we’re talking about now: Is
there something we could do that would automate that process so that we can
algorithmically define these things?
So what we’re talking about is a potential future state where the
information is flowing in from a laboratory, from radiology. It’s going in
through an interface engine, and the information is coming out and it’s going
through a — you can think of it as a filter or as a set of rules or a set of
algorithms — in an automated way into a hospital Registry.
Now, there should be no illusions that this unattended sort of flow, because
you’re still going to have people required to review and make sure that the
rules are doing what they’re supposed to do. And, you know, my guess is that in
fact the Registry person won’t be done away with. What they’ll be doing is
doing a different job — to review data — and a lot of the mundane things are
taken out and they can do a more thoughtful process to make sure that the data
is consistent and complete and that sort of stuff.
And so there’s a filtering process there, and then again we’re trying to use
standard interfaces everywhere that we can to decrease the cost of implementing
these interfaces and doing the rest of the work.
So the whole idea of this — and again, this is just one example of lots and
lots of different examples where what we’re trying to say is we’re capturing
this data electronically, rather than then having people read and process the
information manually. Can we set up rules that would allow them the assignment
of billing codes or the inclusion of this data into Cancer Registries or for
purposes of quality assurance? Could we do that in automated way so that the
data stays electronic and really improve the efficiencies and the timeliness
that we can produce these other secondary benefits?
So just to give you an idea, I wanted to go through a few secondary uses of
data that are in place at Intermountain Health Care. And these are all things
that I haven’t done; these are things that are being done by Scott Evans and by
Sid Thornton and lots of different people within IHC.
But one of them is adverse drug event monitoring. We originally got into
adverse event drug monitoring and we were doing it the way most people do it,
which is we ask people to report when there was an adverse drug event.
And then we started thinking and saying, well, how could we — and our
suspicion was that they were tremendously underreported — so Scott Evans and
others put in place and said, well, look, what if we looked at drug levels? So,
I mean, if we saw toxic drug levels from the laboratory data, that might be
indicative of an adverse drug event.
What if we watched and saw when Benadryl orders and Prednisone and other
kinds of antidotes were prescribed, any kind of treatment that would normally
be a treatment for a drug event, what if watched for those things in the
And what we found is that in fact electronically we could put rules in and
through that electronic surveillance we increased roughly tenfold the detection
of adverse drug events in the system, doing that.
We do nosocomial infection monitoring, and again, it’s a fairly simple rule.
And what it does, you know, the computer knows when the patient came into the
hospital; it knows their white count, if they’re afibrile et cetera, and the
system simply watches to see people who are hospitalized who then get a fever
or we watch the X-rays, and there’s a fairly simple natural language processing
on the X-rays that says, you know, is there a new infiltrate, new signs of
pneumonia? And it watches for those things.
And so a combination of knowing when the patient was admitted, that they got
a fever after they were admitted, their white count went up after they were
admitted et cetera, we have a way of detecting people who in fact got
nosocomial infection, that acquired an infection after they are in the
In the billing area, we’ve only done this in a small area, but in labor and
delivery, we have a fairly comprehensive labor and delivery program that
watches medications that are given to the mother. It produces a tracing of the
labor intensity and pressures and fetal heart rates et cetera.
And we have a set of rules basically that look at that data stream and
automatically assign the billing codes for that labor and delivery work so that
it’s a rule-based application of billing codes as opposed to our standard
We have a set of programs for reportable disease.
As all of you know, there are a number of diseases that we’re required to
report to the state health department, and it took a lot of manual work to
produce those reports.
Now, actually what’s happening today is that we electronically produce them
and then somebody writes them down and hands them off, again manually, to the
state, and what we’d like to do is in fact set it up so that we can just send
them electronically to the state.
But in terms of us gathering the data, what happens now is that the system
looks at antigen and antibody tests that are happening in the laboratory, the
culture results from the laboratory, and it knows the list of things that are
reportable diseases, and it produces a report every day that says these are the
cases of reportable diseases that you have in the hospital.
In terms of quality improvement initiatives, we support, in particular for
diabetics, a “how am I doing report” that physicians can run. And
what it does is looks at hemoglobin A1c results. And what it can tell a
physician is, for your diabetic patients, your diabetic patients have
hemoglobin A1c levels of 8.1 on average. And throughout IHC, the average is 8.5
percent, and the best within IHC is actually, you know, 6.5 percent.
So a physician can look at the diabetic population they’re managing and
know how they’re doing relative to their peers and relative to the best
practices within IHC.
And it’s had a remarkable effect on what physicians do. I mean, when they
see themselves as an outlier in that kind of quality measure, they change their
behaviors, and it’s been a real eye-opener.
There are also things that we’re doing in clinical research — transurethral
resection of the prostate — and the clinical research in these cases are
things that we do that ultimately end up being primary patient care, because we
create a rule, or create process changes, that change the quality of care.
But in the case of TURP, for instance, what we found is that there was a
huge variation that didn’t seem to correlate to much. I mean, some people would
go home within two days and other people would take as many as five days. And
we could look at the computer data and try and figure out why that was and what
the difference was.
What we found out basically with the TURP surgery is it depended entirely on
when they pulled the catheter. If you pulled the catheter a day early, you have
the same outcomes and everybody went home earlier. And if you left it in
another day, they basically stayed another day. And there were no differences
Another thing where we did research — there was national literature about
pre-term induction of labor. And in that particular case, you know, they said
if you induce labor before 39 months, there’s a much higher risk of
complications in the baby.
And the physicians within IHC said, well, yes, that’s probably true for
those bad physicians nationally.
DR. HUFF: What we’re able to do is go to the data in the computer system and
in fact we could show exactly the same statistics on our own population and say
when we do it within IHC, it’s exactly the same way, and we’re doing it roughly
at the same rate as nationally.
And so it provided the justification for in fact implementing that rule
through education and other means, and, again, dramatically reduced the
pre-term inductions and dramatically reduced the number of ICU days for the
babies et cetera.
So those are just some things. These are the kinds of things that we’re
talking about that are, quote/unquote, “secondary uses of data.”
So this is kind of a more complete thing if you break it into categories.
There could be billing, which — we’re going to hear, I think today, more
opportunities about billing and assignment of billing codes and other things;
morbidity and mortality reporting; quality; patient safety; clinical trials.
Clinical trials really comes to mind especially in the case of
post-marketing information on drugs and devices and particularly also for
enrollment into clinical protocols. One of the hardest things about clinical
trials is in fact finding people to enroll in the trial, and the computer can
be actually a big help in enrollment in trials.
Clinical research we’ve mentioned.
Health population statistics. The whole idea, and this is an area where I
want to hear a lot of discussion from people more knowledgeable than myself
about this, but the whole idea of — you know, if we’re interested in obesity
in the country, I mean, we’re taking I don’t know how many — I’m guessing that
we probably have a thousand or two thousand weight measurements a day, probably
more than that, into our electronic health record systems. I mean, we could
almost have a daily report on how the population of Utah is doing relative to
And so, you know, I think there are untapped possibilities there that we
need to think about.
Public health in general, bio-surveillance, reportable disease reporting,
Cancer Registries, et cetera — again, I think there’s a real opportunity for
automation in those areas.
So why should NCVHS study this topic?
To me, I’m just asking the question: Is there something that needs to be
It may be that after we get the data in, we say there aren’t any new
standards, there aren’t any new needs for policies, everything’s going along
great. But it may be that in fact we do need some new standards or we may need
some new funding or we may need to suggest some demonstration projects, some
other things that in fact would encourage this, or maybe there’s some new
policies or maybe there’s some changes in incentives that need to happen that
would encourage this to happen faster, all for the benefit of increasing the
quality and safety of patient care.
I think there have been questions. One of the other questions was: Should
this be happening in Standards and Security? It may be that this is in fact
very interesting to the Populations Subcommittee as well as to the Quality
Workgroup, and this is an initial sort of discussion to say: Should we broaden
this? Should we in fact raise this issue to the full Committee in some way? And
so it’s good Simon’s here, and it’s good you’ve got a retreat coming up. You
might talk about it a little bit in the retreat to see if there’s some common
interest across the Subcommittee. So, that’s the idea.
Okay, now this next thing is just sort of a little bit about the theoretical
basis of some of this kind of secondary re-use of data, and this is going to be
a characterization of sort of how you proceed from primary data to more and
more higher level inferences that can be made from the data.
So, in some sense you start out, all sort of clinical care and decision
making starts out with primitive observations. And even perceptions. You know
— colors of things, temperatures of things, the appearance of the patient,
shapes, sizes, all of that sort of stuff.
And then what happens, either in a person or in a program, you can make
inferences from that.
So if I’m looking in somebody’s throat and I see particular colors and other
characteristics, or I can assert that there’s redness there, there’s erythema.
And if I’m palpating lymph nodes and I feel a particular size and shape, I can
say that there’s lymphadinopathy here and I can talk about the distribution of
And they tell me that they have pain in the throat, I can say they’ve got a
sore throat. You know, from the heat, basically you get an accurate
temperature; you can get a temperature from a thermometer or some other kind of
interest and you can read voltages on a color(?) counter and get white blood
And you see hemolytic colonies on an arger plate(?) and you can that they’re
beta hemolytic colonies there. So you see certain patterns.
So the next level then, you can apply some further processing to that, and
when I take the redness and the cervical lymphadinopathy, I can say there’s an
inflammatory process going on.
And knowing things about the plate, I can say that this is a positive strep
culture. And if I apply a rule to the temperature, I can say that it’s not just
38.9, but I can apply a rule and say this constitutes a fever. And the white
blood count is increased for this particular population et cetera.
And I can apply yet then another inference process, taking into account then
that I have inflammation, a positive strep culture, fever, and plus I can say,
well, they’ve got acute streptococcal pharyngitis. And I can add in some other
facts about the fact that there’s status post splenectomy or other things and I
can actually come to the fact that this person is in an immuno-compromised
So, the idea here is that in secondary use of data, you start out with these
low-level perceptions, primitive data, and every place that I’ve got these
little brown or yellow circles, there’s some processing going on, and it’s
processing that either happens in people’s brains
or it’s processing that can happen in a computer system that can assert then
that — again, if I have inflammation, a positive strep culture, fever,
increased white blood count, I can assert then that there’s an acute
And that assertion can happen because a clinician, an expert clinician,
assimilated that data in their brain and made that assertion or it can happen
because a computer program has that logic in it and it can make the assertion
based on logical processing of that information.
Now, there are a couple of things about this process. What happens often is
that people assert information — what you’ll get, for instance, on a problem
list is you’ll get that this patient has acute streptococcal pharyngitis.
And that’s good information, but if I could get the other information down
at the primitive level, what I can actually do then is quality control on the
process, okay, because I can look down there and say, you know — because there
might be all kinds of other data and I’d say, oh, you know, there could be a
whole another explanation for that based on the primitive data.
And so you’re in this situation where if it were possible, you would like to
get data at the most
fundamental level, because that allows then the most scientific and
thoughtful and systematic way of analyzing the data so that you look at the
inference processes and actually see if it’s correct. And that’s where you get
into quality assurance and all of the other things.
Now, the down side of that is data collection is costly, so it may be a lot
easier to just catch the assertion of this thing on the problem list than it is
to catch the individual temperatures, and so there’s always a tension: How much
good data can you afford to collect versus, you know, what you can get easily
and support the process?
But if you take away this particular example but think of the process in
general, what we’re doing in all of these things is we’re proceeding from data;
we’re trying to apply rules and come up with inferences from that data, things
that we know, new things that we can assert that we know because of what we had
in the data before.
So you apply that pattern again and again. If we can take the primitive data
of hemoglobin A1cs out, we can do some rules and some inferencing and assert,
oh, your average of diabetics is sort of in the middle of the pack, or it’s a
lot worse or it’s a lot better than the average physician within IHC. So you’re
aggregating that data for a particular purpose.
So that’s the general process, and sort of the general thought behind a lot
of this is proceeding from one kind of primary data to more sophisticated
inferences that we can make from the data that would serve quality purposes,
all kinds of other kinds of purposes within the institution.
So just a couple of observations. Data capture is costly in terms of
people’s time and computer programming and instruments. The closer you capture
the data to the level of perceptions, the more inferences you can make. And raw
data allows testing whether the inference processes are accurate.
Related issues — an idea that was put forth and one of the terms that Chris
Chute has used, talking about this, is, quote/unquote, “aggregation
logics.” So in a sense, what you’re doing is — especially applicable to
classifications are that you can start with primitive data; you can apply a set
of logic and come up with a new conclusion or you can assign an ICD-9 or an
ICD-10 code to a particular set of things.
And part of the whole idea of these aggregation logics are that you’re not
now dependent upon written descriptions in a book that a person or an expert
coder has to understand, but what you’re actually doing is creating computable
rules that a program can execute.
And if we could get people to share those algorithms, and in fact CMS or
others who are doing billing to say “this is exactly what we mean, this is
exactly the evidence that we need. If you’re going to bill us for a stay for
diabetic ketoacidosis, this is exactly the kind of data that we need to support
And if those could be computable rules, rules that can be executed against
standard data structures and standard terminologies, then it takes all of the
ambiguity out of assigning those particular billing codes, and we can do it in
a more efficient and timely way.
I would point out that having said that we’re going to do the secondary use,
I don’t think we want to go to the extreme and think that we can do everything
by secondary use of data because there will be lots and lots of times when the
data you need wouldn’t necessarily be collected as part of standard clinical
If you have a research question about correlation of diets or race and
ethnicity, other kinds of things, there may be questions that you want to ask
that are specific to that research study that wouldn’t be part of routine
So I don’t think anybody should think that a lot of our national surveys or
other things are going to go away because we get into the secondary use of
data, or that
they would all go away. I guess some of them would go away or we’re probably
not going to benefit from this technology.
The same sort of thing is true obviously in clinical trials. You’re looking
at a very specific thing and, you know, if you’re looking at impact on liver
enzymes then for that trial, you’re going to specifically ask for the
collection of liver enzyme levels to test the hypothesis that’s being studied.
But I think the point is that in spite of all of the potential for this
activity, I think we need to recognize that we’re still going to want to do
very focused clinical trials as well as clinical surveys and other things that
lead to specific kinds of data.
So I’ll stop there. My intent was really to just lay some groundwork in
terms of definitions and thought processes and let the other experts that are
going to testify in fact delve into more of the details and strategy here, so
I’ll stop at this point and —
MR. REYNOLDS: Any questions? Any questions for Stan?
DR. FERRER: Stan, it seems at IHC you’re doing a lot of quality assurance by
providing that performance metric back to the clinician. Because of the
competitive nature of the clinician, he wants to, quote, “perform equal to
his peer or better.”
If you go to the next step and you say, you know, we want to report that
information, you know, when does the comfort level of the clinician become a
barrier, if you will, once you start crossing the public reporting arena?
The reason I ask is because CMS is driving towards public reporting of
performance measures, and oftentimes, you know, that quality assurance —
actually, that trust, if you will, is broken once you start doing things like
that. So I’m just curious as to how —
DR. HUFF: There are a lot of interesting issues. Our focus is on improving
quality, and we kind of approach the same mentality as with adverse drug event.
We’re trying to be non-judgmental, because as soon as you start imposing
penalties for reporting, reporting will go to zero, so your ability to manage
the process — at the same time, you know, as a physician and an intern, I was
much more comfortable with medical care when I knew all of the physicians. And
now that I’m just a deadbeat pathologist informaticist —
DR. HUFF: — I’m much more like a regular consumer when I go, you know, to a
physician. And so I have a great sympathy for wanting to have public metrics
that would tell me: How do I choose a good physician?
So I guess I would like to see this continue. I think the way we’re doing it
is successful, trying to improve the quality of physicians. I think it’s a
separate question about, you know, publicly available metrics. I would like to
see them, but I don’t know how to do it in a way that wouldn’t in fact cause
all kinds of other problems in sort of decreasing the people gaming the
numbers. I don’t know how to solve that.
MR. REYNOLDS: Stan, as you envision this secondary use, once some of those
secondary uses became standard and the algorithms and other things became
standard, then basically data could be pulled from anywhere into your inference
engine, whether it’s yours or anybody else’s, and used to draw these same
conclusions on a more general basis than just, in your case, Intermountain
Health Care, right?
DR. HUFF: I missed the question part of that.
MR. REYNOLDS: Since this is primer, I’m trying to make sure I understand.
DR. HUFF: Right. I mean, and Clem may speak to this more directly, but, for
instance, the HEDIS measures that are in place, that’s specifically one of the
things that we’re talking about.
I mean, you could establish and basically say,
okay, for an institution, we want to know, you know, what percent of your
eligible women are having mammograms. That could be an automated report that
doesn’t require compilations.
Basically, you look at the electronic medical record, and if the eligible
women have a report that is a mammogram and there’s a standard code in LOINC
for the mammogram report, you know, more than one, if that report exists, I
count that as a statistic. And I can do that as a query on the database.
And you could implement that, you know, broadly across the nation, and in
fact it’s simple enough that you could get weekly reports from people about how
they’re doing against that particular measurement.
So that’s exactly right. I mean, it would be agreed that this is the exact
algorithm for determining that, and I think you would see a lot of variability
go out of the assignment of those as well as a much more efficient process in
assigning that kind of statistic to a given institution.
MR. REYNOLDS: Simon?
DR. COHN: Yes. Stan, thank you, and Clem, welcome. I haven’t looked at your
presentation yet, Clem, and if I’m going to say something that you would likely
have said, I will apologize.
But, you know, obviously I like very much what I’m seeing here. However,
there is that thing that sort of worries me a little bit, and I guess I’m just
trying to think of how this fits in. It’s the issue of data quality.
Now, Dr. McDonald may want to use an example probably of talking about lab
data being more reliable than physician-entered data. Now, that’s something we
all recognize, that I guess as we use these things and we make inferences and
we try to automate all of this, there’s, at least in the world that I see, a
lot of not very clean data out there, data that’s not likely to get a whole lot
cleaner by the introduction of electronic health records. So that is arguable.
And I’m just wondering: Does that fit somewhere into this vision?
DR. HUFF: I think it’s a very important issue, and I agree with you that it
certainly is dependent upon data quality.
I would argue that, though, making it electronic does in fact have an
opportunity then to improve the quality of that data because — a couple of
things, just an anecdotal —
Some of the other things, for instance, that go into that diabetic report
besides the hemoglobin A1c were whether the physicians were doing monofilament
line tests and other things. And we tried it two different ways.
We said — you know, we just asked the physicians to put this data in, and
it was hit and miss whether they put it in or not and whether it was reliable
And then we started producing reports, and they showed up as having, you
know, basically a zero for their monofilament line tests and they went, oh, hmm
— you know, I want to put it in.
But even beyond that, even beyond just sort of the “I want to be good
and I don’t want to show up as an outlier,” it really is if the clinicians
understand why the data is being collected, then they’re tremendously more
cooperative in supporting collection of that data.
So, it’s really about patients getting better and understanding if there’s a
reason that we’re collecting that number, that it’s going to support the
quality of patient care, then they’re tremendously more motivated to do that
than if there’s just sort of a request for this kind of data and no explanation
about why and nothing ever comes back to them as a result of that.
So, making it electronic and then creating that feedback loop so that they
see some outcome from the data input I think tends to increase the quality of
the data that’s entered. And that’s probably in fact one of the best ways that
I can think of.
But you’re right — absolutely — that the inferences you make are totally
dependent upon the quality of data entry that happens at the bedside and at
that clinical level.
DR. McDONALD: I’m going to touch on this, but the question has to be:
Quality for what? And the answer —
PARTICIPANT: Microphone, Clem?
MR. REYNOLDS: Microphone.
DR. McDONALD: Well, I won’t repeat that. I’m going to say a little bit about
DR. COHN: Good point.
DR. McDONALD: Should I start?
Well, I really am happy to be back. I had four great hugs when I came in
DR. McDONALD: That hasn’t happened to me in about a month, so —
DR. McDONALD: — that by itself was worth the trip.
MR. REYNOLDS: Question. I thought you were going to comment on Simon’s
DR. McDONALD: Well, that’s going to come out of my —
MR. REYNOLDS: Oh, okay, but Steve’s still got a question.
DR. McDONALD: Sorry — I’m sorry.
DR. STEINDEL: Actually, it’s somewhat of a question that may be touched on
by future speakers today, or maybe future speakers in the future, because it’s
an aspect of secondary use of data that’s generally not talked about, and there
are really two types — I look at it as two types — of secondary uses of data,
and Stan did a very good job of talking about the type of secondary use of data
where we draw inferences. We take data, we apply rules engines to it, and from
those rules engines, we get some type of conclusion, whatever it might be and
however complex it might occur. You know, that depends on the nature of what
we’re asking, the nature of the data that’s coming in.
And then you also touched on the other secondary use of data in your
comments about HEDIS measurements, because a lot of secondary use of data is
process. But we count things, we determine the rates of things. These are
relatively simple secondary uses of data, where we have an element; we just
count that element, we determine the rate of this or the rate of that, and we
use that for quality control purposes or other purposes.
But one thing, that when we were looking at population health reporting for
CHI, which involves secondary use of data — virtually all population health
reporting in some way or another comes from secondary uses of data — and one
of the observations that we came to during that report was: Inherent in the
population health statistics reporting system is consistency of data.
Now, we can track the way the data changes over the years. And if all of a
sudden we introduce electronic data that’s been determined by inference-based
engines, is that data the same as data that was put into place by humans
applying their own inference-based engines, their brain, and putting this
And that’s something we determined we couldn’t answer at that point in time,
but it’s something that I hope we’re going to touch on eventually during these
DR. HUFF: You know, I think we have exactly that issue, that it’s not unlike
when you start using the new coding system or something. I mean, you know, I
think in almost all cases you’re going to have some delta there, and in that
first implementation you’re going to wonder how much of the delta is due to the
new methodology versus how much of that delta might be some real change in the
data, and I don’t know any way to get around that.
I think you have that question, but I guess I wouldn’t be stopped by moving
forward because of that question. I’d just try and figure out all kind of ways
we could to mitigate and understand how much of the delta was due to one thing
or the other.
DR. STEINDEL: And I think that’s the point I was going to make.
I agree with you 100 percent — we’re always going to have that perturbation
and we shouldn’t stop because of the fact that it might exist. But we do have
to understand that it might occur.
DR. HUFF: Yes.
MR. REYNOLDS: Okay, Maria, you had a comment?
MS. FRIEDMAN: Actually, it just relates to what Steve was just saying.
There’s not only the issue of data comparability in quality but there’s also
the issue of database comparability in quality. Are we collecting and measuring
things the same way and keeping them the same way? And when you start trying to
take it up a level and do population health statistics or these other kinds of
inferences outside of your own institution, how do you get these databases to
talk to each other and do it correctly?
This is a problem we’ve been, you know, dealing with when I was at AHRQ in
the ’80s, and I don’t think really it’s gone away any.
MR. REYNOLDS: Jeff, you had a comment or question?
MR. BLAIR: Yes. I’ve been listening not only to Stan’s testimony but also
the questions. I see we’re digging right into this issue right away and we’re
beginning to, on surface areas where there may be difficulties or imperfections
or falling short or flaws, all of which I think is absolutely, totally
So, I just wanted to add one thought to counter-balance that —
MR. BLAIR: — and that is that is that if this exercise, however long it
takes us, whether it’s six months or 12 months or 18 months, to try to see what
we could do to move the ball forward to capture clinically specific information
once at the point of care and use derivatives for other uses, if we only get 20
or 30 percent of the way, we will have made such a tremendous improvement in
quality of care, in cost of care, in clinical research that I just feel like
we’re entering an extremely important area.
So, the reason I make my comment is not that we don’t need to identify all
of these difficulties but that we try to keep a mindset that as we identify all
of these difficulties, we don’t have to get a hundred percent, we don’t even
need to get 50 percent, for this to be an extraordinarily valuable process.
MR. REYNOLDS: And before I turn this over to Clem, Stan, off to the side you
need to help Jeff and I. On Chart 6, you’ve got that person sitting behind that
PC. Everybody else on your charts is normal, and you got that person sitting
there with a bad haircut and limited —
MR. REYNOLDS: I think it’s a medical records guy and Jeff thinks it’s the IS
guy, so —
MR. REYNOLDS: — we need a clarification.
The next presenter, for those of you on the Internet who might not know him,
is Dr. Clement McDonald. He was an esteemed member of this Committee at one
point and —
MR. BLAIR: He’s not esteemed anymore?
MR. REYNOLDS: I’m not done here, Jeffrey.
MR. BLAIR: Oh, he’s not a member of the Committee.
MR. REYNOLDS: That is correct. But it’s always good when you have somebody
come and you hear the words, like their insight, their expertise, their
personality are missed on this Committee, so, Clem, welcome. We’re excited to
hear from you.
AGENDA ITEM: Presentation — DR. CLEMENT J.
DR. McDONALD: Thank you, Harry. Could I, just for a matter of tracking my
life out, what’s the actual schedule here? I mean, when am I supposed to be
finished and when is this —
MR. REYNOLDS: You have till 2:20, 2:30.
DR. McDONALD: Okay. And then can I leave after that?
MR. REYNOLDS: No.
MR. REYNOLDS: Depends upon the quality of your presentation.
DR. McDONALD: Okay. Well, the only reason is I got a flight. That’s my only
constraint. And I wouldn’t have done that if I thought I shouldn’t have done
So I’m going to touch on a number of these things. So the bottom line is,
among the things that should come out of this, there’s a lot of good areas for
research that I heard amongst the questions. And I may rebalance my
presentation as I go, so if I flip through some slides very fast, it’s because
I probably shouldn’t have put them in there.
DR. McDONALD: I want to discuss sort of two
roads, and I really wasn’t quite sure of Stan’s position, but I think we’re
pretty harmonious in what he’s saying and thinking. Also, I’m going to have
this from a point of view of trying to build a community system because I think
a lot of the data you need to do these things need to cross institutions to
The two roads are that standardized existing codes employ multiple — I
mean, existing content and employ it for multiple purposes, perhaps with some
supplementation to kind of beef up something so you could get to it without
huge effort for some existing purposes.
Secondly, is capture everything in coded form as a primary effort and use
it to support a host of second efforts.
And I’ll take these in order.
So the background and current realities — I think the value proposition
for the information infrastructure is that we standardize once and spread the
cost over many uses.
And the big three of standardizing targets — I’m just going to do these to
highlight them; there’s really other dimensions — is IDs for patients or
knowing who’s the same patient, IDs for providers and knowing who’s the same
provider, and IDs for observation and reports. You can go down lower, but this
is hard enough.
But I want to remind you that a few clinical applications needs less. You
can actually get by with sort of some minimal standardizing, and that’s not
going to be useful for secondary.
So one example is we do community report delivery to physician’s offices.
And, bottom line, all you have to standardize is the physician, because you
deliver them and they figure out the rest. You know, actually it’s for printing
out and putting in a chart for practical purposes so they keep them on line,
too, and they can find them by knowing the patient’s name. They don’t really
worry that there’s two John Smiths in this model; it really works pretty well.
And this is what it looks like, but that’s not important.
DR. McDONALD: Well, there it was again. The middle white part is actually
the patients’ names, which we had to block out.
So it provides clinical utility while dodging the standardizing work. It
requires only standardizing the providers in the HL7 messages, and that is some
work, but that’s going to go away, we hope, with the NPI, I mean, the National
Provider ID, which we’ve been waiting for just for nine years, I think. Some of
us had dark hair and dark mustaches when this started off. That’s all gone now.
So this is easy to implement, and there’s other easy tasks. There are a
number of easy things you can do.
But the big clinical services need big standardization efforts. You need a
flow sheet if you’re going to bring different systems together. You need to
have the same code to get the flow sheet to work, that simple. Specialized
clinical reports, you need the same thing. Decision support — if you’re going
to dig out who got a flu shot and you’re counting and getting the ones done in
a pharmacy and the ones done in Hospital X and the ones done — you’ve got to
have something standardized. Either that, or you’ve got a whole lot of extra
labor in data collecting.
And the same is true of secondary uses. Epidemiology, clinical research of
all kinds, public health case reporting, quality reporting for HEDIS, pay for
performance — it doesn’t really matter where the flu shot was done; you get
credit if it got done, but you’d like to know where were the other ones that
you didn’t do in your office, and more.
And I want to bring this up because I think that the real challenge here is
we almost have to do secondary uses to get the energy equation to push over, to
get all the standardization done. And I’m talking just on stuff we have now,
not the hard stuff, not the full clinical notes.
I’m just saying if we get the labs and EKGs and the drugs and all this stuff
that’s kind of in reach, just getting that, we still need, I think, to really
think in terms of going for both. Otherwise, we don’t have enough maximum push
to get it all done.
Now, we can deliver secondary services today and we don’t have to wait for
the full source data collection across the spectrum of all sources. And we can
do a lot with what the electronic data has collected now. And granted that we
maybe can’t get perfect public health data collection totally automatically,
but, by golly, we could do things on drug use and drug side effects that can’t
be done without the secondary data. I mean, you just can’t do it.
The Vioxx thing could be stopped. We could do it with stuff that’s kind of
there but it’s just connected. So as long as the patient and the observation
codes are standardized, it can be connected in some way.
So, for a successful LHII — and I guess the most recent term, and I’m still
stuck on my previous NCVHS days where I think we called them Local Health
Information Infrastructures and now it’s been modernized to RHIO, but it’s the
same thing — but these things —
DR. McDONALD: I think it’s the same thing, quite the old one that, you know,
came out of here. Because creating order, standardizing, requires work — it
costs. That is, there’s just no way around it, and we just think what we
standardize, and isn’t that nice, it’ll come out free. It’s the second law of
thermodynamics: You can’t get order without doing work, period.
So we had to find many uses, to spread the costs over many clinical and
secondary services, and for sure we don’t want to design these systems that
will preclude application to secondary services.
So we’re going to just briefly talk, because I was just a little bit miscued
about what this is about, about what we’re doing in Indiana. So we’ve got a
real live LHII, or maybe it’s even a RHIO, with data flowing from all major
Indianapolis hospitals, five systems, 1`5 hospitals. Physicians in the ER get
access and we’re about to turn it on for physicians in hospitals, to the data.
We report push to more than 2300 physicians. We integrate a number of public
health functions, not all that we could; we still working at it. And we have a
citywide research database.
So there’s a lot going on. We have more than 94 HL7 message streams, more
than 50 million separate HL7 messages per year, and that’s not counting each
result. And I want to say HL7 works. My sponsor, HL7 — no, they’re not my
DR. McDONALD: It really does work. We get 30 million accesses per year for
the clinical care at two hospitals. We add about 80 million results per year.
We have 30 years of cumulative data from one hospital, 15 years from two,
and lesser amounts from the other three; 700 million rows of clinical
observations; 45 million radiology image, not studies, because they make a lot
of images per study; 14 million text reports; 30 million physician-signed
orders; two million accesses per month.
And this is a centralized system. People kind of describe it that we’ve been
lumped in with the distributor system. We ain’t. We’ve got one big computer
that sits in one room. We separate databases for some of it, but there’s
cross-links and cross-indexing. And we do the standardization centrally. We
thought we’d just have everybody link them at their labs — they don’t; and we
get experts and they can do it and it works out may be the way to do it.
The whole city has about 165,000 inpatient submissions per year, 450,000 ER
visits and 2.7 million outpatient visits. Now, there are other visits to
offices in other places that we don’t count.
This is what Indianapolis looks like, a Blue State — no, is it red?
DR. McDONALD: It’s blue on this map, whatever color it really is.
And those “Hs” are hospitals. One of them, we lie about — one of
them is built but we’re not really connected to it yet.
All the hospitals contribute discharge summaries, operative notes, radiology
reports, path reports, cardiology reports, tumor registry data, and two-fifths
of them contribute a whole lot more, and public health contributes data as
And I want to point out, though, this is far from everything. This is a lot,
and a lot of it’s text, so, I mean — I didn’t say laboratory; they also have
all laboratory reports. But you can do stuff with text.
We’ve got a lot of other stuff going. We’re trying to weave a bigger total.
I’m not going to go into too much detail on that.
We are actually connected to RxHub, getting drug information on patients
with ER, with permission to get it from anybody that’s in an outpatient clinic.
We have a tumor registry for the whole state now, 15 years of data. We’re
connecting to the EDs across the state for bio-surveillance. We have 36 of 134
We have an agreement with Medicaid to get all their data for the data, using
it for clinical and research purposes under very restricted purposes. We almost
have agreement with Wellpoint, which came from California to say hello to us in
Indiana, and they collared some other operations that are kind of joining up.
Now, these are examples of functions that need standardization of patient ID
and observation ID. So, flow sheet — and if you could really read this slide,
down below these “A’s” and “B’s” et cetera, some of this
stuff comes from different places, and so we’ve indicated some of the different
places by footnoting. But that required a standardizing effort to get the same
hemoglobin from two different places to look like the hemoglobin or be called
the hemoglobin. This didn’t require standardizing — I’m not sure of that one.
Business decision support, same thing, unless you’re going to just stay in
your own environment. Even there, since these hospitals joined together, the
multiple hospitals, very often you’ve got multiple — we’ve got three cardiac
echo systems in our system, so even there you’ve got to do some standardizing
to make it work.
This is a study of preventive reminders about flu shots, and a big effect
with decision support, but you couldn’t do that without standardization.
Epidemiology, there’s just tremendous opportunities there, and I’m meaning
epidemiology in a very broad — with standardized databases. Even if we don’t
get into the new reaches of rich clinical data, just the stuff, the drugs, the
diagnoses, the coded diagnoses.
We have two things we’re especially proud of. We have kind of found — not
me, but some people in our group found an association between erythromycin and
pyloric stenosis among newborns, a tenfold (?), and that was verified out of
Tennessee in Wayne Ray’s group but we did it first.
And there’s a non-association between statin risk and liver disease. If
you’ve got high enzymes, it doesn’t mean you shouldn’t use statins or can’t use
We have a tool now which goes across the city with the goal being to be able
to do research without touching — no-touch research — so you never really
touch any of the data. You just get back summary stuff.
So we have this SPIN tool where you can specify a cohort and then you can
specify the variables you want to retrieve, and then you do the analyses and
you get back statistical analysis. No, we haven’t proven that this is going to
be a break, but we think it’ll make research a lot easier because you can
explore a lot of things before you have to do the big work with the IRB and
This is what the form looks like. I did provide the slides, and if people
really care about this, they can read it off them; this is too long a
This is what one of its cross-tabulations looks like, and you can do things
like logistic regression. We use an “R,” which is a one-letter
programming language. It comes from Bell Labs. They called everything by one
letter. If they did 26 programming languages, they would have been screwed, but
“C” was the first one and then “X” and now “R.”
And then the lessons learned from INPC. HL7 Version 2 works well for results
delivery, but there are problems, and it’s not the HL7 syntax. And this is
something that maybe we need regulations or laws or something about. One to two
percent of the messages are syntactically legal but stink. They’re egregiously
bad messages and there’s just violent disregard for what the intent is of these
And they all boil down to stuffing the wrong stuff into the wrong field. And
it’s just egregious. So you’ve got a field called Numeric Value, and its
numbers will be there, and another field called Units, and you see Values and
Units scrunched into the same field. You see Values and Units in normal ranges
and discussions about where it was done all scrunched into the same field. And
there’s separate fields for those.
Then you’ll see 12 results, like a Chem 12 scrunched into one field.
Literally, you can make those legal by just declaring it’s a text field, and
there’s no way around that in any change in the syntax because you’ve got to be
able to have a text field.
So we basically to have a semantic checker done. We have actually written a
program called “HL7 Lint” which is a beginning on this and it
actually looks for units in the wrong place and it finds most of the bad
messages. Some of them aren’t totally bad, but they’re mostly bad.
And we had one of the experts at HL7 whose name you know who said, well,
that’s okay; that’s what you’re supposed to, make it a text when you can’t fit
it. Now — hey, guys, put this stuff in the right place so we can process it
with computers. And you could say if you get more than .4 percent, .3 percent
bad messages by some semantic checker, you don’t get paid that month, or some
darn thing, I think we could fix this.
You know, I heard about the beauty in those. I love Stan’s slide, all the
wonderful things it can do. But when you really get down on the ground into the
dust, it’s like moths have eaten all the data. I mean, there’s holes here,
there’s holes there, there’s a missing thing there, you know?
We got all the drug data except, well, you know, we’ve got these HMO plans;
they don’t send us the drug data, you know, for Medicaid. And everywhere,
wherever you go, you know, this moth has eaten a damn hole in it and we’ve got
to worry about those moths, and some of it comes from these crappy messages
where we get the moths.
Sorry about this — it’s a side issue. We’ve been working a long time, Stan
DR. McDONALD: — now to get good data we can use for a lot of things.
So a secondary use is public health. So we now do scan results from all the
labs that we get data from, and we find reportable cases and we send them to
the public health department sort of in one big package without any human
review. I mean, except when it gets there and they go, ah, what’s this!
But we find a lot more stuff. For a lot of the tests, it’s not that hard. So
if you’ve got a serology result, it says Hepatitis B, it’s Hepatitis B, or an
antigen. Or a DNA. The cultures get a little trickier, but with a little bit of
parsing and all — because it’s all text; that’s just how it is in blood
cultures and effects. But it’s from a menu, so you can parse them pretty easy.
A given lab always says the same thing — you know, it’s normal, whatever they
say — so we can work it.
This sort of a draft of it. We get the inbound HL7. We use them for storing
for clinical care. We filter them by the LOINC codes; we’ve already mapped in
the LOINC codes. We then do some parsing inside the text to knock out some
Here the biggest regulatory good we could do for public health is require
that labs must use the abnormal flag and mean it, so that instead having to
parse through 5,000 TB cultures that are normal, but they say it 25 different
ways, a lot of times saying no, might go back to tuberculosis, might go back to
blah-blah-blah. Well, you can pick out the first “no” but these
ellipses get tired.
And so when we first went into this, we found all kinds of positive TB
because they had these strings of various TB variants and the “no”
word was only in front of the first one. But if they just said “this is an
abnormal one” in the result, we could really nail it pretty easily, and
it’s really sort of part of the HL7 thing, but they slip on that a lot in the
And then, we were developing this when there was a big outbreak of shigella
in daycare centers and we weren’t able to help this process, but retroactively
we were able to find like three or four times more of it than they were able to
find when we ran them through our system — not three — but some large number.
It’s real time. We get 100 percent of the received. I think we found twice
as many cases than the hospitals find by the usual method, and we get it eight
days faster than the — gee, I don’t know what that stands for — case signing
— and twice —
PARTICIPANT: Health Department.
DR. McDONALD: Health Department, yes, and thank you. And two days faster
than the hospital case signing.
There’s a lot more you can do, as Stan kind of alluded — quality assurance,
pay for performance, outcomes management. There’s just a lot of things you can
do with data, and a lot of this is not radical, wild stuff; the data’s sort of
sitting there. I’m not talking about getting deep into a discharge summary or
the handwritten notes to figure things out. There are just good things, and I
keep coming back to drug adverse effects, really some huge opportunities to do
things with that. And we could find those early, drugs that are bad or bad for
Actually, as an aside, I’ve always had the policy of don’t use new drugs.
You know, they’re never that good, and they always something wrong and you find
out five or six years later.
So I never used Vioxx. [Imitating patient] “But I really need it, Doc.
I just love that drug. I saw it on TV.”
DR. McDONALD: So the second option is capturing everything at the source.
And I actually want to put some cautions up against this. It’s very
attractive, I mean, if you capture it at the source, code it and do it
standardized so we can use it across institutions.
I think we should first cash in on existing flows for secondary purposes.
HEDIS does exactly that and uses existing data stuff and does a pretty good job
on it. Now they’re going to have to stretch further as they go further along.
I think the prime directive should be standardize what we have. Use it for
both clinical and secondary purposes.
The sub prime directive is, if we do capture discrete data, capture it in a
standardized and poolable form, one that you can use it for multiple places.
But data capture costs. That slide was supposed to be something really good,
but I don’t know what it was going to be.
DR. McDONALD: They said you have to close your portable PC now on the
airplane now as I was coming down.
DR. McDONALD: So, there’s a lot of questions. You know, there’s a lot of
research areas in here. Questions about capturing everything as
computer-understandable form. We know almost nothing about the clinical content
of the clinical data we collect. It’s in rough form — the handwritten notes,
dictated notes. We really don’t know what’s there. We don’t know about why
clinicians record it and how they use and why they do it.
I mean, it could be as much as they use it to remind themselves at the next
visit of what they were thinking about, so it may have very little value as
sort of a formal, archival record for talking about the population and it may
have huge value for making sure you can take care of that patient because
you’re using your own way.
And just converting current content to formal codes I don’t think is going
to be the answer, except when we can do it automatically, and there are some
really nice opportunities for that, I think, hopefully.
Now, I think we’ve got to remember absolute ignorance. There was a great
article in Science about 20 years; it talked about absolute
ignorance. And that’s the stuff — you know what the melting point of lead is;
it’s that you didn’t even know there was a question there.
DR. McDONALD: You know, you didn’t know what you didn’t know. It wasn’t
anything anyone ever thought of.
And that’s the stuff that comes out of like Einstein, you know, when he came
up with relativity, and there’s a whole lot of things where they didn’t know
there was stuff like that. So we don’t know what we don’t know, and I think
there’s a lot of that in the discussion of computers and medicine. We really
don’t know nothing yet. We don’t know what’s what, I think, in areas.
And we ought to keep in mind that process, that we can just assume that we
can do it — you know, we got this and we can do that with it.
So I think we frequently confuse the computer with data. You know, we get
the computer — and this is to your point — we got the computer now, we got
the data. No, you don’t. Now, there’s a billion questions you can ask patients
and no one ever asked those billion, and whatever this third party wants, this
new idea is in the billion they haven’t asked, those answers.
So, computerizing by itself doesn’t produce data. It can, and it will, in
some settings, and we’ve got to figure out what we want to ask and how we want
to ask it.
So clinical data has extreme high dimensionality and a deep hierarchical
and/or network structure. We don’t fully understand it. But there’s zillions of
ways to ask things.
Users hate to read 20 variable standardized assessments, and we’ve got this
beautiful data collection from nursing and nobody wants to look at it. I mean,
it’s useful for computer stuff because we actually use that to trigger things,
but if you go down pages after pages —
Humans love well-formed human summaries, you know, discharge summaries.
That’s the first thing they read in our environment.
And there’s good and bad things about both of them and we don’t just know
how they link or whether you can connect them to each other and we can’t ask
every question every time.
So we can’t just map clinical narrative to codes, at least not for
statistical use. We need to do this formal questionnaire development and
reliability testing, which everybody hates to do — I hate to do it. And then
you get 20 questions to ask which you just thought you could do in one
So, things like the Hamilton score and the CAGE for alcoholism, Braden for
the bed sore, these are things that formalize and standardize. There’s lot of
them, and they’re good ones.
So that’s the effort we have to think about doing in a lot more space to use
these things as data, you know, on a formal statistical basis for a lot of the
secondary purposes. And we require much tweaking of questions. You usually get
good, reliable questions. How do we ask them so that people know what you meant
always, the same way? Sometimes you have to ask more than one question. It
sounds like it’s redundant to get reliable answers.
So we don’t know what content is worth doing this for, and we’ve got to sort
of sort that out.
Now, and the other kind of things to think about — the Ottawa Ankle rules.
This kind of explains or highlights another thing we have to do as a research
So this is this guy named Stile(?) from Ottawa, I guess, because they name
it the Ottawa Ankle rules. And they asked him: Who should we get X-rays on,
ankle X-rays on? And the guy said, well, we don’t know. If it feels right, you
get it, you know?
And everybody said, well, you get it if X is true, you get it if Y is true,
and they listed all the X and Ys, about a hundred variables, finding things.
And then they said, let’s just collect this on a thousand patients.
So they collected these hundred variables on a thousand patients. They got
ankle X-rays on all of them.
And then they analyzed it, and they found out which of those hundred
variables, about six of them, were predictive. And if you use those rules, you
can save 25, 35 percent of the X-rays you would have done otherwise without
being more specific.
And this is sort of what the process I think something like this has to be
across a lot of medicine if we’re going to rationalize it. And the other lesson
here is that never are all the questions useful. I mean, I’ve never seen an
analysis that didn’t boil down a hundred questions to six or eight. I mean,
there just isn’t any predictive value, at least statistically, and I think
probably really beyond all that redundancy. We don’t know which of the hundred
things we’ve asked about are the good ones.
So further, I don’t think narrative can ever be replaced, at least in my
lifetime. Of course, that may not be that long. The purpose may very local, as
I said, to job providers’ memory about how they thought about something or it
may be for literature’s sake or maybe for poetic sake, whatever. We’re not
going to get rid of that, I don’t think.
We don’t know the ideal tradeoff between narrative and structured
information, but that’s what we have to find. We have to find: What are those
grabbers we should be always getting, you know, as a number, as a score or as a
questionnaire, because it’s just so valuable, or getting under certain
circumstances because it’s valuable?
We’ve got to do that homework, we’ve got to do that research. We can’t just
count on it happening because we’ve got computers. This is outside of the
I think eventually we can expect the computer to understand the narrative to
some degree and that’ll help a lot, but there’s a lot of research questions.
So, that, I guess, is the end of my talk, now that I’ve seen that I don’t
have any more slides. But I think that if we could kind of promote such
research agenda on this, we can more done faster. And I don’t mean just
research, on racing to get — but figuring out how to get from where we are,
and not just putting computers in offices but figuring out what the questions
should be, the variables should be, what are the predictor things.
In drug trials, they actually know these things, in a lot of drug trials,
and they do these kind of surveys and questionnaires. And we ought to be
thinking about which ones we should be asking in clinical care and when and how
MR. REYNOLDS: Clem, thank you. Questions? Marjorie.
MS. GREENBERG: Well, I wanted to thank our current former members. I can’t
tell a lie: I provided one of the hugs.
DR. McDONALD: I want to come back!
MS. GREENBERG: It’s great to see Clem again. And I want to thank Stan for
bringing this to us and then bringing it back to us, and I frankly find it very
exciting that the Committee is exploring these issues and questions and I do
think that it really does belong not only on individual subcommittees but at
the full Committee level.
I think only Simon was at the retreat of the Quality Workgroup.
DR. COHN: Yes.
MS. GREENBERG: Were the rest of you there?
DR. COHN: For one day.
MS. GREENBERG: What? And you were only able to be there part of the time.
But, I mean, this is so related to what they spent a lot of time talking about.
Don Dettmer was there and John Halamka and — well, your colleague from —
DR. HUFF: Brent James.
MS. GREENBERG: Yes, Brent James, et cetera. And they were talking about the
— and are now kind of trying to think this through about how they want to move
ahead with their work plan, but exactly what you said, Clem — just because we
have IT and even electronic health records et cetera doesn’t mean that we’re
going to have the road map for quality or have the answers to the quality.
And there are probably just a few places in the country that have as much
development as you have there in Indianapolis and John Halamka, who is, you
know, in Massachusetts also with a RHIO, where they have a very electronic
We have all this, we have the lab, we have all of this, but we don’t have
the road map for quality and we haven’t really thought through, and there isn’t
any kind of road map for how having all this IT and this electronic stuff will
lead us to better quality data.
So I’d say at a minimum the Quality Workgroup needs to be a party to this,
and I’ll make sure that they get copies of these presentations and also refer
them to the transcript when it’s available.
But, you know, I think it’s very much what they are thinking about.
Now, there are other applications, as you said, as well — the population,
the public health applications, would be more probably in the realm of the
Populations Subcommittee and they are, I think, having a conference call in the
next week or so to talk about their future work plan.
So this is really a very good time for this is really what I’m saying
because there are several subcommittees, work groups to whom this is very
relevant who are just thinking through their work plans now, and so, as was
suggested by you, Stan, the Executive Subcommittee would be a good venue for
this discussion, too, but I just wanted to reinforce that and to say that I do
think we need to get this type of discussion before those groups sooner rather
DR. McDONALD: I just wanted to not leave the impression — I did, I really
agreed with everything that Stan said, and there there is the issue about
trying to the decision rules to kind of figure out higher level things.
But the other thing is that what happens at least I think at shops that have
been informatically inclined for a long time is what the IT does do is getting
people thinking maybe more rationally about what they’re collecting, and so you
do collect some things that you need and you should have collected as a formal
piece of information.
And the idea of the to tell how well a diabetic sensation is going through a
monofilament, that just could be one example. In this form that I described,
140 questions, actually it averages out about 70 on average.
It actually was a wonderful process because we worked the informatics with
people, with nursing, to decide what they needed to collect in initial
And stuff like: Where do you go? Do you have a place to go when you leave
here? You know, a nice thing to know, and early on especially.
But, you know, they were doing histories in physicals and repeating this
stuff that everyone else was collecting and then they did the Kscore(?), you
know, so we could intervene on people who were alcoholics. I think they did a
little, a mini — and I don’t know if it’s the Hamilton, but there’s some other
depression scores, a survey instrument, which could cue people to someone who’s
depressed and maybe again we could intervene.
So the informatics thinking is a helpful thing, not just the computer, to
kind of decide why we’re collecting what we’re collecting and collect the stuff
you’ll make the basic decisions on.
MS. GREENBERG: Could I just add — actually, I had a question, too, and that
is whether in this distilling from all the different types of questions that
one could ask down to what is really most predictive et cetera, in your work on
this, have you used the RASH Analysis, or is there any particular analytical
tools you’ve used or is it more consensus building or —
DR. McDONALD: Well, statistics, logistic regression, you know, that’s what
we use. I mean, the data says these things predict 80 — well, you could
usually pick more than one subset and come out very close.
But practically speaking, you know, after the tenth variable — you don’t
need to collect everything in a formal way to get the same decision power,
that’s all, and it’s usually a ten-to-one ratio of the variables to the
particular instances (?).
MS. GREENBERG: Okay, thanks.
MR. REYNOLDS: Okay, thank you. Clem, we’ll let you get to the airport. Why
does it hit me that I can look and see a National Enquirer
headline that Dr. Clement McDonald says moths are eating the important health
MR. REYNOLDS: Why does it look like it could turn into something. Thanks. It
was really great having you.
DR. McDONALD: You’ve got to be careful of what you say!
MR. REYNOLDS: That’s right. Thank you. Great seeing you again, too.
DR. McDONALD: Thank you.
MR. REYNOLDS: Okay. Vivian, they’re going to get you set up and then we’ll
get started on that.
DR. COHN: Take a five-minute break?
MR. REYNOLDS: Why don’t we do this?
MS. GREENBERG: Five-minute break.
MR. REYNOLDS: Why don’t we just take the break, take a 15-minute break now,
get it set up, and then we’ll come up and we’ll have two hours left to finish.
Everybody okay with that? Let’s do that.
MR. REYNOLDS: Okay. Our next presenter today is Vivian Auld. She’s going to
cover the National Library of Medicine standards related activities. So,
Vivian, thank you.
AGENDA ITEM: Presentation — VIVIAN A. AULD
MS. AULD: Okay. Can you all hear me? Good.
Thank you for giving me this opportunity to talk to you today.
One of the reasons that I really wanted to talk with you is to give you an
update of the various projects that NLM is doing relating to standards. Many of
them are only known in pieces by different members of the Committee, and so I
want to make sure that all of you have a complete picture. And it’s not your
fault that you don’t have this complete picture; it’s because we’ve been so
busy doing, we haven’t necessarily been telling people what we’re doing. So I’d
like to fix that today.
I just want to give you a little bit of context of why this is important and
why we have a role in this. NLM’s view is that electronic health data standards
are a key component of the National Health Information Network and they’re
needed for efficient health care, research, public health and emergency
detection and response.
Underlying NLM’s interest in health data standards is the assumption that
EHRs will make it easier to deliver relevant information at the time and place
important decisions are actually being made.
And our specific interest is in the subset that deals with data content,
standard vocabularies, mapping between clinical vocabularies, and
administrative code sets.
And what I have here on the screen is a list of some of the activities that
have been taking place in the last year that you all are extremely familiar
with, starting with the creating of ONCHIT back in April, 2004; the HIT
Strategic Framework; Secretary Leavitt’s 500-day plan; the report on Nationwide
Health Information Exchange, and the ONCHIT RFPs that have just been put out
The reason that I mention these is because NLM has been working very hard to
make sure that all of our programs and activities are aligned with these
various projects and that we’re contributing wherever we can. Both Dr. Lindberg
and Betsy Humphreys have been in many, many conversations to make sure that
we’re on target for these.
And I have here a slide talking about NLM’s long-range plan. The short story
here is that it says in our long-range plan that we’re going to do this, and so
One other thing that I’ll point out is that you made the recommendations to
the Secretary that NLM act as a central coordinating body within HHS for
Patient Medical Record Information terminologies and mapping recommendations or
mapping between clinical and administrative data, and that is also something
that we’re actively working to make sure that we’re supporting.
What this coordination covers is:
The uniform distribution of designated standard vocabularies through the
Unified Medical Language System Metathesaurus.
Reducing peripheral overlap in establishing explicit relationships between
the standard clinical vocabularies.
Aligning standard clinical vocabularies with standard record and message
Mapping between standard clinical vocabularies and administrative code sets
and/or other important vocabularies.
So what I’m going to do today is based on that definition of coordination
and give you an overview of various projects that we’re working on.
So, let’s start with UMLS. I hope that you all recognize this page; this is
our website, the main page for the UMLS, and this is where you can go to get
information, documentation, find out when our last release is, et cetera, et
cetera. And it also links you to all the various tools that are components of
In general, what’s happening with the UMLS is that we’re moving it from a
research project to a production system. And this has several different steps
that are rather painful to go through but the end result is going to be a very
positive, sound system.
This includes transitioning from our research branch to our production
branch. And the production branch are the same people who for all these many
years have been — they’re the same departments that have been taking care of
creation of Medline, so we have a lot of experience that we’re building into
Not only are we moving this from one set of staff to another but we’re also
migrating to new computer systems, both the hardware and the software, so that
we are making sure that we’re no longer following the research model but the
production model, which has different requirements for firewalls and security
et cetera, et cetera.
And we’re also adding new staff to make sure that we’re supporting
improvements to the documentation, training people so that they know exactly
what it is that they’re using, quality assurance, and customer support.
And overall this movement from research to production is going to have the
greatest effect on the Metathesaurus release files, but it’s also going to have
some reciprocal effect on some of the other tools within the UMLS.
And I just want to point out that this does not mean that the research
branch is going to be out of the picture. They’re a group of very talented
individuals and we’re going to give them the opportunity, because they’re not
worrying about day-to-day production, they’ll be able to instead focus on
continuing to update and bring in new features and capabilities of the UMLS.
So let’s talk about the Metathesaurus itself.
The latest release that we have is 2005AB, which came out in June of this
year, and there’s a little over a million concepts. Concepts are terms that are
grouped by meaning, so if you have one vocabulary that talks about myocardial
infarction, another that talks about heart attack, they would be linked at the
There are 114 source vocabularies within the
UMLS, so it’s rather big, and it represents 17 different languages.
MR. BLAIR: Could you just clarify that? What is the distinction — maybe
give an example of languages versus — what was the first term you used? It was
MS. AULD: There’s 114 sources within the UMLS, so by sources I mean SNOMED,
RxNorm, LOINC, MeSH et cetera, et cetera; all the ICDs.
By different languages — for example, we have MeSH translated into all 14
different languages, so you can get it in Spanish and German and French, et
MR. BLAIR: Thank you. It was probably on a slide; just couldn’t see it.
MS. AULD: No, it’s not.
MS. GREENBERG: We did need it.
MS. AULD: What we’ve been doing with the UMLS for the most part is in 2004
we made major changes to the structure of the Metathesaurus by introducing the
Rich Release Format as an input and output format by changing it so that we can
represent mappings and allowing for content view flags so that we can help
people to create specialized subsets.
All of those background changes were made in 2004, and this year we’re
really starting to reach a point where we can start harvesting the fruit from
all of these changes.
Because of all the transition efforts during 2005, we only have three
updates. In 2006, we’re going back to quarterly updates. The final release
schedule that we’re going to end up with sometime in the future is yet to be
determined. It’s really going to be a question of how are we going to
synchronize the updates from the critical sources that are identified as
standards? And that’s actually an area where it’s something that we’re going to
have to figure out as we go forward, but we’re looking for input from the
community to help us make sure that we’re doing that correctly.
I was mentioning that we have the Rich Release Format as a standard
submission format. This really makes it easier so that the source providers can
give us their data in a uniform format so that we can quickly and efficiently
get it into the UMLS. Right now, it generally takes roughly two months to
invert a new source, and we’d like to cut that down so that we can do it much
And we’ve been testing with HL7 code sets and with RxNorm, and we’re hoping
to add new sources in the very near future.
As an output device, the Rich Release Format enables us to insure that we
have source transparency so
that we can insure that what SNOMED gives us, what the College of American
Pathologists gives us for SNOMED, is the same thing that you can get out of it
on the other end.
And the only other thing that I wanted to mention here was the content view
flags, which are going to allow us to pre-define subsets. Right now, we only
have these set up so you can get sources that don’t have any copyright
restrictions, but in the very near future we want to set this up so you can
easily get all of the HIPAA standards or all of the CHI standards or the part
of the CHI standards that are applicable to your situation.
So that brings us to MetamorphoSys, which is the installation program that
goes along with the Metathesaurus. And our goal here is, because we have 11`4
sources, there are very few people, there’s very few entities, who can make use
of everything that’s in the Metathesaurus. So we want to make it very easy for
people to create a useful subset. And we’ve been making some changes in the
last few months and we’re going to continue to do so.
And as I was just commenting, there are currently three default subsets that
you can specify within the Metathesaurus so that you can specify just Level 0
categories, or vocabularies, those that don’t have copyright restrictions; the
Level O plus SNOMED CT for use in the United States, and an RxNorm subset.
We also currently have a feature so that you can create your own subset or
your own rules for creating a subset but you can’t save that across versions of
the UMLS, so in the next version that we’re going to release in November, you
will be able to migrate it from version to version.
And as I said, on the MetamorphoSys, this is where you’ll be able to specify
the HIPAA, the CHI, in the same subsets.
The Knowledge Source Server is our web-based system that allows you to
search the UMLS without having to load it on your own system. It’s also a
program interface and it allows you to download the various components of the
We are working on a new version for it. On the back end, this is going to
provide implementing web services to make it easier for people to access the
system. It’s going to be XML-based.
On the front end, we’re going to have portals that allow people to customize
their view of the Metathesaurus. Instead of us telling you this is exactly how
you should see it, we’re going to let you say, this is how I want to use it.
We’re going to have the prototype for that done by the AMIA meeting in 2005,
later this year, and implementation in 2006.
And what I’ve listed on this page are just the other — painted a complete
picture of the UMLS, if you don’t know it already. The other three components
are the Semantic Network, the SPECIALIST Lexicon, and the natural language
processing programs. None of those have any specific changes, so we won’t talk
about those anymore.
Also wanted to give you an update on RxNorm. I asked Dr. Nelson what he
wanted me to talk about, and this is what he said, and if there’s any mistakes,
they’re my mistakes, not his, because I didn’t let him look at my slides.
We are currently producing monthly releases of RxNorm. We plan to have
weekly releases available by the end of the year.
We are maintaining a harmony with the UMLS. Because RxNorm exists as a
source in and of itself, that you can get stuff from the UMLS but it’s also a
source within the UMLS, we need to make sure that those are in synch.
So we’re doing this two ways. First, we include RxNorm updates in every
Metathesaurus release, and we also re-think Rx-Norm files after every
And we’ve been making some major improvements in the process and product
code. We’ve been learning a lot from what we’ve been doing over the last year.
We no longer have the problem of returning RxNorm, which I’m sure will make
many people happy. We’ve also been working a lot to improve how we’re training
staff so that we can bring more people on and make this a more efficient
We’re incorporating more sources. For First DataBank and Micromedics, we
have agreements signed and in place. Gold Standard, we’re going to have an
agreement any day now. Medi-Span, they are reviewing that agreement and we hope
to have that in place very shortly. And we are also getting NDC codes where
we’re able to obtain them, which includes the FDA website, and (?) was just
telling me that by October, we will be able to get those directly from them in
a more complete format through the structured product label. It’s a very good
And this is a place where RxNorm is being put into use, the CHDR system.
It’s a joint DOD/VA project that facilitates the exchange of clinical data
between their two independent systems.
So on the DOD side, you have their Clinical Data Repository that uses —
MR. BLAIR: You might have this on the chart, but could you tell me what
“chedder” stands for — c-h, what?
MS. AULD: That’s what I’m just telling you now.
MR. BLAIR: I missed the acronym.
MS. AULD: Yes, it’s the Clinical Data Repository/Health Data Repository.
MR. BLAIR: Thank you.
MS. AULD: And it consists of the DOD Clinical Data Repository, which uses
First DataBank, and on the other side you have the VA Health Data Repository,
which uses NDF/RT. And RxNorm is the link between the two that allows them to
communicate, so it’s mapping the First DataBank and NDF/RT so that they can
And I believe that this system will be operational by the end of this year,
I think October, but I’m not positive on that.
That’s all I was going to cover with RxNorm.
I want to talk a little bit about some of the harmonization efforts that NLM
is working on. Our primary focus is on insuring that the vocabularies that we
directly support and maintain are in alignment, because we do not want to pay
for the same thing to be created in more than one system unless we absolutely
So the three that we are concerned with are LOINC, which we support through
a contract with the Regenstrief Institute; RxNorm, which we directly developed,
and SNOMED CT which, as you know, we have the contract and license agreement
for use in the UMLS.
So, first let’s talk about the harmonization between SNOMED and LOINC. This
is actually our biggest concern, because there’s a lot of overlap between
SNOMED and LOINC.
We have been talking with both the College of American Pathologists and
Regenstrief to come to an agreement for how we can resolve this.
One of the biggest factors affecting this is that CAP has an agreement in
place with the National Health Service, the U.K. National Health Service, that
limits what they can do.
The two options that we have for how to move forward on this are defining
the specific scopes for SNOMED CT and LOINC such that any future development
that they create for the two terminologies will be mutually exclusive. This
probably isn’t going to work because of that agreement with the National Health
So our other option is to clarify the appropriate usage of each of these
vocabularies within the U.S., and if we do so, we would be flagging that usage
within the Metathesaurus through the Content V Flag. It’s not the best
solution, but it’s probably the most workable solution at this point in time.
DR. FITZMAURICE: Excuse me, Vivian. What is “CVF?”
MS. AULD: That’s the Content V Flag, which I talked about in the UMLS pages.
DR. FITZMAURICE: A flag on top of each variable that’s either up or down?
MS. AULD: Effectively. It allows you to create that subset within the UMLS
so that if something has a flag for usage of LOINC within a specific area, it’s
going to get that flag and you can just pull that subset.
DR. FITZMAURICE: Thank you.
MS. AULD: Clear as mud, isn’t it?
We’re also looking at harmonization between SNOMED and RxNorm. This one is
colored by the same agreement between CAP and the National Health Service but
it’s not as much of a problem in this case.
SNOMED and RxNorm have different definitions, or different views, for what
constitutes a drug, so what we’re doing is within RX norm, we’re making the
links explicit so that it helps to clarify what those differences are and how
they should work together. And this is something that we’ve gone a long ways
towards creating all the necessary links but there’s still a lot to be done.
And we’re also, in a general sense of harmonization, making sure that we’re
coordinating all of our efforts with ONCHIT, especially in view of the RFP that
they recently put out.
One other project that we are working on which half of it affects
harmonization is the contract that we have in place between NLM and HL7. Their
contract has two parts.
The first is aligning HL7 message standards with CHI standard vocabularies.
This piece is under the auspices of NLM. And this really has two parts to it.
We’re specifying which subsets of standard vocabularies are valid for
particular message segments, and we’re also asking HL7 to replace the lists of
coded values that they maintain with subsets of standard variables where
So if they have a set of coded values that is more appropriately in SNOMED,
we’re asking that they talk with CAP and make sure that those are over in
SNOMED rather than HL7. And it’s again we just don’t want to end up covering
the same information in two different places.
DR. COHN: Vivian, can you just clarify — your slide says 2004.
MS. AULD: That’s when the contract started.
DR. COHN: Oh, really? So it’s been going on almost a year already.
MS. AULD: Yes.
DR. COHN: Okay, thank you.
MS. AULD: And the contract will last for three years total. The first part,
the NLM-initiated part, will last the entire three years. The second part that
I’ll talk about in a minute will only last for two years unless we decide to
The second part of this contract is on behalf of ASPE. Suzie Burke-Bebee is
the technical lead on the Federal side for this.
And what this is doing is creating implementation guides for transmitting an
entire electronic health record between systems. And it’s intended between
systems that are not designed to talk to each other. We want to make it so that
they can talk to each other.
They successfully designed a prototype and next April we’re going to test it
out between live systems. And we are on the schedule, I believe, to give a full
presentation of this entire project in February of next year, I believe, so I
won’t waste your time with any more on that.
MS. AULD: The next thing that I want to talk about is the various mapping
projects that are underway at NLM. And this probably is going to have the
biggest impact to the conversation that you’re talking about, secondary use of
clinical data. There are 90 projects underway.
The first are looking at mapping between CHI standards and HIPAA code sets,
and specifically what we’re trying to facilitate here is mapping between
clinical vocabularies and administrative vocabularies so that you can gather
the information at the point of care and automatically generate appropriate
So the first map is SNOMED CT to ICD-9-CM, and it will eventually be SNOMED
CT to ICD-10-CM. This map is being created by the College of American
Pathologists, but we also have NCHS working with us on this, we have CMS
working on it, we have AHIMA helping to validate it. And this will likely be
the first draft mapping that we have available on the UMLS.
We’re also working on SNOMED CT to CPT.
MR. BLAIR: What was on that last one?
MS. AULD: Yes?
MR. BLAIR: I’m sorry — let me get over. On that last one, is there an
approximate availability date?
MS. AULD: The draft map will be available by the end of this year. It’s not
determined yet when the final map will be available.
MR. BLAIR: And that’s for the SNOMED to the HIPAA terminologies?
MS. AULD: That’s for the SNOMED to ICD-9-CM. It will be available by the end
of this year.
MR. BLAIR: Okay.
MS. AULD: The SNOMED CT to CPT, we have CAP and the American Medical
Association working on this. We also have Kaiser, we have Simon, working on
this as well. They’ve put together a proposal for how we might want to go
forward with this. And again we have CMS working on this as well to make sure
that we’re fitting in to their goals as well. There’s no estimation date for a
draft map from that project.
The third one that we have is a mapping between LOINC and CPT, so for this
one we have Regenstrief and the American Medical Association working on it. The
draft map is being created by Intermountain Health Care under Stan’s direction.
And that’s probably going to be created in three different phases, the first
phase effectively being the things that are just incredibly obvious — A, no
questions go to B. We can just take care of the ones that are very simple and
get those out of the way.
And then it goes to the phases, to the most complex where we’re just not at
all clear on how you would map from one item to another and there’s going to
have to be some form of decision rules built into it.
So those are three CHI standard to HIPAA code sets that we’re working on.
We are also working on projects that are mapping from SNOMED CT to other
vocabularies. The first one is
MedDRA, which is the Medical Dictionary for Regulatory Affairs. And this is
a mapping that we have been told, I think it’s by the FDA, that they could
definitely use this map, but the usage case is not completely clear, so we want
to make sure that we have a very distinct usage case so that we know exactly
what map needs to be created before we proceed any further. And I think we’re
close, but close can be a relative term.
We’re also looking at mapping between SNOMED CT and the ICPC, which is the
International Classification of Primary Care. In order to do that, we’re going
to make use of a map that already exists between SNOMED CT and ICD-10 which
we’re still trying to get a copy of. Many people, many good people, are working
hard to try and facilitate that.
But on the other side, we have a map from ICD-10 to ICPC that we already
have in the UMLS, and once we have that second piece, we’ll be able to go
directly from SNOMED to ICPC.
The next one is SNOMED CT to Medcin. Medcin is not currently a source within
the UMLS, so we’re working on getting that incorporated. It’s not a
straightforward process, so we have some very good people working on trying to
figure out exactly what part of it should be represented and how and once
that’s done, then we can start working on the actual mapping.
SNOMED to MeSH, which is the NLM Medical Subject Headings, that’s a project
that’s going to start up sometime this fall under Dr. Nelson’s care and
And CAP has provided us with the mappings that they have between SNOMED CT
and the various nursing vocabularies, specifically NIC, NOC and NANDA, so we
have those available on the UMLS.
So, there are several key assumptions about mappings, but most of these came
out in what I was just saying on the previous slide. But they are worth
reiterating because this isn’t going to work unless we follow these
First, the participants in any mapping project have to include the producers
of the vocabularies on both ends, prospective users and recipients of the
output. And, for example, this would be health care providers, payers, as
testers and validators. In other words, you have to make sure that you have all
your bases covered in terms of who’s developing it so that they can make
appropriate changes to the vocabularies and also the people who are actually
going to be using it so we can give them a worthwhile product.
Once you create a mapping, you have to update it every time the source on
either end was updated, so we’re trying to make sure that it is part of the
process of updating a source to also update the map at the same time. We’re
trying to streamline that so that it doesn’t become an added burden.
All these mappings are going to be distributed in the UMLS. They can also be
distributed in other formats as well, but they’re definitely going to be in the
UMLS. And they’re going to be governed by the terms applicable to the sources
on both ends.
And mappings are still an R&D problem. It’s not something that we can
just give you a final product right now and know 100 percent that it’s going to
do everything it needs to do. It’s something that we’re going to put on the
table, get people to use it, and then as we use it, we can improve it as we go.
And that’s all I wanted to tell you. So, questions, please.
MR. REYNOLDS: Vivian, thank you very much. I know Jeff has a question first,
MR. BLAIR: Vivian, thank you. A very informative presentation helps us —
where’s that mike? Oh, gee, I haven’t even asked a question yet. It just fell
MR. BLAIR: I had no idea that NLM was working on all of these mappings. I am
delighted to hear that, and Godspeed.
MS. AULD: Thank you.
MR. BLAIR: I would be interested, very interested, if some of the major
health care IT software development vendors, the folks that are producing
commercial electronic health record systems — have you begun to have interest
in some of these mappings, and if so, or in SNOMED in particular or RxNorm? In
short, if you have had interest, what areas are they most interested in, in
terms of incorporating it or using it in their electronic health records?
MS. AULD: Very good question. I have had side conversations in various
conferences of people who find out that we are working on various mapping
projects and are extremely interested. But the conversation usually doesn’t go
beyond their saying “we see that it might possibly have an impact on
development of our systems.” But they don’t go into specifics.
So, in other words, I’m getting a lot of people who are intrigued that we’re
doing all this work, but they’re not telling me exactly how they’re going to
use it because they’re waiting to see what we’re going to produce first.
MR. BLAIR: That’s understandable. So let me take the question one level
MS. AULD: Okay.
MR. BLAIR: Have any of these vendors indicated what they would need to
enable or facilitate their adopting these terminologies? Have they expressed
they need any types of support or incentives or anything else other than, you
know, making these available?
MS. AULD: I don’t have an answer to that.
MR. BLAIR: Okay.
MS. AULD: I’m not hearing answers to that question. But it’s something that
I’ll definitely take back and see if we can explore that and try and get an
answer to it.
MR. BLAIR: Or the question might even pursue it as the corollary — what are
the impediments to them rapidly adopting these terminologies and mappings when
they’re available? Vivian —
MS. AULD: Okay, I’ll see if I can find out.
MR. REYNOLDS: Okay, we got Michael, Simon and then Steve.
DR. FITZMAURICE: Several questions. The MedDRA use case, if you map SNOMED
CT to MedDRA and you map SNOMED CT to CPT, can you then map CPT to MedDRA using
the daisy chain? And then can physicians report adverse drug events using CPT
and this mapping will turn it into MedDRA?
MS. AULD: In theory, yes, definitely.
MR. BLAIR: Wow.
DR. FITZMAURICE: In theory, of course this can play, I guess, but is that
maybe an end goal or —
MR. BLAIR: Wow.
DR. FITZMAURICE: — is that something that would be desirable?
MS. AULD: If the use cases of the two mapping pieces fit nicely together,
then yes, you can definitely create the map from MedDRA to CPT — no, CPT to
DR. FITZMAURICE: The reason I’m asking is that Congress is considering bills
to have patient safety events reported to a database, to a research agency
perhaps such as AHRQ, and it would be very useful if this mapping could make
things easier for physicians, for hospitals, to report something that they may
want to report voluntarily. So that’s what prompted that question.
DR. FITZMAURICE: Next question. When new codes are needed as a result of
this mapping (?) — if we had a SNOMED code, then it would map perfectly. Is
CAP very amenable to producing these new codes that would improve the mapping?
MS. AULD: Yes, they’ve been very helpful, very interested in making sure
that they do whatever needs to happen to make these usable products.
DR. FITZMAURICE: Great.
MS. AULD: They’re definitely on board.
DR. FITZMAURICE: When does the current contract at NLM has with SNOMED run
out, and what are the implications if it does run out?
MS. AULD: I believe it runs out the end of 2007. We have set this up so that
if we choose not to renew it, we have the right to perpetually use the latest
version of SNOMED in the UMLS and we can build on that to fill the future need.
DR. FITZMAURICE: It seems to me that five years may not be enough time for
us to construct all the value of SNOMED and that if we could continue a good
working relationship, maybe pay them something to continue making improvements,
then that might be beneficial for all of us. Is that being thought of?
MS. AULD: Yes, it’s being considered, it’s being discussed. We’re getting
input from various other Federal agencies and those outside the government with
their recommendations for whether we should or shouldn’t renew it, what would
be required in order for them to use it, things of this nature.
DR. FITZMAURICE: I’ve got two last questions.
DR. FITZMAURICE: I’m going to the first slide you have on Page 3.
MR. REYNOLDS: Michael, this would be considered a hearing.
DR. FITZMAURICE: It has to do with the RxNorm update and using RxNorm. As
RxNorm has agreements with FDB, Micromedics and others, how can the users use
their information content with the RxNorm link? I can envision a use case
where, oh, now I’m using Micromedics; I want to use First DataBank. Can I daisy
chain through the RxNorm name to use that information content, and then do I
have to get licenses from both?
MS. AULD: That is the purpose behind RxNorm. It’s intended to be a map
between the various sources within it.
I should know, but I don’t know whether or not you have to have agreements
with the sources on either end. I would imagine that you do have to — Randy’s
nodding his head — because we’re using the UMLS model wherever appropriate and
that is definitely the model that we use within the UMLS. So I would expect
that you would have to have agreements with both..
DR. FITZMAURICE: The last question is: In the past two years, AHRQ has been
very happy to support NLM to the tunes of several millions of dollars to do a
lot of work. Can you tell us —
MS. AULD: I forgot to mention how much we appreciate that.
DR. FITZMAURICE: Which work is being funded by AHRQ of what you presented?
MS. AULD: Which is being supported by AHRQ? The mapping efforts are
definitely being supported. A lot of the NLM/HL7 contract on what I call the
vocabulary side is being supported by it. RxNorm development is definitely
being supported by the funds from AHRQ.
There are pieces of the UMLS, but I couldn’t tell you expressly which those
are. And I mentioned that we have a contract with Regenstrief for the
development of LOINC; you’re partially covering that.
I think those are the big ones.
DR. FITZMAURICE: Great. And we’re very happy to do it because you have the
expertise that we don’t have, and it’s a pleasure to help work together on
patient safety issues.
MS. AULD: We definitely appreciate it. Thank you.
MR. REYNOLDS: Simon?
DR. COHN: Gosh, after that, I’m not sure — it’s hard to even think of a
question you haven’t asked, and further I’m precluded from asking any of the
really good questions, but I’ll ask one anyway.
This morning, we actually heard a number of presentations from people in
MS. AULD: I apologize for not being here.
DR. COHN: Well, that’s no problem, but we told them we would ask you a
question about all of it. And I don’t know whether this is an issue that is
related to lack of communication, lack of understanding, or really lack of
functionality, but a number of them, as they talked about obviously expanding
into SIGs issues relating to — gosh, what was it? —
MS. GREENBERG: Prior authorization.
DR. COHN: — prior authorization, et cetera, kept asking for — geez, really
what we need to do is to have access to what they described as a central
information code set repository, so they sort of knew what was there, what was
out there in existence, so they weren’t out there replicating everything and
starting from scratch.
And obviously I couldn’t ask you why there was this gap. What are we missing
here? In some ways, you would think that the Metathesaurus might be such a
repository, but maybe that’s not exactly what they need, or maybe there’s
something that’s more than what the Metathesaurus is. So maybe you can help us
MS. AULD: Well, let me ask a question back. Are they talking about a
description of the entire code set or are they talking about specifics of the
DR. COHN: I think they were talking about a repository where they can go to
to get all of the codes that they might need for whatever purpose.
DR. FITZMAURICE: They may be subsets, but they’re subsets for a particular
application, say, of SNOMED, for example.
DR. HUFF: For example, in the coded SIG, you know, there’s a part that
describes the kind of actions where you’re supposed to take the drug or you’re
supposed to rub the drug on your skin. There are routes and that sort of stuff.
And they need a particular code set that fits into their model for that exact
And what they’re looking for is to say, well, we can come up with a content
for now, but we don’t want to keep that forever, we don’t want to distribute it
to everybody. We want it someplace where everybody can get who wants to use
this new standard coded SIG.
And I think the question is, you know: Is that something you guys see within
your purview —
DR. COHN: Well, I think I also heard it in a slightly different way also.
They didn’t want to even make it up. They wanted to go to a place and identify
terms or these hundred terms and be able to pull them in so they didn’t even
have to make them up. Stan, am I off on that? That was the other piece.
DR. HUFF: Well, I think yes. I think I probably heard both things, but, I
mean, in some cases I think they are the experts that would come up with a
list; in other cases they wanted to say, gee, we think it’s likely that
somebody else has done this, you know? Is there some place to go to find them,
kind of thing, so —
MS. AULD: From what you’re describing, I think the Content V Flag within the
UMLS would probably be a very good way to pull those subsets together, but
that’s with the assumption that that code set already exists within the UMLS.
As long as it already exists there, we could definitely create these subsets.
We just need somebody to give them to us or tell us what’s needed so that we
can work with the experts to pull those together.
If the codes don’t already exist, I would still want to hear about it
because then we can help to facilitate in making sure that the correct people
are coming together and creating them and putting them into a format so that
people can use it.
So I think my short answer would be: Whoever made that comment this morning,
come talk to me and tell me what is needed so that we can find a solution.
I would think that, depending upon the exact nature of what it is that they
are looking for, it may or may not make sense for it to be in the UMLS. But
even if it’s not, we would want to work with you to make sure that it’s in the
MR. REYNOLDS: Okay, Steve?
DR. STEINDEL: Yes, thank you. A lot going on, Vivian.
MS. AULD: Yes.
DR. STEINDEL: I’m sure you’re having fun.
MS. AULD: I am.
DR. STEINDEL: A couple of observations and questions.
First of all, you made the comment and I think it’s been lost. You made it
several times but I don’t want it to get lost. These mapping projects, I think
we really need emphasize that they’re use case specific.
MS. AULD: Definitely.
DR. STEINDEL: That just because we have a map to this and a map to that, it
is a use case specific map and it can be used for that purpose and validated
for that purpose. But if we try to use that map for another purpose, it may not
MS. AULD: And to take that one step further, we definitely envision that
there will be multiple use cases between two specific sources. So there’s not
just going to be one SNOMED to — for example — CPT mapping; there is likely
to be several mappings, depending on the use case.
DR. STEINDEL: That brings to my comment, because I’m very concerned about
the comment that was made about the — well, ICPC has a map to ICD-10 and once
we map SNOMED to ICD-10, then we automatically get a map to ICPC.
MS. AULD: In this case, that does work —
DR. STEINDEL: Yes, I imagine in this particular case, it does.
MS. AULD: — because the usage cases do match up —
DR. STEINDEL: Yes, but —
MS. AULD: — we believe.
DR. STEINDEL: — it’s not going to be a full map to ICPC and SNOMED.
MS. AULD: No.
DR. STEINDEL: So, you know, I think —
MS. AULD: Thank you for clarifying that.
DR. STEINDEL: — also the same analogy that Mike was using where we have a
map to this to CPT and a map to that to CPT, therefore, we must have a map
between that — it’s the same type of analogy here. So I think we need to be
careful with extending these thoughts about mapping.
MS. AULD: Exactly. And that’s why my response to him was it depends on the
DR. STEINDEL: It’s on the use case. And that’s what caused me to emphasize,
re-emphasize, so we make sure that it’s in multiple places in the transcript.
MS. AULD: Exactly.
DR. STEINDEL: Then to my specific question, and this was a question that
this Committee had a lot of problems getting an answer to when we were looking
at PMRI terminology, and that is: What is the penetration of use of SNOMED CT
in this country now that the license is in place and it has been available
through the UMS for about a year? Does the Library have any indication of that?
MS. AULD: No, and that is actually something — in preparing for this
testimony, I was talking with various people around the Library working with
UMLS, and one of the things that we would very much like to do is find out how
many people are actually using the UMLS as a source for the various
vocabularies. We don’t have an answer to that. We know how many users we have,
we know how many people are downloading it, but we don’t know necessarily what
they’re using it for and how.
We do collect usage information once a year, but we’re still in the process
of developing a good questionnaire so we can get useful information. And part
of that will show us in the future how many people are using it to get SNOMED
For the moment, most of the responses that we’re getting show that the UMLS
is being used for research but not much beyond that.
DR. STEINDEL: But that still doesn’t indicate that they’re using SNOMED CT?
MS. AULD: No. We don’t have anything specific for that at this point. But
it’s definitely something that we want to resolve, because on one hand it’s
nice to have 114 different sources because you can encompass a very broad
spectrum, but is it the right spectrum, is it what is really needed out there?
And are there pieces of that that we’re expending resources to maintain that
don’t necessarily need to be maintained?
That’s something we would like an answer to, and at some point in the future
I’d like to be able to give you an answer to this question, but I don’t have
DR. STEINDEL: Thank you.
MR. REYNOLDS: Jeff has a follow-on on that and then Randy and then that’s it
for this particular session.
MR. BLAIR: A couple of dimensions to this in terms of the acceptance and
adoption of SNOMED CT.
I’ve heard some criticisms of the direction in the U.S. to utilize SNOMED CT
and the UMLS, and it’s hard for me with my limited knowledge to sort out how
much of that is an opposition because SNOMED is not licensed for free in those
other countries that are doing the criticism versus valid criticisms.
And as you probably know, Vivian, I and many of us on this Subcommittee have
strongly supported the idea of clinically specific terminologies, SNOMED, LOINC
and RxNorm being at the core. I would hate to see all of the progress that
we’ve made derailed if some of those criticisms turn out to be accurate and we
haven’t prepared Congress or the Administration to realize that there are
impediments to adoption that we don’t have in our plans.
So it kind of gets back to that other comment that I made, which is if
vendors haven’t been adopting it, what do they see as the impediments? They may
or may not be similar to the international criticisms. But I think we ought to
be aware of those criticisms and make sure that they’re projected in our plans
so that somebody doesn’t come up and blindside us in the future, saying we’ve
gone down this path, that it’s flawed, because we haven’t addressed X and X and
X and X.
So the other piece that I might suggest is that, assuming you’re familiar
with the GELLO initiative? —
MS. AULD: Yes.
MR. BLAIR: Great. And I would hope that since that seems to be an effort to
facilitate the standardization and exchange of the rules for clinical decision
support, and since SNOMED is probably a very likely candidate to be part of
that, I would hope that there’s good and close communications between NLM and
SNOMED and the GELLO initiative.
MS. AULD: I do not know what our relationship is at this point in time, but
I will definitely take that back to Betsy and see what we can do to facilitate
MR. BLAIR: Thanks.
MS. AULD: Definitely.
MR. REYNOLDS: Okay, Randy?
DR. LEVIN: Just go back to this — Randy Levin from FDA — go back to
something Simon brought up about the SIGs and the terminologies for SIGs, that
FDA has been working with HL7 and other groups to harmonize on the dosage forms
and working with the Europeans, Japanese and the Canadians on standards for
units and measures for routes of administration as well as dosage forms and
standards for packaging.
We’ve been working with NCI to put that with their Enterprise vocabulary
service, to put that all in the NCI thesaurus, which will make its way into the
UMLS, so —
MS. AULD: Yes.
DR. LEVIN: — just to answer that, that we have terminologies for many of
those things that will eventually make it into the UMLS.
DR. FITZMAURICE: A question of Randy — are those terminologies compatible
with what terminologies we’re using in the United States, such as the SNOMED,
or any other U.S. terminologies?
DR. LEVIN: It’s compatible with the terminologies that we use for regulatory
purposes and that would be in the labeling for each one of those.
DR. FITZMAURICE: Compatible with MedDRA?
DR. LEVIN: MedDRA doesn’t have those terminologies but it’s compatible with
our regulatory and as well as the regulatory purposes of these other regions.
DR. FITZMAURICE: And the regulatory terminology is in UMLS?
DR. LEVIN: It will make its way into UMLS because we’re collaborating with
the NCI, using their thesaurus for our terminology.
DR. FITZMAURICE: Okay.
MS. AULD: I don’t remember the schedule, but I know it’s on the schedule.
It’s — yes.
DR. FITZMAURICE: It makes sense, yes. NLM is the focal point for the
terminologies. We’re putting everything else there.
MS. AULD: Yes.
MR. REYNOLDS: Vivian, thank you.
MS. AULD: Can I make one last comment about SNOMED before —
One thing to remember when you’re talking about, when you’re hearing
criticisms about, the usefulness of SNOMED and impediments to adoption is the
reason that NLM supported and worked so hard to establish the license with the
College of American Pathologists is because all the evidence was pointing to
SNOMED being the best choice for that segment.
And so our goal was to remove the barrier of cost as an impediment to use.
Now that we’ve removed that barrier, we want people to start using it and give
feedback and work to try and improve it to see whether or not it really is the
best solution or whether something else needs to happen.
So I’m glad that people are looking at it critically, but their criticism
needs to be constructive and coming to either NLM or CAP so that we can feed it
back into the process and improve it.
MR. REYNOLDS: Okay. Vivian, thank you —
MS. AULD: You’re welcome. Thank you.
MR. REYNOLDS: — very much. Our next panel that will come up —
MS. WARREN: Just one comment, Vivian. When you’re seeing how many people are
downloading SNOMED from the UMLS, I’ve actually done that myself. It takes a
very long time and it would be cheaper for me to buy it from CAP on a disk
that’s already done. So you might want to look at talking to CAP about how many
people are coming directly to them and buying it on a CD as opposed to just
downloading it there.
MS. AULD: Okay.
DR. STEINDEL: A corollary to that. We’ve made a decision at CDC to extract
it from UMLS and put it in a format that we can provide to our public health
MS. AULD: Oh, okay.
DR. STEINDEL: — because of what —
MS. AULD: Because of that very issue, okay.
DR. FITZMAURICE: So does that mean that everybody can be your public health
partner and get it for free?
DR. STEINDEL: Everybody except certain agencies within HHS.
MS. AULD: Thank you all.
MR. REYNOLDS: That was the friendly banter portion of the program.
MR. REYNOLDS: Okay, our next group, I’ll go ahead and start introducing them
while we’re getting set up. We’re going to continue to talk about secondary use
of clinical data to support billing, SNOMED CT and ICD-9-CM, James Campbell
from University of Nebraska.
And then we’ll move directly into auto-assisted coding. Valerie Watzlaf of
AHIMA and Mary Stanfill of AHIMA will take us through that, so, James, as soon
as you’re set up, you’ve got the ball. And we welcome all of you.
AGENDA ITEM: Presentation — DR. JAMES R.
DR. CAMPBELL: Good afternoon, Mr. Chairman. Thank you for the invitation to
come and speak. My name is Jim Campbell. I’m an internist at the University of
Nebraska Medical Center, and I think by way of full disclosure, I’m also a
working member of the Clinical LOINC Committee and of the HL7 Clinical Decision
Support Committee; I’m a member of the SNOMED editorial board and I chair the
mapping activity for that group.
MR. REYNOLDS: You should be sitting up here.
DR. CAMPBELL: As I was preparing my thoughts for this presentation, I was
musing on the material that was sent out, and I was observing this morning that
in an age where, and when, we are successful in implementing a complete,
nationwide, decentralized NHII-based clinical record, all clinical information
becomes of secondary use in a statistical sense.
And so I wanted to respond to the Committee’s questions in a little bit
more basic way, but I think in one that will be relevant, and in light of Ms.
Auld’s comments, I also find that some of the material that I have prepared
hits upon similar areas and so I’ll try and skim over those where appropriate
and/or contrast issues where necessary.
In general, I would like to hearken back very briefly to the information
architecture that the Committee proposed back in 2003 for national vocabulary
convergence. I would like to talk a little bit about problems with use and
re-use of information within each of those three clinical layers and then
comment just briefly on issues of knowledge information and what I call the
“inferred medical record” and what is ahead for that.
I don’t think that I need to basically revisit what has happened, and I
really applaud the Committee for their works in helping to push information
technology convergence forward.
All of this is based upon a three-layer model in which core reference
terminologies serve the central care needs of patient systems. It integrates
closely with legacy clinical data sets that have to be brought within the core
and also deals with external or administrative systems in such a way that the
use cases for those systems or for those users are met.
And this is where mapping primarily comes in, but I’m going to talk a
little bit more in detail about what I call mapping in just a little bit.
The core reference terminologies that we’re talking about, of course, are
SNOMED CT, LOINC, RxNorm and its relationships to NDFRT and the unified medical
device nomenclature system.
And I just want to revisit very briefly a definition that may not be
familiar to all our audience, and that is that a reference terminology is a
concept-based vocabulary system which employs compositional forms — that is,
it pieces together necessary elements of a definition in order to come up with
a definition that a computer can understand about that concept.
And basically it ties that all together within a network of meaning or
relationships which is distributed along with the terminology, and that is a
necessary and integral part of the whole, if you will, if you’re going to
deliver on all of the vision of what reference terminology should provide for
clinical information systems. And I’ll come back to that briefly.
Just to review quickly Layers 2 and 3 in the clinical code sets, or legacy
schemes, we have nursing classifications, ICFDH, ICPC and a variety of drug
databases in use in the U.S.
And then of course in the administrative classifications, we’ve got systems
like MedDRA, DSM, ICD-9, ICD-10, CPT, common dental terminology, national drug
codes, HCPCs et cetera.
Starting then with a discussion about re-use of information within the core
itself, I would just like to revisit briefly why a single convergent model is
necessary for core content.
Interoperability basically is all about re-use of information between
systems so that, for example, my clinical system can send out a query, gather
information on my patient from other systems, bring it together in some sort of
a homogeneous way. This requires agreement on the fundamental elements which
define the concept. Okay, that’s central to reference terminology.
In addition, decision support technologies demand the additional
relationships of the reference terminology. Those relationships are just as
much as part of it as the terms and the concept identifiers that go along,
because without that, the whole thing basically falls apart and the computer
can no longer understand what’s happening.
So, both of these elements basically must be consistent and robust if we are
going to have shared systems and if we’re going to have decision support and
So, important shared use cases for the core reference terminologies, those
four that we’re talking about, are comprehensive, accurate and scaleable
recording of all health care events, and also, sharing of clinical information
in support of the NHII vision.
Now, I would suggest to you right now that we still have some barriers to
the appropriate shared use of information within the core. And some of that
inconsistency comes up because of differences in granularity and definition
between those few areas where we still have overlapping classes, and as all of
you described those very well.
And so I’ll not dwell on this too much but, basically, duplications within
this reference terminology core create data islands, because if we have systems
recording in these different duplicate codes, then basically we can support
messaging, we can send them information back and forth, but the machines cannot
understand it in an interoperable way.
And the three particular areas, some of which have already mentioned, are
SNOMED CT observables and LOINC, SNOMED CT medicinal products, along with NDFRT
and the RxNorm clinical drugs. And then finally, one that’s a little bit lower
down on the clinical radar perhaps, but
SNOMED CT physical objects and the UMDNS.
If you take a look at what problem this duplication creates for clinical
systems, here we have a representation, let’s say in my system, of systolic
blood pressure in SNOMED CT. And we have another representation, in a different
system, of that same concept in clinical LOINC, okay? We have definitions; we
can message them back and forth, but we can’t share meaning.
Now, presumably we can create equivalence mapping between the two and
resolve the question of making sure that I store it in my system the correct
way, but basically mapping does not provide complete clinical decision support
because of those other variables that I was talking about, basically the
relationships, which are important elements of the reference terminologies.
So you can imagine — and this is what I would hope would happen within the
discussions and so forth that are occurring within the National Library of
Medicine, is that these two discrepant models are basically replaced by a
shared model which pulls together all of the necessary meaning from the two
systems to uniquely define the concept, to easily equate it between the two
systems, and to give us all the necessary information that we need for clinical
decision support as well.
Such integration of core content within a shared model converges the
editorial effort, assures interoperable content, and, finally, eliminates
duplication as we go down the road. As Ms. Ault said, I think that creating
clear boundaries and understanding of editorial responsibility and how we share
work on this is really important to re-use within the core.
Now, right now barriers to some of that convergence are:
We don’t necessarily have agreement upon the model for those overlap
There needs to be a commitment from the terminology developers so that
convergence can occur.
It requires shared funding on business plan, which obviously has to be
And we’ve also got to recognize that the vendor community and the user
community for sure by and large do not have a very good understanding of this.
In fact, they find the whole area confusing, and they just want something
delivered that they can use reliably and reproducibly, and I think it’s our
duty to give that to them.
So what comments I would make in terms of re-use of information within the
core — I think it’s important to our success overall that further
consolidation efforts, which are going on, I think are important to promote
them and to encourage them, to endorse the NLM development of a single
convergent model so that we can have a road map for how we’re going to bring
these systems together.
And finally, I think there’s also educational efforts that need to happen to
basically begin to bring our vendor community and our clinical users up to
speed as to how all of this is going to fall together.
Now, I’m going to move from the core out to the Layer 2, where we’re talking
about clinical legacy systems that are important to the core because of their
content but are not represented well there right now. And those are basically
the systems I mentioned. Arguably, the most important are those for primary
care and nursing, but I think you probably have some discussion about prior
authorization. Nonetheless, I think we know the targets we’re talking about.
I think it’s important to understand that the goals of Layer 2 consolidation
are not simply mapping, and I want to define mapping in just a second in a way
which I hope makes that clear, because in Layer 2 we basically have, and we
recognize the fact, that there is content information in many of these systems
that is not now in our core systems. And this was especially true, for example,
in nursing for a long time.
So we have basically two goals to accomplish when we’re merging Layer 2
First of all, we have to model the clinical content in such a way that it
becomes consistent with the core. And then, secondly, we have to create the
mapping so that legacy data can be used for research and education.
Now, I think the best example of success here has been what’s been going on
within the nursing community in terms of bringing the nursing classification
systems into SNOMED CT.
There’s been a convergent nursing terminology work group now that’s been
meeting for four to five years that’s been shepherding this effort. Members of
this committee have basically chaired and directed that effort. And, by and
large today, we can see represented within that core are the nursing concepts
that we need, plus the maps that link them outward, as we saw from the NLM
A caution here: I think we’re talking about two different maps, okay? But I
wanted to bring up a project which basically the primary care work group of the
College of American Pathologists has been working on, and that’s NICPC.
Because of the fact that we believe it’s important content, because of the
fact we believe it needs to be clearly represented within the core clinical
systems, this is something that the primary care working group has
been pushing forward first as a modeling effort to make sure that we have
all the content and then secondly as a map.
Now, I would observe, in answer to Dr. Fitzmaurice’s earlier question, if
you map A to B and B to C and C to D, and if you have an equivalence-based map,
and even if you control all the variables, the frequency of correctness of that
map will always be less than one. And how far less than one it will be will
depend very much, very heavily upon what is the difference in granularity, the
editorial assumptions, and the content focus of these systems.
So, mapping always gets you part of the way, but it doesn’t get you all of
the way, especially if your goal is to transfer 100 percent of meaning.
Now, within this ICPC modeling project that we’re proposing from CAP, and
I’d be glad to share the working documents with anybody who would like to see
those, we’ve developed use cases to assure the content coverage is carried
forward for reason for encounter and also all of the concepts in use in the
primary care records.
But at the same time, this would support future research with clinical
systems that have employed ICPC for research in the past.
To give you a little bit of an idea about
complexity, I want to talk in just a second about why we’re interested in
knowledge-based maps, or rule-based maps, but this is one concept out of SNOMED
and how it might map to ICPC, where you can see we have up to 15 separate rules
that carry that one individual concept into different ICPC codes.
The difference here is one of granularity. Basically, we’re talking about
codes, or classifications, which focus on very different levels in the clinical
record. Both are correct, both have important information, but a simple
equivalence map is probably not going to be adequate for our ICPC map insofar
as we have organized it.
This is something that basically we have developed the use case for, and I’m
going to discuss the use case that we distributed to you for ICD-9 CM in just a
second. This has been reviewed and endorsed by the SNOMED and ICPC community.
We are basically putting together the work plan and the project costs. We’re
currently seeking endorsement from primary care organizations in the U.S. to
see if there’s a buy-in as to the need for this, with a notion that we were
going to submit it to the National Library of Medicine as a funded project.
Now, I’m pretty sure this is not the project you were talking about just a
little bit earlier.
MS. AULD: Yes, it’s a separate project.
DR. CAMPBELL: Okay. So you can see that we’ve already got examples of the
right hand and the left hand not necessarily always knowing what each other are
But this, I believe, is an important area, and whatever is the ultimate
architecture that needs to be adopted, you know, that’s a further discussion,
but I think we recognize the importance of this to overall utility of clinical
Now, there’s a number of things that we need to deal with in terms of
barriers to Layer 2 harmonization.
First of all, we’ve got the issues of expansion of the convergent model.
Secondly, we’ve got cost sharing and funding, which basically has to be
negotiated piece by piece every step of the way.
And then something that I wanted to get to, and that is there is very little
information, and this is something that Ms. Ault said earlier, there’s very
little scientific information about what constitutes good mapping, okay?
And I want to define mapping very specifically, as the process of creating
interoperable links from a fully coordinated concept within a reference
terminology to one or more assigned codes in a legacy vocabulary or
Now, there’s lots of times that mapping is applied to other things, and I’m
not trying to say that that is wrong, but this is what I’m talking about today
when I’m talking about mapping because it specifically relates from going to
the core concept where clinical records are out to the external schemes where
we need to, let’s say, generate the ICD-9 CM code, okay?
Now, maps have been in use for a long time. Back home, I’ve used a
computerized record for 22 years. For the past eight years, I’ve used SNOMED CT
in our clinical system, okay? And we have employed maps throughout that entire
time in use.
This, for example, is a clinical care screen from one of my patients where
you can see the patient problem list. And down the center right there, you
basically see the problem list and the interface terminology, which is my term
set, if you will, of SNOMED CT, my subset, that allows me to pick and choose
into my problem list quickly and easily.
You can see that I’ve selected my problems for today’s visit, okay, as a
part of my service recognition. But then in the background the maps that we
maintain from SNOMED CT to ICD-9-CM have basically supplied the billing codes
which go out on my bill, okay?
This is an example that is probably state-of-the-art in terms of what
mapping is like today — one to one,
or equivalency. The problem is that, in my experience, error rates in these
conventional maps are high, and disagreement upon what is a correct map are
And these are problems which I think we need to deal with, as I think was
mentioned earlier. These are technical challenges.
I’d like to enumerate some of the issues that we have to deal with when
we’re talking about problems with mapping between vocabulary systems.
First of all, there’s properties of the vocabulary systems themselves.
There’s differences in scope and editorial policy. These create differences in
assumptions about how things should be organized and at what level of
granularity, for example, if they exist.
Differences in granularity of the classification systems also create
problems which are always dealt with in these one-to-one maps incorporating a
lot of assumptions and heuristics. And the deeper you dig into them, the more
you realize just how difficult it is to understand them many times, the result
being, as Dr. Steindel had mentioned earlier, that the use case has heavy
impact upon a lot of these assumptions and that a universal solution is
unlikely to happen except in the simplest of cases.
Other problems with the vocabulary systems that create technical barriers to
the mapping include problems of context, okay, and this also goes in part to
the issue of use case, that basically it deals with the fact that many of the
complex classification systems have rules built into them which specifically
refer to issues outside of the simple concept itself.
For example, ICD-9-CM basically says this is excluded if such-and-such is
true in the patient record. That’s a patient level exclusion.
There are also context restrictions based on encounter information and
episode of care that are easily demonstrable within the ICD-9 constructs.
And the point is, these need to be dealt with in the maps if you’re actually
going to be successful and have a high rate of accuracy.
There are additional problems of update frequencies and map versioning.
There are problems within the vendor community, too, one of them being that
when the vendor implements a map, there’s always an assumption about their use
case which may or may not correspond with the use case for which the map was
In addition, every vendor has their own information model, which means that
they segregate data within their clinical systems in different ways.
And so the question of what is being mapped frequently differs between
clinical systems, and this in itself creates challenges and problems which
create technical error.
As Clem had mentioned earlier today, within all of this there is little or
no scientific research which really supports an understanding of what needs to
Now, from the standpoint of people who at least have approached me and what
I’ve heard about back within the SNOMED community, the financial and
reimbursement use cases are arguably some of the most important to penetration
of clinical information systems in the U.S. That’s just my personal
In addition, though, we have other clinical systems and classifications that
we need to think about mapping. Right now, I think Ms. Ault has already nicely
summarized what maps are available, and as a matter of fact, she certainly told
me a few things that I didn’t know.
I do want to mention and talk about a little bit more detail, the
reimbursement use case map that basically we’re working on now in the mapping
work group, and there’s been a hand-out distributed, and I think we have copies
for the audience, too, if they wish, on what the use case and definition of
procedures would look like for the ICD-9-CM SNOMED reimbursement use case map.
And I provide that basically — I suspect that most of you don’t need more
nighttime reading, but that it gives you a little bit of an inkling in terms of
some of the complexities that have to be dealt with in terms of managing the
assumptions and so forth that are involved in these maps.
But basically this map is designed to support near-complete or
near-automatic ICD-9 reporting from SNOMED clinical records to manage the
sources of technical error that I have talked about as much as we can to reduce
the error to the smallest manageable portion, to develop a paradigm which
hopefully will be transferable to future maps, because ICD-10-CM is going to be
here real quick.
Fortunately, many of the constructs are similar, so I think that a lot of
what we’ve been working on will carry over.
The scope of this map is SNOMED disorders, clinical findings and
context-dependent categories such as family history and the like, with a goal
of basically supporting U.S. vendors in implementing better clinical
All of this is basically organized around managing that ICD-9-CM map for
reimbursement support and making it more effective and to manage context in
ways which we haven’t been able to deal with before, namely, by extension in
the knowledge-based systems.
So, for example, in this small snapshot, if you will, from the map, you can
see that we mapped two SNOMED codes, the first, AIDS with volume depletion, and
the second, perineal pain, into their appropriate ICD-9 codes.
Now, the first concept happens to have a dual map, which means there has to
be two ICD codes that come out of that if you’re going to be correct, and
that’s handled within the map groups.
So in the first case, the first map group always maps AIDS the same way.
But you can see that the question of the volume loss is managed very
differently, depending upon other patient-level exclusions — that is, whether
there’s been post-operative shock or whether there’s been traumatic shock
that’s contributed to the volume loss.
And so this is an example of the way that patient context creeps into the
map. It has to be managed, if you will, if you’re going to be successful.
Likewise, in the lower example, you can see that whether the patient is a
male or a female changes the coding substantially.
And these are just simple examples that are present in large volume in the
Now, we’re in the position hopefully to release the majority of that for
review later this year because we’ve basically already gone through a lot of
the first round review. But we recognize the fact there’s additional clinical
relevance that must be dealt with and we also want to move it into a
So, for those of you who have been long anticipating GELLO, which is an
acronym for guideline expression language, here is an example of those same
rules which have incorporated the use of the GELLO paradigm along with the HL7
RIM — by that I mean a definition of what we expect the fields to be in the
record and how the data query then would look in terms of support of those same
rules in a standard construct which controls for the information model and
which controls for the expression language as well as the vocabularies
So in terms of where that all is right now, the use case documentation has
been developed and if you’re interested, you can look at the hand-out. The
project has been proposed to NLM and we’ve set up a tentative working plan for
evaluation and feedback. As Ms. Ault mentioned, we’re working with AHIMA on the
validation step, accepting that there always needs to be external validation of
these tools if they’re going to be ultimately accepted in the marketplace.
The standards discussions are underway with the HL7 groups in terms of the
constructs we would use for the knowledge base and how that should look, and
we’re working on deployment for demonstration later this year.
To summarize all of this, I would suggest to you that some basic mapping
requirements for interoperability require:
A clear and unambiguous use case and documentation.
They require editorial decisions that are independently reviewed and
I think that knowledge-based maps are going to become the rule, not the
exception, because everything else basically has too high an error rate.
That external validation of these maps by independent and knowledgeable
third parties must be a part of the business plan.
That there should be publication in the public domain.
And obviously, that release procedures need to be responsive to changes very
much as you said earlier. As either one changes, then the map must change in
response to that.
In terms of what NCVHS might consider doing, strategic agreement on map use
cases that are required for success for deployment of the computerized record,
I think a short list of what we need to do, and reaching consensus on that,
would be very helpful.
Agreement upon knowledge formalisms for mapping and how those procedures
should be developed.
Work in progress in terms of how to develop scientific methods and
evaluating for how we can validate and establish the utility of these maps.
And then, finally, promoting vendor review and acceptance of this whole
architecture as a necessary element of how we’re going to be successful in
achieving interoperable systems.
Now, just a few comments. I was a little surprised when I got the original
message from the Committee about the subject for today. A lot of what was in
there I call the “inferred record,” or the use case being that while
I’m implementing a diabetes guidelines, I notice that most of my patient
records don’t have a complete problem list or don’t have a complete list of
patients with chronic kidney disease, and so I basically have to go data mining
in my clinical records in order to be sure that I come up with all of the
necessary criteria so that I can identify all my patients eligible for the
So my knowledge engineer basically creates a set of rules in the guideline
which search the record for things like serum creatinine and albumen tests and
then implements standard criteria for how to establish the diagnosis. I would
call that an inferred diagnosis or something like that.
I would just like to make a couple of comments about experience that we have
had in the last two years when working on the SAGE project. You may or may not
be aware that the SAGE project basically is an experiment in guidelines
interoperability. It employs only NCVHS core vocabulary standards and all of
the knowledge features that we developed.
We are evaluating right now against three AHRQ guidelines in terms of the
needs, in terms of knowledge development and vocabulary support, and we are
cooperating with HL7 in terms of developing interoperable decision tools that
allow us to share knowledge between systems.
These things are a long way off. And I think that, if we’re talking about
obtaining near-term utility in terms of secondary use of clinical systems, we
need to recognize the fact that sharing knowledge is probably going to be the
most complex thing we do, and it’s tied inextricably to the issue of sharing
vocabulary, because in most of the knowledge offering that we have been doing
within the SAGE project, 50 percent of our effort is actually involved in
tweaking or manipulating the reference terminologies in order to be sure that
all of the necessary structures are there to properly support the decision
features of the guideline.
So in terms of the ability to deliver on the inferred record, I would
suggest to you that it’s equivalent to basically delivering on interoperable
decision support, that it requires not only the vocabulary system and the
interoperable formalism for shared knowledge but it requires that the vendor
community have implemented basically the knowledge deployment software and that
there’s widespread acceptance of these issues of data mining and inferred
diagnoses, which I think is a large question and problem in itself.
And so I think these issues are much further off than the maps which we hope
to be looking at this year, and I hope the convergent terminology which we can
be using within this calendar year.
And I think —
MR. REYNOLDS: Thank you, Dr. Campbell. Valerie, if you — which one’s going
to go first?
MS. WATZLAF: I am.
MR. REYNOLDS: Valerie, go first, please.
AGENDA ITEM: Presentation — VALERIE WATZLAF
MS. WATZLAF: Chairmen Blair and Reynolds, members of the Subcommittee, and
ladies and gentlemen, good afternoon. I am Valerie Watzlaf, Associate Professor
within the Department of Health Information Management in the School of Health
and Rehabilitation Sciences at the University of Pittsburgh.
And with me this afternoon is Mary Stanfill, Professional Practice Manager
of the American Health Information Management Association, or AHIMA.
On behalf of our Association and our 50,000 colleagues, we thank you for
allowing us this opportunity to provide input on issues related to
I believe that most of you are familiar with AHIMA. If not, we have placed
a brief description of the Association in our written testimony and we invite
you to visit our website at www.ahima.org if you wish to learn more about AHIMA
and the health information management profession.
Last year, AHIMA convened a work group on computer-assisted coding, or CAC,
to perform an extensive exploration of CAC technology. The work group was to
research CAC technology and related emerging roles for HIM professionals and
using this research, identify the best practices for evaluating and evaluating
this technology and develop use cases and required skill set for emerging HIM
The outcome of this effort was a set of practice guidelines, or a practice
brief, called Delving into
Computer-Assisted Coding, and it was to assist health care organizations in
preparing for the expanding role of this technology in the coding and billing
process. And a copy of this practice brief was submitted, with our written
testimony, for your review.
The practice brief discusses computerized tools available to automate the
assignment of certain medical or surgical codes such as ICD-9-CM and CPT and
HCPCs from clinical documentation that are traditionally assigned by coding or
HIM professionals as well as clinical providers.
It also outlines the driving forces shaping the current and future
applications of this technology. It examines application of the technology, and
it provides guidance about the steps necessary to position coding professionals
for the coming coding revolution.
It is appropriate at this time to disclose two pertinent current projects
with which AHIMA is involved.
AHIMA’s Foundation of Research and Education has a contract with the Office
of the National Coordinator for HIT to look at how automated coding software
and a nationwide interoperable health information technology infrastructure can
address health care fraud issues.
This project is comprised of two tasks.
The first task is a descriptive study of the issues and steps in the
development and use of automated coding, or CAC, software that will enhance
health care anti-fraud activities. And I am one of the co-principal
investigators on this task.
The second task will identify best practices to enhance the capabilities of
a nationwide interoperable health information technology infrastructure to
assist in health care fraud prevention, detection and prosecution.
The second project work AHIMA is conducting under contract with the National
Library of Medicine, as was mentioned. This task is to assist in the
development, review and testing of mappings between SNOMED CT and ICD-9-CM and
any successor HIPAA standards to ICD-9-CM. And Mary is part of that team that
is working on this project.
Let’s begin first by clarifying some of the terms that we will be using.
Electronic health record, or EHR, is the term we use to refer to
computerization of health record content and associated processes. This is in
contrast to the term “electronic medical record,” which is
computerized system of files that are often scanned rather than individual data
elements. Today when we mention an EHR, we are referring to a system that
captures, manages and maintains discrete health care data elements.
AHIMA defines computer-assisted coding as the use of computer software that
automatically generates a set of medical codes for review and validation or use
based upon clinical documentation that’s provided by health care practitioners.
CAC is often referred to as “automated coding,” but this can be
confusing as it implies a fully automated process with no human involvement
when these applications do require human review and validation for final code
assignment for administrative purposes. We prefer including the term
“assisted” when discussing these applications, as this more closely
characterizes how they are employed.
We must also distinguish between CAC applications and other computerized
tools that are currently utilized in the coding process. There are many tools
available today to assist coding professionals in manual code assignment,
including pick or look-up lists, automated super-bills, logic- or rules-based
encoders, groupers, and imaged and remote coding applications.
All these tools serve to assist a person in manually assigning correct
codes. They do not fundamentally change the coding process; they simply
facilitate the manual coding process.
In contrast, the CAC applications significantly alter the coding process
through automatic generation of codes for review by a coding expert who
validates and edits the codes rather than manually selecting them.
In our research, we found two approaches to CAC applications employed today.
One is structured input, and two, Natural Language Processing, or NLP.
Structured input or text, or codified input, is a form of data entry that
captures data in a structured manner — for example, point-and-click fields,
pull-down menus, structured templates and macros.
Structured input CAC applications are essentially a documentation system
where pre-defined clinical documentation is linked to the applicable code. As
the clinical documentation is created via the caregiver selecting applicable
clinical phrases, the linked codes are automatically assigned.
Natural Language Processing, or NLP, is essentially a form of artificial
intelligence that emulates the way people read and understand so that it can
extrapolate information from the written language the way the human brain does.
This software technology is applied to a text-based document and it uses
computational linguistics to extract pertinent data and terms and then convert
them into discrete data, in this case, the medical code.
NLP-based CAC applications may use either a statistics-based or rules-based
approach to assign the code, and often, a hybrid, or a combination of both, is
employed in the NLP system architecture.
With a statistics-based approach, the software predicts which code might
apply for a given word or phrase based on past statistical experience. The
rules-based approach uses programmed rules, or algorithms.
There is an entirely different mechanism to assist the coding process via
automation, and that’s the concept of mapping from a reference terminology
embedded in an EHR to a classification system. Theoretically, once an EHR
containing a clinical reference terminology — for example, SNOMED CT — has
been implemented, information captured in the EHR during the course of patient
care can be codified using the reference terminology, and an automated mapping
process may be employed to link the content of the terminology to the desired
code sets for secondary uses.
Thus, there are potentially three ways to accomplish the medical coding
First is manual code assignment, with or without the encoding tools.
Second is the use of a CAC application.
And third is mapping from a reference terminology embedded in an EHR to a
We would now like to compare and contrast these three methodologies.
The medical coding involves manual evaluation and review of clinical
documentation and application of coding and reporting rules to assign
administrative codes. This process may include the use of code books or
computerized tools — for example, encoders, automated super-bills, or pick
lists — and it may be performed by multiple individuals ranging from
non-credentialed or credentialed coding professionals to physicians.
In contrast, the coding process utilizing a CAC application is very
different. Structured input CAC applications are used by the caregiver, often
at the point of care, as a data entry mechanism.
The clinician captures clinical information by adhering to the software
application’s pre-defined structure for input. For example, he or she might
select applicable words or phrases from menus or utilize multiple
point-and-click fields to store the information. If a menu or field is skipped,
the application may prompt the caregiver for the missing documentation.
Implementation of a structured input CAC application first involves
developing, or tailoring, the structure for data input so that it closely
matches the clinical information that will be stored and maintained as health
Once clinical information is set up in a structured format, the codes are
assigned to the clinical words and phrases where applicable. The software links
these codes with the correct phrases so that codes may be captured as the
documentation is created. The list of codes that correspond to the
documentation is presented to the caregiver for review and validation and is
subsequently presented to the coding professional for review.
The coding process using an NLP-based CAC application is a little different.
This software undertakes the following processes almost simultaneously:
It evaluates the documentation resulting from a patient/provider encounter
to suggest codes that reflect the written word as documented.
Most NLP-based CAC applications use a combination of the statistics-based
and rules-based approaches. In most cases, the statistics-based approach is
applied first, and if errors are detected, then the rules-based approach is
applied. Then an extensive quality check is usually performed.
And as the software performs this analysis, it may also evaluate patterns
of documentation that are statistically different from the average
documentation for similar cases. In this manner, it identifies potentially
incomplete documentation so that physicians can be queried. Physicians are then
provided with feedback regarding documentation variances to help improve the
And then a list of suggested codes is sent to the appropriate coding
personnel to verify the codes. Code output from an NLP-based CAC application is
normally ranked, based on the computer’s level of confidence in the accuracy of
the codes generated. Typically, it ranks it in different levels. Number one
would be a high degree of confidence; the second would be more questionable
codes, or it could also be inability of the software to determine any potential
In the manual coding process and the coding process using a type of CAC
application, the final code set is determined by a person. However, the process
using CAC differs from the manual process in a significant way. CAC
applications suggest potentially applicable codes, rather than a coding
professional being entirely responsible for code selection. Therefore, the CAC
tools can significantly improve productivity.
It is important to note that CAC applications today are not capable of
generating codes on every single case. Manual coding must still be performed on
cases that don’t readily fit the defined input structure or on types of cases
that the NLP system has not previously encountered and therefore has no
framework from which to suggest applicable codes. So, with both structured
input and NLP-based CAC applications, humans perform some manual code
assignment, but it’s to a lesser degree.
Editing and validation of computer-generated codes may involve the use of
coding tools such as up-to-date code books, coding references, and encoders
that assist in determining the correct code assignment through text prompts.
The expectation is that, as the coding professional becomes an expert coder,
editing codes generated by the software is much less time consuming than an
entirely manual process.
In determining the final code set, the expert editor also applies modifiers
and other payer reporting requirements that often require contextual
information to be accurate — for example, Medicare’s Correct Coding Initiative
Even though CAC applications do not fully automate the coding process —
human review for final code assignment is still necessary — these applications
are beneficial. They increase coder productivity, and they make the coding
process for efficient. Therefore, there is return on investment.
The AHIMA practice brief includes a comprehensive includes a comprehensive
exploration of the advantages and disadvantages of CAC. Reported improvements
in the coding process include:
Improved coding consistency.
More comprehensive coding.
Enhanced coding compliance.
Decreased coding and billing costs.
Faster turn-around time, resulting in decreased accounts receivable days.
And enhanced workflow.
Benefits unique to structured input are related to the documentation
process. Structured input creates consistent and potentially more complete
documentation. The physician is prompted to add specificity to better reflect
clinical details in the ICD system, and this potentially eliminates the
physician queries. Also, structured input systems replace some dictation and
transcription, thus reducing associated costs.
A significant benefit unique to NLP-based CAC is that physicians may
continue to document using their preferred terms.
Lastly, many CAC applications offer mechanisms to query data from their
systems. Therefore, we anticipate improved ability to analyze administrative
data. The use of CAC data for such purposes as Joint Commission auditing, QA
measures, performance studies, credentialing and research is an attractive
feature of this technology. CAC applications may be characterized, then, as
bridge technologies that serve the pressing need to improve today’s manual
CAC significantly impacts workflow. With CAC that uses structured input, the
entire coding workflow from the point of documentation through claim submission
Physicians are directly impacted, as they must document using the
pre-defined structure, tailored to his or her practice. Cost savings are
reported from elimination of transcription and dictation.
But there are reports that some systems increase physicians’ documentation
time, causing a decrease in throughput of patients and increasing waiting
times, in these cases. However, when CAC works well, it does provide a closer
relationship between data capture in real time and code assignment. With
NLP-based CAC, the documentation process does not have to change. In fact, this
type of CAC may be transparent to the physician.
CAC also impacts management of the coding process. The work load can be
managing by routing work to queues based on specific parameters such as the
report type, the particular codes suggested, or the CAC application’s
confidence level. This creates a much smoother workflow and allows coding staff
to focus and become expert in certain specialties. CAC applications also enable
management of the coding process through tracking and administrative reporting.
In short, incorporating a CAC application in the coding workflow has a
profound effect on the coding staff.
It is more difficult to generalize the effect in terms of staffing changes.
I noted that CAC has demonstrated improved productivity. However, specific
performance in terms of coding accuracy for correct reimbursement is largely
In our research, we found that the coding quality in many of the systems
employed has not be assessed in actual practice, and overall, no CAC
application to date has an accuracy rate that meets the existing industry
standard of 95 percent that coding professionals are expected to meet.
Therefore, CAC applications are not at the point where large displacement of
the coding work force can occur.
It should be noted that multiple vendors’ marketing materials claim that
their CAC products will result in reduction in FTEs, and certainly increased
productivity always carries this potential. However, there are no empirical
studies from which to estimate overall staff reduction versus shift in
responsibility or simply relief from the coder shortage. This is not
CAC applications do not address specific reporting requirements and have
been only minimally deployed for reimbursement use cases in the inpatient
setting where the real potential for staffing reduction exists.
Let’s now look at the status of deployment of CAC technology.
Widespread adoption of these technologies has not yet occurred. There are
only a minimal number of CAC applications that address inpatient coding for
CAC applications are most commonly found in outpatient settings such as
physician practice and hospital outpatient ancillary departments or emergency
departments. Structured input CAC is deployed in procedurally driven domains
where documentation is predictable and repetitive. NLP-based CAC is deployed in
specific specialties where the vocabulary is more limited and source
documentation is both limited and available in electronic text format — for
example, in radiology, cardiology and emergency medicine.
CAC applications are bridge technologies that serve the pressing need to
improve today’s manual coding process. Mapping from a clinical terminology to a
classification system is ideal for secondary uses of data. Therefore, before we
discuss the catalysts and barriers affecting deployment of the CAC
applications, we must
describe the coding process when reference terminology embedded in an EHR is
mapped to classification systems. And Mary is going to describe that process.
AGENDA ITEM: Presentation — MARY H. STANFILL
MS. STANFILL: Thanks, Val. Good afternoon. I’m going to compare and contrast
the process of mapping from a reference terminology embedded in an EHR to a
classification system, with the work process utilizing a CAC application that
Val just described.
Together, terminologies such as SNOMED CT, and classification systems like
ICD, provide common medical language necessary for interoperability and the
effective sharing of clinical data in an EHR environment. The benefits of using
a reference terminology in an EHR increase exponentially if the reference
terminology is linked to modern, standard classification systems for the
purpose of generating health information necessary for secondary uses.
This linkage is accomplished through mapping, and we’ve been discussing that
throughout the day today. Dr. Campbell gave you a very specific definition of
mapping. We refer to mapping as the process of linking content from one
terminology to another or to a classification. It’s consistent with his
Essentially, mapping provides a link between terminologies and
classifications in order to use data collected for one purpose for another
purpose, to retain the value of the data when migrating to newer database
formats and schemas, to avoid entering data multiple times and the associated
increased costs and error practice that may be involved there.
Clearly, clinical data captured at the point of care can be efficiently and
effectively used for administrative and secondary purposes. Driven by the
philosophy of “code once, use many times,” after clinical care is
recorded in an EHR using SNOMED CT, mapping tables can be used to identify the
related codes in ICD.
This process allows data encoded in SNOMED CT to be aggregated into
groupings for data reporting and analysis. Mapping avoids duplicate data
capture while facilitating enhanced health reporting, billing and statistical
Okay, we’ve got that point. We’ve been talking about that all afternoon.
Now, the standard method for mapping begins with the development of
heuristics, or rules of thumb used for problem solving and guidelines that
support the use case or the purpose of the map, respecting the conventions of
the source and the target to preserve the granularity and flexibility of both.
Defined mapping rules must be developed and consistently applied to minimize
incompatibility without compromising clinical integrity. The map must remain
context free, meaning care must be taken not to introduce any assumptions or
In order for diagnosis and procedure codes resulting from a map to be
appropriate for use in meeting reimbursement requirements, algorithms that
consider coding rules and conventions and reporting requirements, such as
adhering to coding guidelines and identifying the principal diagnosis, for
example, they need to be developed and they have to be applied to the mapping
The development of maps will not eliminate administrative coding or the need
for expertise in code selection. Fully automating the process of mapping from a
reference terminology to a classification system is challenging because of the
inherent differences between them.
The mapping process is straightforward when the source terminology and the
target match up. But when more information is needed to express the concept in
the target, a CAC application could be used to bring in that contextual
information to further refine the map output.
We have a couple of slides to illustrate the mapping process from SNOMED CT
to ICD-9-CM for just a few concepts at varying levels of detail. On the left is
a SNOMED CT concept ID for that clinical finding. On the right is the mapped
ICD-9-CM diagnosis code. When there’s a direct match between the concept and
the code, the mapping is very straightforward. So, for example, the first
example on the slide you’ll see the concept of hypertension, without any
further specificity, is fully reflected in one ICD-9-CM code, the 401.9. And
this does happen fairly often.
But the second slide represents an instance where mapping is more complex.
Here, the variance between the systems requires additional information in order
to determine the target code.
The concept “esophageal reflux” cannot be assigned an ICD-9-CM
code without additional information because it’s classified in ICD-9-CM as
either with or without esophagitis. This is the patient level exclusion
information that Dr. Campbell was referring to.
Ulcer of esophagus is another example where contextual information is needed
to complete the map because ICD-9-CM classifies an esophageal ulcer as with or
without bleeding. The default code is “without bleeding,” but you
would not want the automated map to always default to that.
So you set up these rules, and this example is an
IFA rule is defined to allow for someone to obtain that additional
contextual information on a particular case. The map output in this instance
would be 530.21 if bleeding, or 530.20 if no bleeding. Somehow, that map output
needs to be — somebody has to finish that thought, right? You need to select
the correct code then that applies to the case. Now, that could be determined
by human review or perhaps a CAC application.
In an EHR with automated mapping from reference terminology to
administrative code sets, the coding professional’s knowledge will expand to
include expertise in clinical terminologies, medical vocabularies, as well as
classification systems. Rather than focusing on code assignment, coding
professionals will focus on management and use of the data. Their role will
include many of the functions Val described with use of the CAC applications,
such as documentation specialist and revenue cycle specialist, but their role
will also include functions such as:
Creation, maintenance and implementation of terminology, validation files
Ongoing review of the auto and manual encoder systems for terminology and
classification systems for improving and optimizing the encoding process.
Also include functions like assisting in the analysis of the enterprise’s
classification and grouping system assignment trends and use of data from
classification into these systems.
Proactively monitoring developments in the field of clinical terminology to
And recommending the most appropriate classification or terminology system
to meet all the required information needs.
Val stated that current CAC applications are just an interim step during the
transition to fully implemented EHR systems. Mapping is the ideal goal for a
couple of reasons. While use of CAC applications can increase productivity and
create a more efficient coding process, the use of standard terminology has the
potential to further increase the accuracy of automated coding and thus further
In addition, it’s possible to more fully automate the coding process in an
EHR with embedded clinical reference terminology mapped to a classification
code set than is possible with the use of a CAC application.
Structured input systems essentially employ manual coding, albeit coded once
at the time the structure is set up, but that set-up is time consuming and it
does not lend itself readily to all the clinical nuances.
NLP-based CAC has improved dramatically over the last several years,
especially the last four years or so, but more research is needed on the
accuracy of these systems and expansion into clinical domains with broader
clinical vocabularies has been difficult to achieve.
The coding process when mapping from a reference terminology in an EHR is
entirely different than the process of using a CAC application, particularly in
terms of the computing process that actually generates the suggested codes.
Human review is still necessary before reporting a code resulting from a map in
order to insure accuracy with regard to the context of a specific patient
encounter and compliance with applicable coding guidelines and reimbursement
As rules-based maps are developed for multiple-use cases and become
increasingly sophisticated, the level of human review at the individual code
level will diminish. Workplace roles will focus on the development and
maintenance, including quality control, of maps for a variety of these cases,
and the development of algorithmic translation and concept representation.
Reduced staffing is expected.
While we focused on workflow and staffing, we also want to address the
impact on data quality.
If clinical data, captured in an EHR at the point of care, is to be useful
for whatever secondary purposes we may find appropriate, data quality is
A common concern with data quality is manipulation of documentation to
affect billing codes. Boundaries between clinical data capture and
reimbursement are necessary to insure data integrity. A clinical terminology
intended to support clinical care processes should not be manipulated to meet
reimbursement and other external reporting requirements, as such manipulation
would have an adverse effect on patient care, the development and use of
decision support tools, and the practice of evidence-based medicine. The use of
a reference terminology embedded in an EHR separates clinical data capture
management from reimbursement. This is expected to improve data quality and
result in more accurate reimbursement as well.
Today, an EHR with mapping from reference terminology to a classification
system is rare, and I think we’ve established more information is needed on
just how often that’s done.
Vivian addressed the availability of the SNOMED CT to ICD-9-CM map.
According to SNOMED International, the purpose of this cross-mapping is to
support the process of deriving an ICD-9-CM from patient care.
The map provides users with an approximation of the closest ICD-9-CM code or
codes. Since SNOMED CT’s scope of content is much broader than ICD-9-CM, less
than 30 percent of the content of SNOMED CT can be mapped to ICD-9-CM.
Lack of widespread adoption of EHRs is a barrier to adoption of these
technologies. Without an EHR, the complexity, quality and format of health
record documentation makes it very difficult to integrate CAC applications in
the coding process.
Among other barriers is the complexity of Federal and state regulations
impacting administrative clinical data reporting and our national reimbursement
structure, resulting in variable and conflicting reporting requirements.
Today, many administrative coding practices are driven by individual health
plans or payer reimbursement contracts or policies requiring health care
providers to add, modify, omit or re-sequence reported diagnosis and procedure
codes to reflect specific payer coverage policies or regulatory requirements
contrary to code set reporting standards.
Not only does the variability in reporting requirements undermine the
integrity and comparability of health care data, it significantly complicates
the development of map rules and algorithms in CAC applications, hampers the
advances in CAC applications, and increases the extent of human review required
in both CAC and mapping technologies.
Current CAC applications rely on human intervention to apply these rules,
limiting the degree of automation and thus the potential return on investment.
The integrity of coded data and the ability to turn it into functional
information require the use of uniform coding standards, including consistent
application of standard codes, code definitions, and reporting requirements.
In addition, variable code set update schedules increase the cost and
complexity of insuring CAC applications and maps are accurate and up to date.
And I notice that Vivian has made a mention that NLM has noticed that, too,
that that update schedule is going to be critical.
The failure to implement ICD-10-CM and ICD-10-PCS is also a barrier. It is
extremely difficult to develop valid maps from current clinical reference
terminology to an obsolete administrative code set. It makes no sense to map a
robust terminology such as SNOMED CT to an outdated classification system such
The anticipated benefits of an EHR cannot be achieved if the reference
terminology employed in the EHR, such as SNOMED CT, is aggregated into a
30-year-old classification system such as ICD-9-CM for administrative use and
When an up-to-date clinical terminology is mapped to an outdated
classification system, the map is less reliable and meaningful information is
lost. Furthermore, extensive guidelines and instructions have been created to
compensate for the difficulties in using the obsolete ICD-9-CM coding system.
This simply adds complexity in developing map rules or algorithms for CAC
There are information technology barriers to adoption of CAC as well. We’re
all aware of the lack of industry standards that make integration of software
In addition, this technology itself has limitations. As Val noted,
structured input is best suited to procedurally driven domains and NLP is
limited to electronic text-based documents.
Performance of CAC applications in terms of quality is unknown. Available
research evaluating existing CAC applications is insufficient. We need research
designed to assess the usefulness of these applications in the administrative
In relation to mapping, heuristics are extremely difficult to define, given
the various reimbursement rules, plus the inherent differences between
reference terminology and classification systems.
Mapping between SNOMED CT and ICD is an imperfect science. It’s very
difficult to adequately represent some of the ICD coding conventions for a
The codes produced by the cross-map must be evaluated in the context of the
complete medical record as well as applicable reporting rules and reimbursement
requirements before being submitted to payers and other external entities.
Reliance on the technology alone carries the potential for increased errors in
the coding process and associated compliance concerns.
Concerns of those involved in the coding process, the potential users of CAC
and mapping technologies, can also be a barrier. These technologies involve
significant change. User resistance to change is a very real factor.
Physicians often resist structured input. Coding professionals resist
complete re-engineering of the coding workflow.
Other concerns are more concrete, such as the cost of CAC hardware and
software, and the pressure to meet health care compliance requirements. The
health care industry today is very sensitive to issues that may result in
allegations of fraud or abuse. If not carefully designed and used with caution,
documentation generated via structured templates may justify more reimbursement
than deserved for the services rendered. Physicians and coding professionals
express concern as to whether the OIG will embrace, or even allow, structured
This lengthy discussion of barriers may sound daunting, but it is really
not surprising when you consider that CAC is a disruptive technology. So what
are the drivers and trends that will produce the natural rate of diffusion?
There are many factors within the health care industry driving this
technology, including the movement to adopt EHRs and create a National Health
Information Network. The continued trend of increases in administrative costs
within the health care is also a factor.
The manual coding process widely employed today is expensive and
inefficient, and there is a recognized need to improve that coding process.
Also, the shortage of qualified coders and increased outsourcing and remote
work sites encourages use of CAC for productivity and consistency gains.
Deployment of EHRs with data input codified in a clinical reference
technology is a catalyst that will cause innovative computer-assisted coding to
become a necessity. Other catalysts that will enable this technology include:
Simplification of reimbursement regulations, so that algorithms can be
designed and more readily deployed and maintained.
Adoption of ICD-10-CM and ICD-10-PCS to facilitate the development of
automated maps between clinical terminologies and classifications systems.
And validation and availability of reimbursement use case maps from
reference terminology to administrative code sets.
The NLM, through the UMLS, continues to play a large role in emerging
national standards for the electronic health record, as we’ve seen. Resources
have been committed and work is underway to validate the reimbursement use case
map from SNOMED CT to ICD-9-CM. Val?
AGENDA ITEM: Presentation — MS. WATZLAF
MS. WATZLAF: Thanks, Mary.
As we have discussed, CAC applications presently are not widely deployed,
and an EHR with mapping from a reference terminology to a classification system
is rare. And we believe the Subcommittee can speed adoption of CAC if you
support the following:
Continued efforts to encourage widespread adoption of EHRs.
Efforts to simplify and standardize the reimbursement framework.
And expeditious adoption of ICD-10-CM and ICD-10- PCS.
Our written testimony also includes three additional areas of research that
you should recommend to the Secretary and some other tangential, long-term
research questions to consider in the future.
As a profession, we are pleased the NCVHS is taking a concerted interest in
issues related to CAC and mapping. We look forward with looking with you in
this venture to help our industry move forward with its understanding and use
of these significant tools.
Thank you again for the opportunity to contribute to your discussion here
today, and Mary and I are ready to answer any questions the Subcommittee may
have now or in the future. Thank you.
MR. REYNOLDS: Thank you very much, all of you, for a very thorough review of
the subject. I’d like to open it to the Subcommittee to ask any of the three
panelists a question. Stan, why don’t you go first.
DR. HUFF: I think all of you mentioned the potential for impact on
productivity of the coding staff. Has there been quanitation, I mean, in your
experience, Jim, with the coding you do? Has that changed staffing or other
things within your health information management group, and what’s been the
experience at other sites that you guys from — Mary — have been —
MR. CAMPBELL: I’m really not in a good position to respond to that, Stan. I
can tell you that our studies indicate our quality of information goes up, you
know, with some of the technology I showed you.
We have such a distributed staff at the Med Center that I don’t have
statistics for that.
DR. HUFF: You ladies?
MS. STANFILL: Stan, is your question in relation to computer-assisted coding
type technologies as opposed to mapping?
DR. HUFF: Yes.
MS. STANFILL: Of course, we don’t have a lot of — we’re speculating,
largely, in terms of mapping.
DR. HUFF: Yes. I guess if we don’t have numbers, I guess I’m asking for your
best idea of what the potential for this is. I mean, if we do computer-assisted
coding and say we achieved we could, you know, ultimately ten years or I don’t
know how long down the road, we achieved 80 percent of all of the billing
assignment was done via computer-assisted coding, would that reduce the overall
cost of doing that assignment by five percent, ten percent, 20 percent, 75
percent? You know, what’s the relative efficiency of this compared to manual
MS. STANFILL: Okay. You want to take a stab at that, Val? There’s two
answers to that, two ways to look at that.
There is some published research that’s been published in the
Journal of the American Medical Informatics Association that
looks at the productivity of an NLP computer-assisted coding application, for
example, an NLP coding engine, and the productivity potential is incredible.
They had someone do some manual coding of chest X-rays and then ran it
through a CAC code, an NLP coding engine, and the manual person took two
minutes per report to code it and the NLP coding engine needed .3 seconds to
code it. I mean, the potential, productivity potential, is incredible.
But we’ll tell you what we’re seeing actually in real work, real processing
of actually using a system because of that edit process and some of that sort
of thing. We’re hearing anecdotal reports of around like 30 percent
productivity savings, those kinds of things. That’s purely anecdotal.
When we talk to the users that actually have these systems in place, they
have a very difficult time quantifying that because a structured input type of
a software application changes their whole documentation process, et cetera. So
when we say to them, what kind of efficiencies did you gain in the coding
process, they can’t really answer that question because it changes so much of
And a lot of the burden — burden might be a more strong word than I need —
but it’s shifted a lot of that to the physician, who’s now doing documentation
a different way, et cetera, so there’s a shift in the process in terms of who
does what. So it’s very difficult to quantify.
But in terms of the Natural Language Processing type of systems that are in
use where it’s purely just suggested codes and the coder’s actually looking at
that and then maybe faster, how much is a coder, we’re hearing that — that’s
where we get that anecdotal report of about 30 percent of their time.
But again, a lot of them aren’t necessarily even able to measure it.
MR. REYNOLDS: Jeff has a question.
DR. HUFF: I wasn’t finished with mine yet.
MR. REYNOLDS: Oh, I’m sorry.
MS. STANFILL: I should probably qualify that, though, Dr. Huff, because
again, when we’re talking about — the NLP applications that are available are
only for very specific subsets, so that’s not like 30 percent saving in any
kind of coding. That’s only very specific cases, outpatient ancillary, just
X-rays, or just their emergency department cases, that kind of thing.
DR. HUFF: Okay. Now, I’m trying to understand if there’s a difference
between what Chris Chute would have thought about as aggregation logics versus
the kind of, quote/unquote, “mapping” we’re doing, and there’s a
little bit of difference in the definition between mapping the way Jim used it,
I think, and the way you folks use it, not much but some.
And I guess the question is — it probably comes down to how you’re
representing the actual computable form of these mappings. If there’s
additional information needed, so, you know — in the example, if you need to
know that the patient was male or female in order to assign the proper code or
if you need to know their age in order to — are we using the standard coded
representation for those observable things like age or sex or other things? Is
that a coded part of a rule per se that you’re creating or you have the mapping
sort of between and then that’s left as some additional information? How is
that being done, or how are people thinking about that?
MS. STANFILL: You’re addressing that to Dr. Campbell in terms of how is the
map being designed?
DR. HUFF: Yes. Looks like I need to ask the question a different way. What
is being done in terms of, quote/unquote, “mapping” end up with
something that’s entirely computable, or in fact are we just setting flags for
people to ask a person the question?
I mean, age I can generally determine from the electronic health record. So
when you do a mapping that needs age, is age referenced in a structured coded
way in the rule that you’re creating or how are you doing that?
MR. CAMPBELL: I slipped past that slide pretty quickly, Stan, but part of
the point that I was making is that if we do want to create interoperable
resources in a real sense of the term, then we have to, in our rules
construction, manage the interface to the information model as well as the
vocabulary as well as the expression language.
And so in the example I gave, which is actually under active discussion at
HL7 now, you know, we were using GELLO as the expression language and we were
using a specific feature of the HL7 RIM to construct. In those cases, there
were observations and procedure history that were needed in order to resolve
Those standard architectures, I think, are needed as a part of anything
that we build. And obviously of how we achieve that I think is happening right
DR. HUFF: I guess part of the question is — and that sounds like something
that’s under discussion and will happen in the future — the
“mapping,” quote/unquote, that’s going on now between SNOMED CT and
ICD-9 is something less than that, or is that in fact exactly what you’ve got
MR. CAMPBELL: No, I mean these are concurrent activities. We have a number
of deliverables in order to create an interoperable map. Obviously, content and
review with our coding colleagues is a significant part of it.
But on the other side of it, we also have the necessary knowledge
constructs and so forth, and it’s a question of pulling all consensus together
on all of those elements to deliver the product, not that, you know, one has to
happen before the other.
DR. HUFF: Great. Thanks.
MR. BLAIR: Since the testimony this afternoon was so simplistic and there
were so few variables, I just thought I’d add a variable in, just for some
I attended an ambulatory EMR road show that the Medical Records Institute
puts on, and it’s taken to different cities around the country, and I attended
a few of them. And I learned a few things that I didn’t know was going on in
One of the things, to my surprise, was that there’s a number of vendors, to
me surprisingly large number of vendors, that are very proud of the fact that
they are offering Medcin as a front end, although I can’t say it’s a front end.
They’re just simply offering Medcin for capturing clinical information.
And, Jim, I know you’re familiar with Medcin. I don’t know if our other
testifiers are. But earlier, Vivian Auld from NLM indicated that they’re
planning — I thought it was further along, but they’re planning on having
mappings between Medcin and SNOMED.
Do you have an opinion whether a market acceptance of Medcin would enable,
encourage and facilitate adoption of SNOMED or whether it would be an
MR. CAMPBELL: Well, in my recollection of this morning’s discussion, there were
a number of comments about the needs for this or that code set. I think these
individuals are recognizing that within the larger construction of a reference
terminology such as we’re talking about as our core, there’s vastly more
information than we need in any specific instance, let’s say, of a patient
encounter in a clinical record. And so we need to focus ourselves.
For example, right now CAP is also developing subsets which are basically
vendor implementation tools, for examples, for problem list or clinical
From my standpoint, Medcin has features of an interface terminology along
with some elements of a knowledge base for primary clinical care that serve
value. Ultimately, reference terminology is at the core of all of our system,
becomes the common vehicle, becomes a necessary common vehicle to the
distribution of knowledge-enabled systems.
And I think that Medcin, for example, is one more attractive aspect of how
to put that together in the same sense that the problem list vocabulary at the
University of Nebraska, which is in use in a dozen other institutions, you
know, is a convenience. Those sites have not had the time, wherewithal or
necessarily the training to go ahead and construct that, and delivering that
vocabulary resource to them enabled them to get their implementation going.
Medcin, which is targeted or linked directly into a core reference
terminology in a clinical system I think becomes a similar vehicle, or can be.
Medcin is not going to grow up to be SNOMED CT and RxNorm because that’s too
big a job, but if it provides a useful and interoperable window for a set of
clinicians to employ their clinical system, then it’s obviously a win-win
There’s a lot of details in the answer, you know, and how that answer would
play out, but I think I would say generally it should be a benefit, as long as
we understand the relative roles that are being played by these schemes, you
know, in the design of a clinical information system.
MR. BLAIR: Any other comments or observations?
Let me ask — I’m going to do groping around here just for my
understanding, and Jim or Stan, correct me if my understanding’s not wrong —
that’s why I’m asking about it.
When I think of Medcin, I think of a pre-coordinated terminology which is
easier to use right now than SNOMED, and I think that’s the reason that it’s
being adopted in the marketplace. And I think of it as something that’s going
to be easier to map from Medcin to SNOMED than some of the difficulties that
you’ve articulated mapping SNOMED to something that has greater limitations,
like ICD-9. Are those assumptions that I’ve just made, do you think that
MR. CAMPBELL: Well, let’s go back to the issues I raised of granularity,
editorial focus, for example.
Medcin has been designed as a clinical system, as an interface or into a
clinical system. I mean, that’s the way it grew up.
Arguably, SNOMED CT has also grown the same way, although initially, you
know, it obviously had a slightly different focus. But the point is it’s
Medcin has never been designed with all the features of a reference
MR. BLAIR: Right.
MR. CAMPBELL: So, again, when you go into your
database and take a look at what’s there and will it support other features
like sharing decision support logic with other clinical systems, you’re going
to find that Medcin is going to have limitations.
MR. BLAIR: Right.
MR. CAMPBELL: That’s why I referenced it primarily as an interface
terminology. You’re right — it is pre-coordinated, because I think in the vast
majority of clinical implementations, most clinicians are not going to cut and
paste SNOMED codes together to define a concept. That is not the way the
interface will be designed.
And nor should it be. It’ll be much easier for them to use — and I just go
back home to show you exactly how we’ve done it for that particular
Medcin, I think, approaches a similar problem. And to the extent that those
Medcin concepts are clearly defined and easily constructed into a
post-coordinated SNOMED, I don’t see any reason that there’s a competition
MR. BLAIR: Well, let me refine my question one last piece, because I think
we’ve sort of cut down different layers.
A lot of the testimony that I’ve heard has enlightened me how difficult it
is to create mappings, especially with CAC computer-assisted coding, from
SNOMED to ICD-9. And there’s also challenges with mapping it to other — to
LOINC and, you know, resolving the overlaps with LOINC and RxNorm.
I have the perception that mapping Medcin to SNOMED is not going to create
similar types of mapping and translation problems, although it may offer new
ones, that overall it’s likely to be an enabler and a facilitator for us to
move forward, rather than an added complexity. Is that assumption correct, or
is it too early to tell?
MR. CAMPBELL: I think it is probably correct. If you take a look, for
example, at the use case of the nursing classifications and what went into
harmonizing them within SNOMED, there was a much greater discrepancy obviously
between the nursing viewpoint of the clinical world and the medical viewpoint
and there were new features that were added to the model in order to clearly
develop and express that, whereas Medcin historically, you know, follows much
at primary care, clinical, medical role, so I would expect their editorial
alignments to be much better.
DR. HUFF: I think the other evidence that you can bring to that is that in
a sense the re-codes were created, and were living in a similar environment to
the Medcin codes, they were codes that were primarily used by clinicians to
document what they were doing for the patient, and they had a much more
pre-coordinated form to them and that got represented in clinical terms Version
3 as what were allowed qualifiers. And essentially that’s a step towards
creating the formal model for how you would de-compose a Medcin term into a
primary SNOMED code with allowed qualifiers for that particular item.
And so I think the problem is much more bounded because, you know, the kind
of thing that you would find in medicine would say something like “the
patient has a cough productive of purulent sputum,” and to post-coordinate
that implies that, you know, you’ve got a primary finding of cough or sputum
production and the assumption that you can have some description of what the
sputum looks like kind of thing.
And so it’s a matter of creating information models, or if you think of it,
of creating a structure that tells you the allowed qualifiers that you can have
with cough in creating that.
And I guess my experience in looking at it is in fact that there’s a lot of
work there but it’s not theoretically a difficult problem. I think there are
border cases where it is true, because you can get post-coordinated statements
that imply a very sophisticated model, but they’re used in a very small number
of cases, and so you get to this sort of point of diminishing returns about
whether it’s really useful to make a model that that’s sophisticated in order
to represent the total semantics of two codes or something. So I think in the
border cases you do, but in the whole, I think the mapping is possible and very
doable and very valuable.
MR. REYNOLDS: Okay. Michael, you had a question?
DR. FITZMAURICE: I just love to sit here and listen to Jeff ask questions,
and Jim Campbell and Valerie and Mary answer them. I’m learning much more than
when I ask questions. But nevertheless —
DR. FITZMAURICE: — I’m, still going to ask a question.
As I listen to this, it strikes me that so much of this is intuition by very
learned people. Does there exist a research to build a body of knowledge about
auto-coding regarding things like data quality, total labor productivity? I say
total labor productivity because in some cases you’re shifting the burden to
the physician, moving from an X-dollar-an-hour to a 3X-dollar-an-hour person.
And are there incentives to make a productive system work?
Let’s suppose it is better to shift the burden from a coder to a physician,
and suppose it makes money for the hospital, makes money maybe for a health
plan. Are there ways to pay the physicians more if they do the coding which
makes the system better and gets to this productivity and lower cost versus
paying them less if they want to stick to the way they’re doing it?
So, do we have a body of research regarding data quality and regarding the
labor productivity and regarding the incentives maybe to make such a productive
MS. WATZLAF: I think that’s where we need research in all of those areas
that you had mentioned. I think we recommended some of those, I think, in our
written testimony, but I think much more research needs to be done.
In the study that I was involved in, I know that we did recommend some of
those things, incentives for physicians and so forth. But again, looking at the
quality of some of these systems and the productivity issues, all of that has
not been assessed nearly as well as it should be, I think.
MS. STANFILL: I’m engaged in a literature review right now, that some are
doing that. But I could tell you that the literature that is out there largely
addresses the technical aspect of these systems but very little addresses the
actual processes in terms of quality and actual productivity, you know, with a
different workflow, et cetera, looking at it from practical aspects of
administrative coding as opposed to simply, you know, using NLP in order to
capture cases for research, that sort of thing.
DR. FITZMAURICE: One of the spurs for my question is that AHRQ is the lead
agency for research into a patient’s safety and quality of care, and good data
on patient safety events is hard to come by, the voluntary reporting nature of
it and the fact that you report it so many different kinds of ways.
And so it seems to me that it’s a field that might be fertile with some
prodding by Congress to get people to report voluntarily. If we would have a
structure for them — don’t lose the free text because we don’t know everything
we need to know about classifying patient safety events and how to describe
them — but to take it and to start coding what comes in and classifying what
comes in and see if we can start evolving into something better and better
that’s less burdensome. It’s going to take some kind of a body of research like
And from our discussion today and from looking before, I don’t see that body
of research that says what’s economical as well as what pays off in terms of
eventually feeding back information that reduces patient safety events. Does
that sound reasonable, that we should start investing in some of that?
MS. WATZLAF: Definitely.
DR. FITZMAURICE: Jim, you’re quiet.
MR. REYNOLDS: Marjorie?
MR. CAMPBELL: I’m sorry?
DR. FITZMAURICE: You’re quiet. Is that a nod of the head, that yes.
MR. CAMPBELL: I was going to say the short answer to your question is no.
DR. FITZMAURICE: Doesn’t exist.
MR. CAMPBELL: And I would suggest that one of the biggest challenges to that
is that there is such a discrepancy between health care organizations in terms
of work organization and work plan and that implementation of these systems
revise the work plan so substantially that before and after comparisons are
extremely difficult and comparison between organizations is almost impossible.
So I think it’s important, but I think it’s also very difficult.
DR. FITZMAURICE: That the best we probably can do is case studies. It’s like
evaluating an electronic health record system. You run into the same kinds of
MR. CAMPBELL: At least in the short term, I would like to see reliable
information on validity, accuracy and information like that which has been
woefully lacking in the past. And that, I think, is doable.
MS. STANFILL: And one of the difficulties in measuring accuracy is defining
an accurate — you know, take a case and give it to several people and you’ll
get it coded several ways.
So, I mean, there are some tools that could be developed that would
facilitate even that research. One might be to have a better set of accurately
coded cases that could be used to be the gold standard, measure against, that
sort of thing. Some of those kinds of tools would be helpful.
DR. FITZMAURICE: We’re going to have a gold standard panel.
MS. STANFILL: Yeah!
MS. STANFILL: We’ll just sit tight.
MR. REYNOLDS: Marjorie?
MS. GREENBERG: Where Jeff’s question educated you, mine may totally confuse
you, but —
MR. CAMPBELL: It won’t be hard.
MS. GREENBERG: — at the risk of doing that, and it’s five minutes after
five, I realize, but I’m trying to kind of put all of this together and
understand in particular how these different approaches of going from clinical
data to administrative statistical data, how they all work, and the
similarities. And so let me just sort of get some input from you to help me
I think I understand mapping, generally. I think the mapping from an
existing terminology to a classification, that’s what we’re doing at NLM, we’re
doing a lot of that. You talked about that, Jim, and that was what Mary
described. And although there are some differences, my sense is that pretty
much you’re all talking about the same process.
Then what Stan I know was talking about the last time — well, the first
time you raised this area, and also somewhat today, I think what you are
referring to, Jim, as inferred, the inferred diagnosis, I mean you’re not
starting necessarily with a terminology like SNOMED CT. You’re starting with a
lot of clinical information in a record. It might be the lab values, it might
be X-ray results, it could be a lot of different information.
So you have a lot of clinical findings, essentially. And that’s what you
were referring to as sort of an inferred diagnosis. Now introduced, comes on
the scene here, the computer-assisted coding. And there are two ways to get to
that, apparently. One is through Natural Language, so that doesn’t suggest that
you would have information coded in something like SNOMED CT, I guess, because
it’s Natural Language.
I don’t know whether that might include some of these clinical findings that
would go into an inferred diagnosis, whether CAC includes pulling information
from lab findings or other types of clinical findings. I assume that it does.
MS. WATZLAF: Normally, with NLP, as long as it’s on electronic text that it
can read, it could take any of that information, read it and then —
MS. GREENBERG: So with the Natural Language side of it, you really are kind
of talking about this inferred diagnosis approach? But what you’re saying is it
suggests various codes. It doesn’t go all the way. It suggests them, and then
the editor goes in and looks at it.
And then the structured text, that could include SNOMED text?
MS. WATZLAF: It could. Right. Normally, with the structured text, though,
it’s more of a template, just like a menu that it would just point-and-click?
MS. GREENBERG: More like a pick list?
MS. WATZLAF: Right.
MS. GREENBERG: A sophisticated pick list?
MS. WATZLAF: Right.
MS. GREENBERG: Okay. Which may not, then use — if you already have the
SNOMED terms, then you would probably get into this mapping scenario rather
than the CAC scenario, is that correct?
But then you mentioned that sometimes when you’re doing the mapping and it’s
complicated, you could insert CAC into it. Is that because then you could pull
in other information, like some of these clinical findings, et cetera?
MS. WATZLAF: I think that’s where the whole coding rules and all of that
would come into play. So I think with this slide where it’s a little more
ambiguous as to what the code would be, then the CAC application would be
applied and help with the end result. Is that what —
MS. STANFILL: I think, Marjorie, what might be confusing here is that
structured input is a form of input. But we also talked about a structured
input CAC tool, which is a specific type of application available today that
uses structured input.
So, both structured input or Natural Language Processing might be used in an
EHR if we had an EHR with SNOMED CT embedded in it. That vendor’s EHR might use
structured input as a data capture mechanism, might use Natural Language
But when we were talking about structured input CAC, that’s a specific type
of software application available today that uses a SIG picklet, that sort of
thing, that are pre-coded. And the systems that we’re familiar with today are
pre-coded in IC-9-CM because that’s the predominant thing used today for
In most of the systems, the use case for the applications are billing.
MS. WATZLAF: And we also did see in some of our research that when we asked
them — we did ask, you know, what about SNOMED CT? — they said they were
working on those but at this point they didn’t have that yet because, of
course, ICD-9 and so forth are the ones that they would need for the billing
process, just like Mary said.
MS. GREENBERG: So the CAC applications currently don’t interface with SNOMED
MS. WATZLAF: They could be, but at this point, they said they were working
on that. They could actually apply it, but they haven’t done it.
DR. HUFF: I think it’s probably worth distinguishing those two parts, or two
different kinds of inferences. I mean, you know, if I were implementing, say,
logic, you know, to carry on the hypertension example, you essentially have
very simple logic that says, oh, somebody has placed the SNOMED code for
hypertension on the problem list; I can infer —
MS. GREENBERG: It’s a no-brainer.
DR. HUFF: — this ICD-9-CM code.
MS. GREENBERG: Right.
DR. HUFF: There’s another way. And, I mean, if you were literally
implementing this and you wanted to be accurate in terms of hypertension, then
you’d say, “Oh.” But if they didn’t do that, gosh, I could actually
look at their blood pressures. I could have a simple rule that said, gee, if I
observed blood pressures that were over 140 systolic and over 90 or 85,
whatever the current rule is, for diastolics, and I saw a sustained pattern of
that over months, and I also saw anti-hypertensive drugs were prescribed for
this patient and I saw that they had hypertensive changes noted in the eyes
from the ophthalmologic, then I could assert, yes, this guy’s got hypertension,
even though nobody put the hypertensive code anywhere in the record.
And I guess maybe the reason to make the distinction is the current mappings
are looking at, you know, complicated versions of the first thing where the
assumption is that the diagnosis codes and some combination of diagnosis codes
are there, and the other more basic logic that would allow me to look at the
more primitive things and assert an inferred diagnosis of hypertension, you
know, we’re not doing as a shared collective work anywhere, and there’s
probably benefit in both of those. The first thing is much easier to do early,
the low-hanging fruit. The second thing is harder to do, but ultimately might
prove very, very —
MS. GREENBERG: Well, there are quality issues, too.
DR. HUFF: Yes, because you get the —
MS. GREENBERG: You could look and say, you know, if they’re not taking
hypertensive medications, but I’ve seen this pattern in all these other things,
so what’s going on here? I mean —
DR. HUFF: Right.
MS. GREENBERG: You can see —
DR. HUFF: Yes, that comes back to that one slide with the levels of
abstraction. If you develop the rule that says, you know, these are the basic
findings that allow you to make an assertion of hypertension, you can do both
things. If you see those and hypertension was not asserted, you go,
“Oh,” you know, something’s — we can assert that.
The second thing is also true, because then you can do quality assurance
where you say, “Oh, somebody asserted that but I don’t see any evidence of
that in the record.”
And that comes back to the fraud detection and all of that other stuff. I
mean, if somebody’s asserting things that you don’t see, blood pressure
readings, you don’t see anti-hypertensive medications, you don’t see any of
those other things in the record, you go, “That’s awfully
suspicious,” you know?
So both of those I think have merit, and it’s nice for you to pick those
MR. REYNOLDS: Okay, Jeff, last question on this?
MR. BLAIR: This is a question for Stan, based on our scope of work here, and
if you say the right answer, then —
MR. BLAIR: — I’m going to take advantage of the fact that Jim Campbell is
here. So, is it within our scope? We indicated that we’re looking at secondary
— you know, capturing information once at the point of care and with
clinically specific terminologies and then using derivatives of that for
reimbursement, public health, clinical research.
Question: Is it within our scope of work here that if we use that clinically
specific terminology to improve the specificity of outcomes data to improve
clinical processes and work flow, that that’s within the scope that you’re
DR. HUFF: Yes.
MR. BLAIR: Okay. Got a question for Jim.
MR. REYNOLDS: Try again, Stan.
MR. BLAIR: Jim, you’re one of the few organizations, or you represent one of
the few organizations that I know that has been using SNOMED, and therefore,
have you, in fact, been able to take advantage of SNOMED to improve the
specificity of your outcomes measurement to improve clinical processes or work
MR. CAMPBELL: I’m sorry, Jeff. I heard everything, but what was the
MR. BLAIR: Have we done it? Have you used SNOMED to improve the specificity
of outcomes measurement which then has flowed into improving clinical processes
or work flows?
MR. CAMPBELL: I don’t know that I can answer that question on into the
question of the work flow. I can tell you that we have been, you know,
implementing quality assurance initiatives — for example, on diabetes, for
example, on community-acquired pneumonia — over the past year and a half, two
years, in alignment with a lot of national priorities and that the information
in our database, in our clinical records system, needed SNOMED in order to be
successful in terms of getting sufficient clinical information to support those
So in everything that we have done along those lines, we’ve been, I think,
very happy with what we’ve gotten out of SNOMED in that respect.
MR. CAMPBELL: Now, I can’t give you then, you know, how many patients were
not hospitalized or how many did not come back to the emergency room.
MR. BLAIR: Okay. But you have gone through this process, and I’m wondering
if you have any guidance or recommendations to the NCVHS then for things that
we should consider that should enable, facilitate, make it easier for you to do
that, because improving clinical processes and work flow is something that’s a
very important thing we want to do and you’re one of the few that have gone
through that process.
Were there impediments that you encountered —
MR. CAMPBELL: Well, there’s other vendors that I could point you to who are
in the midst of, I think, the same process. But in reflection of an earlier
comment, I would say that in 2004 when the National Library of Medicine signed
the contract with CAP, on that next release, you know, our problem lexicon went
out to all of our clients with SNOMED for the first time, because we didn’t
have to worry about contract costs at all those individual sites then in
So at least that’s the first testimonial, even if you haven’t had any
others. And so I think there was no question that cost was a big barrier, and
suspicion, and worries about ulterior motives and long-term contract issues.
So I think that has been very important in terms of pushing the whole
discussion forward, more towards the issue of clinical relevance and outcomes
which is, I think, where it needs to be. Right now, I think one of the biggest
problems is education, understanding and guidelines for how to move forward,
because the whole thing is complicated and it’s not clear just how exactly you
So some of the things we’re looking for in the SNOMED community are
basically implementation exemplars as pathways, if you will, for other vendors
to help give them some assurance of, you know, what the risks and benefits are
MR. BLAIR: Thank you, Jim.
MR. REYNOLDS: Okay. Again, thank you very much for the panel. You obviously
spent an incredible amount of time getting ready and we appreciate you helping
us slug through this thing, which is still going to be quite a bit of a
Stan, I’d like to turn the last few minutes over to you to — we’ll try to
adjourn about 5:30 — go through a summary of today and in case anybody starts
getting out of here before that, thank you. What an incredible amount of
information, what a great panel. We already thanked everybody in e-prescribing
but you personally were really grabbing hold of this and taking it — thank you
so much because most people don’t know this much after a year, let alone after
one-half a day. So, thank you.
DR. HUFF: Well, I want to thank the people who were willing to come and
present because I really do appreciate everyone, you know, Valerie and Mary and
Dr. Campbell and Clem McDonald and obviously Vivian’s testimony, too, about
what’s happening. All very pertinent.
I don’t know at this point that I want to make a summary. I’m excited about
this subject; I guess I would say that. And I was pleased by how much I learned
from the people who testified and wanted to thank them.
Other than that, you know, I’m ready to quit, so —
MR. BLAIR: You mean for the day?
DR. HUFF: For the day, sure. For the day.
MR. REYNOLDS: Seeing nobody jump up to make Stan keep doing, we’ll adjourn
Our plan tomorrow is to talk about future meetings. We can obviously
continue a bit of debrief here and then we’ve got to look at the rest, the
other things that are on our plate plus, you know, meld in the things we heard
today to see what we do next.
So, starting at 8:30 in the morning. Thanks, everyone.
(Whereupon, the subcommittee adjourned at 5:21 P.M., to reconvene the next