NCVHS Quality Workgroup Retreat, June 2-3, 2005

—Summary—

Introduction

The Quality Workgroup of the National Committee on Vital and Health Statistics (NCVHS) met in Washington, D.C. for a two-day planning retreat on June 2-3, 2005. The purpose of the retreat was to lay the foundation for a new charge and work plan for the Workgroup. The retreat participants included Workgroup members; Committee and Workgroup staff; NCVHS Chair Dr. Simon Cohn; and ten leaders in the fields of quality measurement, health policy, health informatics and health care. The guest speakers, in order of appearance, were:

  • Dr. Brent James, Intermountain Health Care
  • Richard Klein, National Center for Health Statistics (NCHS)
  • Ernest Moy, Agency for Health Care Research and Quality (AHRQ)
  • Dr. Joachim Roski, National Committee for Quality Assurance (NCQA)
  • Dr. Trent Haywood, Centers for Medicare and Medicaid Services (CMS)
  • Dr. John Halamka, Harvard Medical School & CareGroup Health Systems
  • Dr. Don Detmer, American Medical Informatics Association (AMIA)
  • Dr. Carolyn Clancy, AHRQ
  • Dr. Jerod Loeb, Joint Comm. on Accreditation of Healthcare Orgs. (JCAHO)
  • Dr. Steven Jencks, Quality Improvement Office, CMS

The speakers’ slides are posted on the NCVHS Website, www.ncvhs.hhs.gov. Workgroup members, other participants and audience members are listed at the end of this document. See also the “Summary of Brainstorming” table, described below.

The retreat provided opportunities to review the current state of performance measurement and quality assessment, highlight exemplars of positive approaches, identify priorities for improving performance measurement, and consider how NCVHS can best contribute to this goal. After several hours of discussion and presentations, the members summarized and prioritized their findings. They reviewed the roles of NCVHS and the Quality Workgroup and, with these in mind, selected a short list of issues and activities on which they believe the Workgroup should focus its efforts. The table “Summary of Brainstorming,” developed by Workgroup member Carol McCall, reflects the synthesis of these initial findings and is intended to assist the Workgroup’s efforts as it revises its charge and develops a new workplan. The Workgroup identified as top priorities advancing the knowledge management uses of the NHII and, more specifically, developing what Dr. Detmer called the intersection between the quality and information technology (IT).

The following sections summarize the major themes of the retreat presentations and discussions:

  • Key concepts underlying the vision
  • Gaps and needs to be addressed
  • Scan of the environment
  • Unique role of NCVHS
  • Next steps

Key concepts in framing the vision

Dr. Brent James of Intermountain Healthcare set the tone for the meeting with the maxim that “Aim defines the system” (W. Edwards Deming), a principle he expressed through the notion of a continuum of information needs, extending from “comparative data for accountability” to “data for learning.” His comments paralleled aspects of the recent IOM report,Patient Safety: Achieving a New Standard of Care. He stressed the importance of and need for decision tools that support the full continuum and that can be used for decision-making at different levels. He emphasized that this can only be truly accomplished by designing systems for the “data for learning” end of the continuum. Dr. James noted that current quality metrics are static and unidimensional, when they need to be dynamic and iterative. He described workable outcomes chains that track the elements of a process/outcomes structure from a final outcome through a series of intermediate outcomes or process steps to the level of actual decisions or behavior. He called for “a shared intellectual tool for the nation,” developed with national leadership, that could be a standardized resource for assessing quality. This led to a discussion of how NCVHS might help in the creation of such a national toolkit.

The six dimensions of quality in the IOM’s Crossing the Quality Chasm and the need for a balanced approach to all these dimensions provided a consistent reference point for the discussion. Several speakers highlighted equity as an especially neglected dimension of quality. The Workgroup was urged to devise an approach to quality measurement that makes sense at a population health level. In this regard, Mr. Moy discussed the links between the AHRQ reports on quality and disparities, and Dr. Jencks noted that CMS now calls itself “a public health agency.” Patient/consumer centricity was another IOM quality dimension that the participants highlighted as a priority, noting that it requires a transition from current provider-centric quality measurement. They agreed that this transition, like the others in the emerging vision, will be a gradual one.

Dr. Detmer discussed ways to lead change and promote self-organization in complex adaptive systems, the latter a concept that informed the Chasm report. He proposed that the three-part schema put forward by the NCVHS Workgroup on the NHII in its report Information for Health—with interlocking information on and for the person, the provider, and the population—provides an information infrastructure that permits learning while dealing with complexity. He observed that the Committee’s role is to be guardians of the vision as well as to monitor gaps in the vision, and he challenged the Workgroup to refine the idea and develop priorities around the intersection between quality and the NHII. In this vein, there was general agreement that health information technology is necessary but not sufficient for bringing about the needed changes. (This theme is taken up below.)

Other themes that informed the retreat discussions were the importance of a system focus and the need for radical transformation. In his concluding remarks, Dr. Jencks proposed a vision for “health statistics of the future and quality of care.” He stated that the vision requires transformation of the entire health care system, not just incremental change, and he urged the Workgroup not to be timid. Several times in the course of the retreat, paradigms were identified that need to be replaced or “broken” to open the way for change. The general approach favored by the group was to identify a 10-year vision for quality and to monitor progress toward it using intermediate objectives.

Appendix: Attendance 
Gaps and needs, pointing to elements of the vision

The need to develop the intersection between quality and IT emerged as an organizing principle for thinking about the Quality Workgroup’s contributions. Dr. Detmer, Dr.  Halamka and others observed that work on the information infrastructure (including the electronic health record) has not yet created the conditions for promoting and assessing quality. Dr. Halamka, whose leading-edge health IT activities in Massachusetts are discussed below, said the community utilities in his system do not include a “quality measurement utility,” a gap he attributed to the lack of guidance on what to measure and the lack of national standards for collecting and transmitting data. He stated, “This is a critical juncture in history. The tracks are being laid for interoperability, but not for quality.”

After posing the provocative question What are we going to do until an American health care system arrives? Dr. Detmer discussed the potential uses of the information infrastructure for knowledge management. This priority became a touchstone for subsequent discussion, linking to Dr. James’  idea of a shared intellectual tool for the nation. The participants focused both on fleshing out this element of the vision and on considering the conditions needed to bring it about. They discussed the idea that such an intellectual tool, in order to be a catalyst for transformation and to sustain a learning paradigm, would demand a kind of ‘flow’.  They envisioned a kind of plug-in module or software “patch” for an EHR, embedded in the flow of care processes, as a way to both receive updates of new metrics and to return information for new discoveries, all part of an adaptive learning system. They agreed that this product will not come about without some form of national leadership, probably by a governmental entity. Both the National Quality Forum and AHRQ, properly funded and empowered, were mentioned as the possible focal point for such an effort, which would need to be broadly collaborative.

Several briefings about existing quality activities on the part of NCHS (for the Healthy People process), NCQA, CMS, AHRQ and JCAHO highlighted specific needs, gaps, and ideas for improvement. Although they noted particular data gaps, Dr. Loeb and his fellow presenters stressed the existence of disparate and proliferating measures and the need for overarching and composite (or clustered) measures.

Dr. Roski urged the Committee to propose a taxonomy of what defines value in the health care system. He also pointed out the challenges of moving from consensus to implementation, especially as more quality assessment begins to take place in the ambulatory domain, and he raised the question of who will bear the cost of performance measurement, which he noted is a public good. In this regard, Dr. Clancy observed that quality systems must stay linked with the bottom line, as in pay for performance initiatives. She also noted the importance of information for research as a byproduct of the EHR.

The group discussed the need for bridges between administrative data and the EHR. They noted the need to accommodate current pay for performance initiatives while also laying the groundwork for quality-oriented knowledge management using the information infrastructure. The Workgroup weighed the trade-offs between different data sources and perspectives (i.e., administrative data vs. the clinical data in EHRs) and agreed that NCVHS should focus on laying the groundwork for the EHR while also being attentive to the short-term need for bridges. Ultimately, as Ms. Greenberg pointed out, what is important is the questions being asked; those providing the answers will have discretion about what information sources they use to do so. She also noted that progress has been made in building capability into administrative records to capture information to meet short-term needs.

Another theme pointing to a possible NCVHS role was the need to educate the public, providers and purchasers about value and the need for data. This includes the need to design quality-related tools for the general public. The issue of the cultural readiness of various stakeholders with respect to information was discussed in this context. Dr. Clancy talked with Workgroup members about the importance of cultivating greater openness to peer review among physicians, and the potential uses of initial and continuing medical education to educate providers.

The environment: stakeholders, key actors, partners, customers

In advance of the retreat, the Quality Workgroup identified the following stakeholders in the quality measurement environment (to which #7 has been added):

  1. Consumers/patients (individuals, organizations)
  2. Providers (physicians, hospitals, pharmaceutical manufacturers, nursing homes, home care agencies)
  3. Purchasers/payers (Medicare, Medicaid, health plans, insurers, employers, coalitions)
  4. Public sector (CMS, CDC, state & local, etc.)
  5. Accreditors/Quality Assurors (JCAHCO, NCQA, NQF, QIOs)
  6. Research (AHRQ, NIH, CMS, NCHS, FDA, IOM, academe)
  7. IT industry and associations (AMIA, AHIMA, etc.)

In the course of the retreat, they identified the following questions about the stakeholders, for further investigation:

  1. What are their activities?
  2. What are their needs?
  3. What desired trends and outcomes are not self-organizing?
  4. Who should set the priorities?
  5. What activities can be leveraged?
  6. Who will bear the cost?

They learned from two presenters about significant potential models. The first, presented by Dr. Halamka, is a cluster of community solutions in Massachusetts that are now grouped into a Regional Health Information Organization (RHIO) which he directs. He described the many entities that serve RHIO functions, and in this context commented on the lack of a quality roadmap, even for such an advanced infrastructure. As the other model, Dr. Detmer described the Northern New England Cardiovascular (CV) Study Group as an exemplary change management module that accommodates learning in the complexity zone. This became a reference point and gold standard in subsequent discussions of the idea of a shared intellectual tool. These two models led to the question of how to nurture the conditions for their replication in other places and for other conditions. The group discussed the possibility of visiting the Massachusetts RHIO and talking with a leader of the N. New England CV Study Group.

The unique role of NCVHS

Because this retreat was directed toward formation of a workplan, the Workgroup kept an eye on the question of appropriate roles for the Committee, and they listed them before beginning their selection and prioritization process. The NCVHS roles they identified were:

  1. Define and be guardian of the vision
  2. Provide a roadmap
  3. Monitor gaps
  4. Educate
  5. Manage expectations
  6. Be a catalyst for change
  7. Gather information
  8. Convene
  9. Identify consensus and gaps
  10. Advise [especially the Secretary, in terms of actionability]
  11. Identify research needs

Many invited speakers, including Dr. James, Dr. Detmer and Dr. Jencks, commented on the unique positioning of the National Committee to play key roles in catalyzing change in the areas outlined above. Responding to the invitation to suggest how the Committee can best contribute, Dr. James suggested that the Workgroup help in the development of a shared intellectual tool. He noted that the mission of NCVHS is broader than what most other groups have undertaken. Dr. Halamka said the Committee’s leadership is needed to facilitate efforts to define quality. Dr. Detmer proposed that in its role as “guardian of the vision,” the Committee hold hearings on the quality-IT interface. Dr. Loeb suggested that the Committee recommend creating a singular mechanism whereby all stakeholders can sit around a table and build a single, cost-effective toolkit. Dr. Jencks encouraged the Workgroup to look at overall system improvement and the new horizon rather than fixing gaps; to focus on the systems-quality interface, patient-centering, and EHR systems; and to develop the vision in a way that is complementary to but different from those of other stakeholders.

Ms. McCall focused the thinking about actions by asking, How can we screw things up? In response, participants noted the pitfalls of focusing on the accountability end of the continuum; designing an inflexible, static system; failing to define and/or align the Workgroup’s role properly; and failing to gain consensus and buy-in.

Actions and next steps: Toward a workplan for the Quality Workgroup

Using a “nominal group process,” the Workgroup members reviewed the points they had summarized in a brainstorming session, and they voted on their highest priorities for the Workgroup’s workplan. (See the table in the attachment for the complete list of items, clustered by topic.) Facilitator Carol McCall stressed that this list is not a workplan, but rather a set of focused elements that represent the foundation for a workplan. The results are as follows:

  1. Quality/IT intersection [8 votes]
  2. (5 votes):
    1. Designing for the continuum from accountability to learning
    2. NCVHS as keeper of the vision as well as monitor of gaps.
    3. Knowledge management as distinct from the information infrastructure.
    4. Patient-centeredness and its impact on the metrics.
  3. (4 votes):
    1. What is quality? — “quality” and “value” as alternative approaches.
    2. How do we maintain and reconcile metrics?
    3. Who sets the priorities, and how?
  4. (3 votes):
    1. Use IOM dimensions to create a balanced approach.
    2. The vision should include a minimum need.
    3. Promoting cultural readiness among clinicians, patients, etc. [relates to education role — creating a receptive environment among stakeholders].
    4. Committee has a role in explaining/educating on vision/value/need.

In summary, the following workplan elements emerged during the two-day retreat:

  1. Revise charge, aligning it with those of other NCVHS groups.
  2. Consult stakeholders and experts.
  3. Devise a set of guiding principles.
  4. Conduct field research, including learning from analogous sectors such as education and the airlines.
  5. Formulate a 10-year vision and milestones in collaboration with other stakeholders.
  6. Devise a roadmap.

SEE ALSO: QWG Retreat Summary of Brainstorming.

Attendance

Workgroup members:

  • Robert Hungate, Chair
  • Justine Carr, M.D.
  • Carol McCall
  • William Scanlon, Ph.D.
  • Donald Steinwachs, Ph.D.

Other NCVHS members:

  • Simon Cohn, M.D., Chair

Staff:

  • Anna Poker, AHRQ
  • Stan Edinger, AHRQ
  • Gail Janes, CDC
  • Trent Haywood, CMS
  • Debbie Jackson, NCHS

Others (not including presenters):

  • Christina Pavel, AHRQ
  • Dan Rode, AHIMA
  • Paul Youket, CMS
  • Susan Baird Kanaan