A Fascinating Audience Vote on Quality and Transparency

By  |  October 7, 2007 | 

At my just-completed annual hospital medicine CME course, we held a fascinating session on the future of quality measurement, transparency, and pay for performance (P4P). The discussants – Andy Auerbach, Peter Lindenauer, and Kaveh Shojania – all emphasized the limits of process measurement, particularly noting the problems of unforeseen consequences, playing for the test, and flawed measures. The panelists see outcome measurement (e.g., mortality, readmissions) making a comeback, something I’m noticing as well.  

But perhaps the most interesting part of the session was a series of audience votes. Using a computerized audience response system, sorta like Regis uses when he polls the audience on Who Wants to Be a Millionaire, I periodically ask the audience to weigh in on key questions.  

So I asked the crowd of about 550 physicians about the impact of transparency on their practice and their institution.

First, I asked what these physicians (four out of five were hospitalists, and thus in the cross-hairs of today’s quality measurement and reporting efforts) think of today’s measures, particularly the Medicare “Core Measures”:

  • 18% thought the measures bore no relationship to their quality of care
  • 63% said that the measures capture a portion of their quality, but perhaps 90% is going unmeasured
  • 18% thought they captured a decent slice of quality
  • 2% thought they accurately captured the quality of their care

Then I asked how much their hospital’s leadership (CEO, Chief Medical Officer, the Board) focuses on the publicly reported measures.

  • 37% said a great deal, as much as they focus on hospital finances (a high bar, indeed!)
  • 47% said a moderate amount, not quite as much as finances but an awful lot
  • 13% said “not much” – people mention it and are embarrassed by bad results, but that’s it
  • 3% said “not at all”

Next, I asked how much their patients were focusing on quality measures; perhaps the C-suite’s focus on the quality measures was coming from patients threatening to vote with their feet:

  • 1% said a great deal (lots of patients choosing hospitals and docs based on results)
  • 6% said that some plugged-in patients were paying attention
  • 43% said not much; patients sometimes mention the results but it rarely influences behavior
  • 50% thought patients were paying absolutely no attention to public reports of quality.

The most damning result came when I posed the following statement: “P4P is really just a CMS (Medicare/Medicaid) scheme to save money, wrapped in the flag of quality.”

  • 79% of the audience agreed with this statement, 21% did not.

Medicare clearly has some work to do if it cares about getting the buy in from docs.

All of this is further proof of the point I made last week: the biggest surprise of the quality revolution is the immense power of transparency, which does not appear to be mediated through consumer-driven pressure. I argued that the pressure generated from public reporting is leading to lots of internal P4P (bonusing based on scorecard results), which may ultimately make payer-generated P4P unnecessary.  

And if you think the reporting is generating action now, just imagine how powerful it will be when patients actually pay attention to the results!

These survey results are also sobering evidence that this group of docs – a group that has been subject to more quality measurement than any other, and, in general, has bought into the quality thing – believes that today’s quality measures don’t really capture much of importance. Moreover, these physicians, by a wide margin, believe that the “quality revolution” may in reality be a big cost-saving scheme by the payers.

The bottom line: everybody’s got a lot of work to do if we’re going to get this thing right.


  1. Joe Miller October 16, 2007 at 9:05 pm - Reply

    Your commentary makes me wonder why patients do not take a more active role in evaluating quality… and more specifically will they ever be informed healthcare consumers?

  2. Bob Wachter October 18, 2007 at 5:50 am - Reply

    Yep, it is pretty extraordinary. Remember Bill Clinton’s experience – he received his CABG from the lowest ranked cardiac surgeon in New York State.

    But perhaps that was then, and this is now? Well, no. I recently asked the members of UCSF’s Epidemiology & Biostat Dept how many ever checked quality measures before making medical decisions for themselves or loved one. Answer: one person (out of about 50). If anybody was going to follow and act on quality data, it would be them!

    When will patients start following these data? When they are much easier to use, when the measures become more meaningful, and when health professionals use the data themselves for decisions and recommendations. Right now, when a family member or friend calls me for a hospital or doctor recommendation, I still mostly base this on reputation, word-of-mouth, and pedigree (ie, medical school, residency). Patients won’t start using quality data until we do it ourselves. Best guess: in 3-5 years.

  3. Brian Clay October 21, 2007 at 1:14 pm - Reply

    With lack of health literacy in the general population being such a prominent problem, it is not surprising that patients are not more proactive in attempting to evaluate the quality or outcomes of their physicians. Surveys in this area repeatedly show that while patients think medical care could be improved, most people feel that their OWN doctor is “one of the good ones.”

    I agree with Bob that a much higher degree of facility of use and the inclusion of hard outcome measures rather than process compliance will move patients in the direction of wanting to know and make use of quality data. I also find it highly unlikely that initiatives such as P4P will result in such a move.

    The ironic thing is that I have the sense from the hospitalist community that we want to generate this data and find the right measures, because we know that we can make the greatest strides toward improving care by accurately measuring the problems we have now. The quality improvement drive in hospital medicine is substantial, and a lot of us are excited to be a part of it. I suspect that there is frustration that our quality improvement work (and what is targeted for improvement) is being at least partially driven by entities such as CMS and The Joint Commission and their regulatory requirements.

    I hope that in 3 to 5 years I see a patient in our emergency room who tells me that they came to our hospital instead of another based on outcomes data that they looked up or read about. Hasn’t happened yet.

    Brian Clay
    Division of Hospital Medicine, UC San Diego

Leave A Comment

About the Author: Bob Wachter

Robert M. Wachter, MD is Professor and Interim Chairman of the Department of Medicine at the University of California, San Francisco, where he holds the Lynne and Marc Benioff Endowed Chair in Hospital Medicine. He is also Chief of the Division of Hospital Medicine. He has published 250 articles and 6 books in the fields of quality, safety, and health policy. He coined the term hospitalist” in a 1996 New England Journal of Medicine article and is past-president of the Society of Hospital Medicine. He is generally considered the academic leader of the hospitalist movement, the fastest growing specialty in the history of modern medicine. He is also a national leader in the fields of patient safety and healthcare quality. He is editor of AHRQ WebM&M, a case-based patient safety journal on the Web, and AHRQ Patient Safety Network, the leading federal patient safety portal. Together, the sites receive nearly one million unique visits each year. He received one of the 2004 John M. Eisenberg Awards, the nation’s top honor in patient safety and quality. He has been selected as one of the 50 most influential physician-executives in the U.S. by Modern Healthcare magazine for the past eight years, the only academic physician to achieve this distinction; in 2015 he was #1 on the list. He is a former chair of the American Board of Internal Medicine, and has served on the healthcare advisory boards of several companies, including Google. His 2015 book, The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age, was a New York Times science bestseller.


Related Posts

By  | September 17, 2018 |  0
Last October, I wrote about the process some healthcare organizations are engaging in to develop written compacts between physicians and the hospitals where they practice. The point of my post was that there are inevitably some generally accepted (but rarely articulated) expectations that the two parties have of each other, and it can be valuable to […]
By  | August 8, 2018 |  2
By: Ethan Cumbler, MD, FHM, FACP Sarguni Singh, MD University of Colorado School of Medicine What’s the purpose of attending documentation on a resident’s progress note? The answer might seem obvious, but stop and think about it. Often, the attestation is a detailed exposition of the attending’s own thought process on the diagnosis, differential, and work-up […]
By  | August 1, 2018 |  4
I have a theory. There is a simple thing hospitalists can do that can enhance relationships with our patients, and even, I bet, improve patient satisfaction scores. The catch is it is not something you can do for yourself; you can only “pay it forward” for somebody else. We know patients who trust their physicians […]