More on Quality Reports: Lessons From SAT Scores

By  |  December 14, 2007 | 

My older son is gearing up to apply to college (:-\

and so I bought him one of the Bibles, the Fiske Guide. The book is cleverly written – enough academic factoids to get parents to spring for it, leavened with enough social scene skinny to get kids to read it. The Guide dutifully lists ranges of SAT scores for accepted applicants at 300 schools, but then adds this shocking caveat:

their zeal to make themselves look good in a competitive market, some
colleges and universities have been known to be less than honest in the
numbers they release. They inflate their scores by not counting certain
categories of students at the low end of the scale, such as athletes,
certain types of transfer students, or students admitted under
affirmative action programs. Some colleges have gone to such extremes
as reporting the relatively high math scores of foreign students, but
not their relatively low verbal scores…(p. xix)

Since writing a piece in JAMA a couple of months ago on the need for standards in quality reporting (described in this blog posting), we’ve gotten a bit of blowback from folks who worry that healthcare is too complicated for such standards, or that this kind of overregulation is making the practice of medicine numbingly robotic. The meta-message is that we can police ourselves perfectly well, thank you.

But I think Fiske’s SAT story is instructive: once hospitals are really competing on quality and safety, gamesmanship is inevitable (“Oh, yes – we exclude patients over age 80 in calculating our mortality figures, and widows and widowers in our readmission rates. It’s all there in the fine print – didn’t you see it?”), and we’ll need standards to create (and enforce) a level and credible playing field. I don’t want to minimize the challenges, but the SAT shenanigans are a window into what happens when the standard-setting can is kicked too far down the road.

One Comment

  1. tholt December 14, 2007 at 4:51 am - Reply

    I come from the land of Lake Wobegon where “all the women are strong, the men are good looking and the children are above average”. We simply don’t accept public announcements of our weaknesses. This is particularly true of physicians who like to compete and win. If we get public reporting there will be manipulation of the data so every clinic’s quality is strong, every physician’s patient satisfaction report is good looking and every hospital’s safety record is above average.
    I believe public reporting of good, useful data is very important and worth pursuing. I wonder if it will really work.

Leave A Comment

About the Author:

Robert M. Wachter, MD is Professor and Interim Chairman of the Department of Medicine at the University of California, San Francisco, where he holds the Lynne and Marc Benioff Endowed Chair in Hospital Medicine. He is also Chief of the Division of Hospital Medicine. He has published 250 articles and 6 books in the fields of quality, safety, and health policy. He coined the term hospitalist” in a 1996 New England Journal of Medicine article and is past-president of the Society of Hospital Medicine. He is generally considered the academic leader of the hospitalist movement, the fastest growing specialty in the history of modern medicine. He is also a national leader in the fields of patient safety and healthcare quality. He is editor of AHRQ WebM&M, a case-based patient safety journal on the Web, and AHRQ Patient Safety Network, the leading federal patient safety portal. Together, the sites receive nearly one million unique visits each year. He received one of the 2004 John M. Eisenberg Awards, the nation’s top honor in patient safety and quality. He has been selected as one of the 50 most influential physician-executives in the U.S. by Modern Healthcare magazine for the past eight years, the only academic physician to achieve this distinction; in 2015 he was #1 on the list. He is a former chair of the American Board of Internal Medicine, and has served on the healthcare advisory boards of several companies, including Google. His 2015 book, The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age, was a New York Times science bestseller.


Related Posts

By  | March 7, 2018 |  0
I am angry. Perhaps, you are too. As a physician, it is heart-wrenching to watch people unnecessarily die from gun violence. As a mom, it strikes fear in my heart to know that our nation’s children are not safe in our schools. I vividly remember being a resident on call in the ICU when I […]
By  | January 4, 2018 |  0
As a personal advocate for value-based care, I was lucky enough to do my Hospital Medicine Fellowship at an institution that placed a strong emphasis on improving and optimizing value for its patients. As a (mildly naïve) first-year fellow, I recall being impressed by an atmosphere and culture that appeared to embrace the “less-can-be-more” movement. […]
By  | November 21, 2017 |  0
As a Johns Hopkins undergraduate, I used to run the Welch lecture series in medical history. Through this role, I learned about an interesting tidbit – the origin of the word “rounds.” Johns Hopkins Hospital had a circular ward where the infamous and quotable Dr. Osler made his “rounds” to see patients. While medicine has […]