Physician Assessment Gone Bad

>
By  |  September 28, 2015 | 

 

Some in our profession have begun to think hard about our future pay and incentives, especially given the vagueness of the recently passed MACRA legislation.

MACRA replaced the SGR and put in place a model of compensation that in theory balances both physician accountability and just rewards for hard work.  The two MACRA tracks, the Merit-Based Incentive Payment System (MIPS) or the Alternative Payment Model (APM) are long on promise, however, but short on substance.  Read here for more (a brief and outstanding NEJM commentary).

So much needs working out, and first among them requires us to deliver an appropriate means by which to assess physician achievement.  We have no functional framework to work off of and no history as a guide. How do you evaluate a provider when few valid instruments to measure performance exist? How do you reward a doctor for value when your metrics do not deliver?

With the above in mind, I jumped on a scorecard recently released by ProPublica.  The journalists took on the arduous task of appraising 17,000 U.S. surgeons and ranking them by their adjusted surgical complication rates.  The project went very broad; the investigators also assembled a knowledgeable collection of consulting providers to assist, and they made their methods known. The publication raised quite a stir, and as you would expect, folks had a lot to say–and some of it not too nice. Docs pushed back.

 

NBC

 

Patient advocacy groups and spokespeople, of course, responded with a unified message: “You have had a long time to get your house in order.  You did not and here is your report card. Now go fix.” Even with the criticism and alleged data flaws, notable physicians also defended the effort:

Because the choice wasn’t between building the perfect report card and building the one they did. The choice was between building their imperfect report card and leaving folks like Bobby with nothing. In that light, the report card looks pretty good. Maybe not for experts, but for Bobby.

I bring the above for you to consider because parties both inside and outside of medicine have begun to move.  Whatever data we have, suboptimal or not, will get aggregated, analyzed, and coiffed, and like it or not, will be the standard against we measure ourselves and adapt.

But then something excited my attention.  RAND just released the following: A Methodological Critique of the ProPublica Surgeon Scorecard.  Some might call it a critique, but in my book, it’s close to an evisceration:

RANDS

The study authors had problems with data definitions, risk adjustment, and statistical methods.  A flawed, but noble effort, but not one on which we want to base our surgical choices, and certainly not one on which to judge doctors for performance and compensation.

I am glad to see the appraisal, and not out of spite.  I welcome the ProPublica effort (in fact, I contribute to ProPublica). However, for those of us who receive ratings, who know what the measurements mean and how they translate–the RAND publication serves as a valuable reminder. We want it all right now. However, in our haste to implement a system to grade docs and consume data, our eyes might be a bit larger than our stomachs.  For the moment, we need a more generous serving of caveats, buyer bewares and speed bumps–and far fewer top ten lists and red, yellow & green flags.

 

Share This Post

2 Comments

  1. Charles Carter September 29, 2015 at 8:25 pm - Reply

    This is nice to see. Of course, those who are broadly critical of our profession won’t care. The goals are indeed laudable, but I strongly suspected the validity of the results when I found myself listed as performing a number of TKA’s and lap choly’s.
    The whole quality measurement/ performance is somewhat backwards from where I sit, but I’ll spare you the long harangue. I think most of us are open to constructive criticism, but statistics base on who-knows-what data offer little opportunity to improve one’s practice or examine our weaknesses.

  2. Mighty Casey September 30, 2015 at 10:20 am - Reply

    I agree 100% that ProPublica’s Surgeon Scorecard is, in short, kind of a mess. It is, however, at least a way to start a conversation about measuring outcomes in a way that patients, not just number-crunchers in medical administration, can see and understand.

    This is why I do what I do, when I’m not covering hospital medicine as a journo – I’m working my fingers to the bone to try to build literacy on the part of what I call gen-pop (the average human) on how to collaborate with their clinical team(s) on their healthcare, and how to understand if they’re hearing science or woo-woo when assessing treatment options.

    Together – smart clinicians and savvy e-patients – we just might have a chance at building a better healthcare delivery system. Including working together to improve PP’s scorecard idea.

Leave A Comment

For security, use of Google's reCAPTCHA service is required which is subject to the Google Privacy Policy and Terms of Use.

About the Author: Bradley Flansbaum

Bradley Flansbaum, DO, MPH, MHM works for Geisinger Health System in Danville, PA in both the divisions of hospital medicine and population health. He began working as a hospitalist in 1996, at the inception of the hospital medicine movement. He is a founding member of the Society of Hospital Medicine and served as a board member and officer. He speaks nationally in promoting hospital medicine and has presented at many statewide meetings and conferences. He is also actively involved in house staff education. Currently, he serves on the SHM Public Policy Committee and has an interest in payment policy, healthcare market competition, health disparities, cost-effectiveness analysis, and pain and palliative care. He is SHM’s delegate for the AMA House of Delegates. Dr. Flansbaum received his undergraduate degree from Union College in Schenectady, NY and attended medical school at the New York College of Osteopathic Medicine. He completed his residency and chief residency in Internal Medicine at Long Island Jewish Medical Center in New York. He received his M.P.H. in Health Policy and Management at Columbia University. He is a political junky, and loves to cook, stay fit, read non-fiction, listen to many genres of music, and is a resident of Danville, PA.

Categories

Related Posts

By Suchita Shah Sata, MD, SFHM
September 30, 2022 |  0
If you were designing the perfect hospitalist job description, what would be the optimal workload to achieve high productivity? This was the crux of the discussion during September’s JHMChat. The conversation featured Drs. Marisha Burden, Moksha Patel, Mark Kissler, and Elizabeth Harry as well as researcher Angela Keniston, coauthors of “Measuring and driving hospitalist value: […]
By Lanna Felde, MD
November 22, 2021 |  2
Last week’s #JHMChat saw an all-star guest lineup including Drs. Adam Rodman, Zahir Kanjee, Laura McNamara, and Shane Warnock of Beth Israel Deaconess Medical Center debating the value of the routine daily physical exam – is it worthwhile or a waste of time? This question was recently debated in our Point: Counterpoint series and got […]
December 11, 2020 |  0
I’ve been thinking a lot about endurance recently. For the third time, COVID-19 is surging in the U.S., and this time, it’s not localized to New York or the Sun Belt – it’s everywhere. Healthcare workers exhausted from the first and second waves – and those in smaller Midwestern communities who are just now experiencing […]
Go to Top