Can Computerized Decision Support Get Docs to Toe the Line on Quality?

A humorous and telling story about quality measurement, decision support, and human nature:

I was visiting professor at a very good academic medical center a year or so ago. On these trips, one of the fun things I get to do is meet with the residents. Sometimes they present a clinical case to me, but this day they wanted to talk about healthcare policy. So I thought I’d check out what they knew about the new world of quality measurement and transparency.

“Who admitted a patient with pneumonia last night?” I asked the bleary-eyed, overcaffeinated group of 20-somethings, each looking only slightly older than my kids. Three interns hesitatingly raised their hands.

“If I wanted to figure out whether you provided high quality care,” I continued, “what should I look at?”

“I think we saved this guy’s life,” one beamed. “Yeah, and I had a lady who was confused, hypotensive, and hyperglycemic,” said another, “and we did a really good job taking care of her. She’s much better this morning.”

“That’s great,” I said, “but your quality is actually being measured in a different way. Here are the things that insurers and the government are looking at – did you give pneumovax and flu vaccine, did you document smoking cessation counseling, did you obtain blood cultures in the ED, did you check an oxygen saturation, did you give guideline-recommended antibiotics, and did you give the antibiotics promptly?  What do you think?”

“Well, those are important, I guess,” one intern said haltingly.

“Now,” I pressed on, “you should also know that your performance on these measures is posted on the Web. Would you” – I pointed to a resident half-dozing by a computer terminal – “mind going to this site (Medicare’s “Hospital Compare”), type in our zip code, and let’s see what happens.”

And she did. We quickly navigated to the hospital’s pneumonia performance, put in a few local comparison hospitals (read: competitors), and saw that the hospital wasn’t doing particularly well – at the mean on a few of the measures, below it on others.

“Now let’s say that you were the czar of quality at this hospital,” I said, fully knowing that the Director of Quality was sitting, slightly red-faced, in the back of the room, “and you saw these results. What would you do to try to improve the performance?”

“I’d give the residents a lecture on pneumonia and quality.”

“Naw, that wouldn’t do anything. Maybe they should buy us a nice lunch!” The group giggled.

“No, I’ve got an idea,” excitedly said a third. “When you admit somebody with pneumonia and start to enter your orders into the computer system” – this was one of the 15% or so of U.S. hospitals with computerized provider order entry – “then the computer could remind you to give the pneumovax and flu vaccine.” I was impressed – they had stumbled onto the premise behind clinical decision support, the Holy Grail of the quality movement. But before I could open my mouth to endorse this strategy, another resident, one of the docs who had admitted a pneumonia patient the previous night, sprang to life…

“Oooooh,” she gushed, “I actually think that happened!” I asked her what she meant. “When I admitted my patient last night,” she continued breathlessly, “I remember a pop-up box that said something like, ‘Did You Remember To Give Pneumovax?’”

“Wow,” I said, “so what did you do?”

“Oh, I clicked out of it. I was too busy last night – I got creamed, 9 admits!”

The quality officer in the back nearly seized.

“Well, that’s interesting,” I went on. “I see how that could happen. So let’s say you’re the quality czar and the interns are ignoring your hard-fought-for pop-up messages. What would you do then?”

“I’d email the intern any time he clicked out of one without giving the medicine.”

“Nah, that would be a waste of time. The ‘terns never read their e-mail.”

“I got it,” said a thin guy in scrubs in the back row. “When the interns don’t follow the guidelines, I’d send a letter” – amazing how, in the e-mail era, letters now have the gravitas formerly reserved for telegrams – “to the Chief Residents!”

The Chief Residents are the Gods of any residency program: part-psychotherapist, part-camp counselor, part-Sensei. So this was a great idea: if the CRs knew that the interns weren’t with the program, they’d make things right before you could say William Osler.

“Oooooh,” said the Chief Resident sitting to my left. “I’ve been getting all these letters for the past few months.”

“And what do you do with them?” I asked, anticipating the answer.

And with that, she slowly, deliberately, pointed to a corner of the room, where there was a barely touched pile of letters, about 3 feet high.

It’s been a while since this episode. I’m hoping the Quality Director has recovered. This quality thing is not as easy as it looks.