Suppose I told you one of your patients:
–Didn’t vote in the last five elections
–Bought a 24 pack of Snickers bars and 12 pack of Cheetos on Amazon every three months
–Participates in a Burger King Whopper loyalty discount program
–Owns a Harley Davidson
What conclusions would you draw as they relate to future illness, adherence, or their state of health?
That is a darn good question. You might infer an awful lot–but would it tell you more than knowing the patient’s zip code and prescription refill list only? Sometimes simple wins.
This is the dicey world of analytics—and if you need proof have a look at what is out there now.
Which gets me to the matter I want to highlight—the results of a JAMA study whose primary conclusion I cite below (given what we are all trying to do at Geisinger I had tremendous interest in the findings).
“This work reaffirms that the social environment is associated with health outcomes. However, these results suggest that information about the environment in which a person lives may not contribute much more to population risk assessment than is already provided by EHR data. Although this result does not mean that integrating social determinants of health into the EHR has no benefit, researchers may be able to use EHR data alone for population risk assessment.”
We have tons and tons of data. Tons. But if you can reliably predict who will have an untoward clinical outcome, in the case of the study more frequent or higher numbers of hospital visits, with what we already had 3, 5, or 7 years ago, why go through the hassle. A “just the facts ma’am approach” might suit us all fine for a lot less sweat and cost.
So what did the investigators do?
Firstly, with rule building, you start with derivation set of patient data and test it in an f/u validation cohort (90 and 122K patients, respectively–a generous sample). The authors looked to see whether the model they assembled using neighborhood characteristics, again, added to what we already had, could predict those folks that might have a shorter time to use of the ED and wards, as well as a shorter time to hospitalizations due to accidents, asthma, flu, MI, and CVA. If you can identify them early, you intervene with preemptive interventions.
And I’ll be darn, contrary to my priors, the extra juice did not improve risk prediction of events beyond what clinicians could already glean from the EHR. A patient’s age, sex, race/ethnicity, and insurance status plus a few other things might be as good as a data run with a variable stack of employment, education, and home value thrown in. Zip code is everything.
Here it is in the words of the researchers–repeated again:
“[…] information about the environment in which a person lives may not contribute much more to population risk assessment that is already provided by EHR data. Although this result does not mean that integrating social determinants of health into the EHR has no benefit, researchers may be able to use EHR data alone for population risk assessment.”?
I have seen very few studies of this sort; clinically relevant modeling with social and behavioral variables included in a large dataset purposed for risk prediction. And as always, I discover. Lessons like this over a career humble and make you wise.
Not all that glitters is gold–and venture capitalists and futurists take heed. Not every vendor or seller has rigorously studied its product, or outcomes of its application have gone unpublished. And in my book, that amounts to nothing more than high priced woo-woo peddling.