Man vs. Machine: Comparing Physician vs. Electronic Health Record-Based Model Predictions for 30-Day Hospital Readmissions

A human-plus-machine approach to predicting 30-day hospital readmissions is better than either human or machine alone was the finding of Oanh Nguyen, MD, MAS, Anil Makim, MD, MAS and colleagues in a recent Journal of General Internal Medicine article.  

Unplanned readmissions within 30 days account for $26 billion of annual Medicare costs. Since 2012, hospitals have been subject to financial penalties under the Centers for Medicare and Medicaid Services Hospital Readmissions Reduction Program (HRRP) for excessive all-cause 30-day readmissions among patients with an index hospitalization for an increasing number of targeted medical and surgical conditions, including congestive heart failure, acute myocardial infarction, pneumonia, and chronic obstructive pulmonary disease. These federal penalties have stimulated intense efforts to develop readmission reduction intervention strategies, which are highly resource-intensive but have been only modestly effective when indiscriminately applied to all hospital inpatients
Identifying hospitalized patients at high risk for readmission before they are discharged can enable interventions to be targeted to those at the highest risk and therefore most likely to benefit. Electronic health record (EHR)-based risk prediction models incorporating granular clinical data (i.e., vital signs, hospital-acquired conditions, laboratory results, etc.), are superior to approaches using claims-based administrative data, but are limited by the types of information documented and encoded in the EHR.  A growing body of evidence suggests that social, functional, and behavioral factors are associated with increased risk of readmission, and that incorporating this information into prediction models improves readmission risk prediction across a variety of conditions. However, at present, this information is not uniformly available in EHRs

Clinician perceptions of readmission risk are readily ascertainable and may incorporate valuable information on severity and complexity of patient illness, as well as information on social, functional, and behavioral factors unavailable in the EHR, but the comparative accuracy of physician predictions for 30-day readmissions is not well established

The researchers conducted a head-to-head comparison of the performance of physicians versus a validated EHR-based prediction model in predicting 30-day hospital readmissions in a large, diverse cohort of hospitalized adults with a wide range of clinical, social, functional, and behavioral needs. The researchers found both clinicians and the EHR model had similarly modest discrimination for readmissions, though each strategy had unique strengths and blind spots. The EHR model was better at predicting who would be readmitted but overestimated readmission risk overall (i.e., high sensitivity but low specificity), while clinicians were better at predicting who would not be readmitted but underestimated readmission risk overall (i.e., high specificity but low sensitivity). A human-plus-machine approach incorporating clinician predictions as a variable in the EHR model had significantly better discrimination and also best optimized sensitivity and specificity. 

The researchers’ findings have several implications for hospitals and health systems developing workflows to identify and target hospitalized patients at risk for 30-day readmissions. Hospitals with limited care transitions resources or lacking real-time predictive analytic capabilities could opt for a “human-only” approach to reducing readmissions, where readmission reduction interventions would be targeted to patients who were identified as being at high-risk for readmission by their inpatient physicians using a 1-question screening tool that could be embedded directly into an EHR. Resources would thus be allocated to those most likely to potentially benefit from intervention, though many high-risk patients would be missed as a tradeoff of this approach. Hospitals for whom readmissions reduction is a high priority—and who have actionable, predictive analytic resources—could consider a “human-plus-machine” approach to target a larger number of patients for intervention, though some patients identified in this approach may be less likely to be readmitted, and thus may benefit less from intervention.

Readmission risk prediction strategies should seek to incorporate clinician assessments of readmission risk to optimize the accuracy of readmission predictions.

Dr. Nguyen’s research is focused on understanding and optimizing hospital care in safety-net settings and developing pragmatic approaches to addressing social determinants of health, particularly on the development and evaluation of transitional care strategies that address social vulnerabilities among individuals with complex medical needs.

 

 

 

Anil Makam, MD, MAS, is an academic hospital medicine physician and a health services researcher. His research is at the intersection of geriatrics, hospital medicine, and post-acute care, specifically focusing on the role of long-term acute care hospitals (LTACs). Dr. Makam applies health services research and epidemiological methods using Medicare claims, EHR data, and prospective observational cohort data to examine predictors and variation in LTAC use, comparative effectiveness of the LTAC model of care versus alternative care settings, and patterns of recovery for older adults transferred to LTACs.

 

Man vs. Machine: Comparing Physician vs. Electronic Health Record-Based Model Predictions for 30-Day Hospital Readmissions

Nguyen OK, Washington C, Clark CR, Miller ME, Patel VA, Halm EA, Makam AN. J Gen Intern Med. 2021 Jan 14. doi: 10.1007/s11606-020-06355-3.