Causal thinking for decision making on Electronic Health Records: why and how

Matthieu Doutreligne, Tristan Struja, J. Abécassis, Claire Morgand, Leo Anthony Celi, Gaël VaroquauxarXiv 2023 · DOI · arXiv

Abstract

Accurate predictions, as with machine learning, may not suffice to provide optimal healthcare for every patient. Indeed, prediction can be driven by shortcuts in the data, such as racial biases. Causal thinking is needed for data-driven decisions. Here, we give an introduction to the key elements, focusing on routinely-collected data, electronic health records (EHRs) and claims data. Using such data to assess the value of an intervention requires care: temporal dependencies and existing practices easily confound the causal effect. We present a step-by-step framework to help build valid decision making from real-life patient records by emulating a randomized trial before individualizing decisions, eg with machine learning. Our framework highlights the most important pitfalls and considerations in analysing EHRs or claims data to draw causal conclusions. We study the impact of various choices at every step, from feature extraction to causal-estimator selection, illustrating the effect of albumin on sepsis mortality in the MIMIC-IV database.

Introduction

In healthcare, the objective is rarely just to predict an outcome, but to determine which intervention will improve it. Standard machine learning (ML) models are often "shortcut learners"—they identify correlations that do not imply causation. For example, a model might predict high mortality for patients on ventilators, not because ventilators are harmful, but because they are reserved for the most critical cases.

Using observational data like Electronic Health Records (EHRs) to guide decisions is difficult due to confounding by indication (doctors treat specific patients based on health status), temporal biases (like immortal time bias), and model sensitivity. This paper proposes a structured approach to overcome these hurdles.

Methods: The 5-Step Analytic Framework

We propose a rigorous framework to transform messy EHR data into reliable clinical evidence by emulating randomized controlled trials (RCTs).

Figure 1: The 5-step analytic framework for causal inference, detailing identification of confounders and selection of causal estimators.
  1. Framing (Target Trial Emulation): Defining the research question using the PICO(T) framework: Population, Intervention, Control, Outcome, and Time.
  2. Identification: Categorizing variables into confounders (to adjust for), colliders (to ignore), and mediators using Directed Acyclic Graphs (DAGs).
  3. Estimation: Moving beyond simple regression to "Doubly Robust" estimators like Augmented Inverse Probability Weighting (AIPW) and Double Machine Learning (DML).
  4. Vibration Analysis: Systematic sensitivity analysis to test how results change when observation windows or variable sets are tweaked.
  5. CATE (Conditional Average Treatment Effect): Using ML to identify which specific subgroups (e.g., by age or severity) benefit most from a treatment.

Case Study: Albumin in Sepsis

We utilized the MIMIC-IV database, analyzing 18,421 patients with sepsis to evaluate the effect of albumin versus crystalloids on 28-day mortality. As shown in the population characteristics, the treatment groups were not balanced at baseline, with the albumin group exhibiting higher severity markers like SOFA scores and lactate levels.

Table 1: Baseline characteristics of the study population from MIMIC-IV, illustrating the imbalance between treatment groups.

Key Insights and Results

  • Recovering the Gold Standard: By carefully adjusting for confounders, our analysis matched the "null effect" found in major randomized trials (like the ALBIOS trial), finding no significant overall difference in 28-day mortality (Odds Ratio ~0.93).
  • The Immortal Time Trap: We demonstrated that extending the observation window for treatment from 24 to 72 hours artificially made albumin look effective. This occurs because patients who died early never had the chance to receive the drug, leaving only survivors in the treatment group.
  • Subgroup Discovery: While the average effect was null, causal ML models revealed that albumin was significantly more effective for patients in septic shock, males, and those over 60.
  • Estimator Performance: The AIPW estimator combined with Random Forests proved to be the most reliable and stable method across various sensitivity tests.

Conclusion

Causal framing is essential for translating EHR data into actionable clinical advice. While machine learning provides powerful estimation tools, it cannot replace the structural rigor of trial emulation and expert-led confounder identification. Future work must continue to address unobserved confounders and improve the generalizability of these frameworks across diverse hospital systems.

Summary updated

Abstract

Accurate predictions, as with machine learning, may not suffice to provide optimal healthcare for every patient. Indeed, prediction can be driven by shortcuts in the data, such as racial biases. Causal thinking is needed for data-driven decisions. Here, we give an introduction to the key elements, focusing on routinely-collected data, electronic health records (EHRs) and claims data. Using such data to assess the value of an intervention requires care: temporal dependencies and existing practices easily confound the causal effect. We present a step-by-step framework to help build valid decision making from real-life patient records by emulating a randomized trial before individualizing decisions, eg with machine learning. Our framework highlights the most important pitfalls and considerations in analysing EHRs or claims data to draw causal conclusions. We illustrate the various choices in studying the effect of albumin on sepsis mortality in the Medical Information Mart for Intensive Care database (MIMIC-IV). We study the impact of various choices at every step, from feature extraction to causal-estimator selection. In a tutorial spirit, the code and the data are openly available.
BibTeX
@misc{doutreligne2023causal,
  author = {Matthieu Doutreligne and Tristan Struja and J. Abécassis and Claire Morgand and Leo Anthony Celi and Gaël Varoquaux},
  title = {{Causal thinking for decision making on Electronic Health Records: why and how}},
  year = {2023},
  eprint = {2308.01605},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.01605},
}
Figure