Uncovering interpretable potential confounders in electronic medical records

Jiaming Zeng, Michael F. Gensheimer, Daniel Rubin, Susan Athey, Ross D. Shachter

TL;DR

This paper introduces a framework that uses Natural Language Processing (NLP) to extract interpretable confounders from unstructured clinical notes in Electronic Medical Records (EMRs). By identifying terms that predict both treatment assignment and patient survival, the researchers were able to adjust observational study results to more closely align with the "gold standard" of Randomized Clinical Trials (RCTs).

What Problem Does This Paper Address?

In medicine, Randomized Clinical Trials (RCTs) are the gold standard for determining if a treatment works. However, they are expensive and slow. Researchers often turn to observational studies using EMR data, but these are frequently plagued by selection bias.

For example, a surgeon might only operate on the healthiest patients, while sicker patients receive radiation. If you just look at the data, surgery will look "better" simply because the patients were healthier to begin with. While researchers try to adjust for this using "expert-selected" variables (like age or cancer stage), these structured fields often miss the nuanced clinical "gut feelings" or specific comorbidities documented only in a doctor’s free-text notes. This paper aims to bridge that gap by mining those notes for "hidden" confounders.

Methods

The authors developed a multi-step pipeline to turn messy clinical notes into actionable causal insights:

  1. Text Preprocessing: They used scispaCy (a specialized NLP model for biomedical text) to identify clinical entities and converted notes into a "bag-of-words" representation using TF-IDF weighting.
  2. Confounder Discovery: To be a confounder, a variable must influence both the treatment assignment and the outcome. The researchers trained two separate Lasso regression models:
  3. One to predict which treatment a patient received.
  4. One to predict patient survival (using a Cox Proportional Hazards model).
  5. The Intersection Heuristic: They identified "potential confounders" by taking the intersection of variables that had non-zero coefficients in both models.
  6. Causal Estimation: They then performed survival analysis (using techniques like Propensity Score Matching and Inverse Propensity Weighting) using three sets of variables: structured data only, text-derived "intersection" terms only, and a combination of both.

Datasets

The study utilized data from the Stanford Cancer Institute Research Database (SCIRDB) and the California Cancer Registry:

  • Prostate Cancer Cohort: 1,822 patients with over 550,000 clinical notes.
  • Stage I Non-Small Cell Lung Cancer (NSCLC) Cohort: 749 patients with nearly 650,000 clinical notes.

The researchers specifically chose these cohorts because established RCT results already exist for them, providing a benchmark to validate their method.

Key Insights and Ideas

  • Text as a Proxy for "Clinical Intuition": The model uncovered terms like bladder and urothelial as major confounders in prostate cancer. Clinicians realized these terms identified a specific subgroup: patients diagnosed with prostate cancer during surgery for bladder issues. These patients were sicker and had worse prognoses, explaining why surgery previously looked "worse" in some observational prostate data.
  • Interpretability is Mandatory: Unlike "black-box" deep learning models (like Transformers), this bag-of-words approach allows oncologists to look at a specific word (e.g., alk or severe) and understand why the model flagged it as a source of bias.
  • The "Shift" Toward Truth: The most important insight is that adding these text-derived terms consistently shifted the Hazard Ratio (HR) of observational data toward the HR reported in RCTs. It essentially "de-biased" the EMR data.

Key Results

The framework successfully reduced selection bias across four different treatment comparisons:

  • Prostate Cancer (Surgery vs. Radiation): Using only structured data, the Hazard Ratio (HR) was a biased 2.51 (suggesting radiation was significantly better). Adding the text-derived confounders shifted the HR to 1.54, moving it much closer to the RCT-reported equipoise (where HR ≈ 1.0).
  • Prostate Cancer (Surgery vs. Monitoring): The HR shifted from 2.71 to 1.10 (a 59.1% shift toward the RCT result).
  • Lung Cancer (Surgery vs. Radiation): The model identified the alk mutation and terms like severe (indicating COPD or emphysema) as key confounders that shifted the survival estimates toward clinical expectations.

Limitations and Future Work

  • Sample Size: The study relied on relatively small cohorts (hundreds to thousands of patients), which limited the use of more complex deep learning architectures.
  • "Observable" Bias Only: The method can only find confounders that are actually written down. If a doctor makes a decision based on a factor they never record in a note, this method cannot account for it.
  • Proportional Hazards Assumption: The study relies on the standard Cox-PH model; future work could explore non-linear models or time-dependent confounders.
  • Ambiguity: Current simple NLP struggles with negation or ambiguity (e.g., "it is unclear if the patient has chest pain"). Integrating more sophisticated context-aware models is a logical next step.

Summary updated

Abstract

Randomized clinical trials (RCT) are the gold standard for informing treatment decisions. Observational studies are often plagued by selection bias, and expert-selected covariates may insufficiently adjust for confounding. We explore how unstructured clinical text can be used to reduce selection bias and improve medical practice. We develop a framework based on natural language processing to uncover interpretable potential confounders from text. We validate our method by comparing the estimated hazard ratio (HR) with and without the confounders against established RCTs. We apply our method to four cohorts built from localized prostate and lung cancer datasets from the Stanford Cancer Institute and show that our method shifts the HR estimate towards the RCT results. The uncovered terms can also be interpreted by oncologists for clinical insights. We present this proof-of-concept study to enable more credible causal inference using observational data, uncover meaningful insights from clinical text, and inform high-stakes medical decisions.
BibTeX
@inproceedings{zeng2022uncovering,
  author = {Jiaming Zeng and Michael F. Gensheimer and Daniel Rubin and Susan Athey and Ross D. Shachter},
  title = {{Uncovering interpretable potential confounders in electronic medical records}},
  booktitle = {Nature Communications},
  year = {2022},
  doi = {10.1038/s41467-022-28546-8},
}

Loading related content...

Figure