Abstract
Randomized clinical trials (RCT) are the gold standard for informing treatment decisions. However, observational studies are often plagued by selection bias, and expert-selected covariates may insufficiently adjust for confounding. We explore how unstructured clinical text can be used to reduce selection bias and improve medical practice. We develop a framework based on natural language processing (NLP) to uncover interpretable potential confounders from text. We validate our method by comparing the estimated hazard ratio (HR) with and without the confounders against established RCTs. We apply our method to four cohorts built from localized prostate and lung cancer datasets from the Stanford Cancer Institute and show that our method shifts the HR estimate towards the RCT results. The uncovered terms can also be interpreted by oncologists for clinical insights. We present this proof-of-concept study to enable more credible causal inference using observational data, uncover meaningful insights from clinical text, and inform high-stakes medical decisions.
Introduction
In the hierarchy of medical evidence, Randomized Clinical Trials (RCTs) sit at the top. By randomly assigning treatments, they eliminate selection bias—the tendency for doctors to prescribe specific treatments based on a patient's health status. However, RCTs are notoriously expensive, time-consuming, and often exclude complex patient populations.
Electronic Medical Records (EMRs) offer a vast alternative for observational research, but they are inherently biased. For instance, a surgeon might only operate on the healthiest patients, while sicker patients are referred for radiation. Standard observational studies attempt to correct for this using "expert-selected" structured variables like age or cancer stage. Yet, these structured fields often miss the nuanced clinical "gut feelings" or specific comorbidities documented only in a physician’s free-text notes. This research introduces a framework to mine these "hidden" confounders from unstructured text to bridge the gap between observational data and RCT-level credibility.
Methodology
The researchers developed a multi-step NLP pipeline designed to transform messy clinical notes into actionable causal insights. The process follows four primary stages:
- Text Preprocessing: Using scispaCy, a specialized NLP model for biomedical text, the team identified clinical entities within over a million notes. These were converted into a "bag-of-words" representation using TF-IDF (Term Frequency-Inverse Document Frequency) weighting to highlight significant terms.
- The Intersection Heuristic: To qualify as a confounder, a variable must influence both the treatment assignment and the patient outcome. The framework identifies these by training two separate Lasso regression models:
- A model to predict the treatment received by the patient.
-
A Cox Proportional Hazards model to predict patient survival. The "potential confounders" are defined as the intersection of variables that maintain non-zero coefficients in both models.
-
Causal Estimation: The study then performed survival analysis using Propensity Score Matching and Inverse Propensity Weighting. They compared three models: one using only structured data, one using only text-derived terms, and a combined model.
Datasets and Validation
The framework was applied to data from the Stanford Cancer Institute Research Database (SCIRDB) and the California Cancer Registry, focusing on two primary cohorts:
- Prostate Cancer: 1,822 patients with over 550,000 clinical notes.
- Stage I Non-Small Cell Lung Cancer (NSCLC): 749 patients with nearly 650,000 clinical notes.
These cohorts were selected specifically because high-quality RCT results already exist for these conditions, providing a "ground truth" benchmark to validate whether the NLP-derived confounders actually improved the accuracy of the observational findings.
Key Results: Shifting Toward the Truth
The most significant finding was that the inclusion of text-derived confounders consistently shifted the Hazard Ratio (HR) of observational data toward the HR reported in RCTs, effectively "de-biasing" the EMR data.
- Prostate Cancer (Surgery vs. Radiation): Using only structured data, the HR was a biased 2.51 (suggesting radiation was significantly better). Incorporating text-derived confounders shifted the HR to 1.54, moving it much closer to the RCT-reported equipoise (where HR ≈ 1.0).
- Prostate Cancer (Surgery vs. Monitoring): The HR shifted from 2.71 to 1.10, representing a 59.1% shift toward the RCT result.
- Lung Cancer (Surgery vs. Radiation): The model identified the
alkmutation and terms likesevere(often indicating COPD or emphysema) as key confounders that adjusted survival estimates toward clinical expectations.
Clinical Interpretability
Unlike "black-box" deep learning models, this bag-of-words approach provides high interpretability. For example, the model flagged terms like bladder and urothelial as major confounders in prostate cancer. Upon review, oncologists realized these terms identified a specific subgroup: patients diagnosed with prostate cancer during surgery for bladder issues. These patients were generally sicker and had worse prognoses, explaining why surgery previously appeared to have worse outcomes in raw observational data.
Limitations and Future Directions
While promising, the study notes several limitations. The method can only identify confounders that are explicitly documented; if a physician makes a decision based on a factor they never record, the bias remains. Additionally, the current NLP approach struggles with negation (e.g., "no chest pain") and the nuances of clinical context. Future work will explore more sophisticated, context-aware models and non-linear survival models to further refine the extraction of these critical clinical insights.
Conclusion
This proof-of-concept study demonstrates that unstructured clinical text contains vital information for causal inference. By uncovering interpretable confounders, researchers can produce more credible observational studies, helping to inform high-stakes medical decisions when RCT data is unavailable.
Summary updated