Abstract
Identifying when to give treatments to patients and how to select among multiple treatments over time are important medical problems with a few existing solutions. In this paper, we introduce the Counterfactual Recurrent Network (CRN), a novel sequence-to-sequence model that leverages the increasingly available patient observational data to estimate treatment effects over time and answer such medical questions. To handle the bias from time-varying confounders, covariates affecting the treatment assignment policy in the observational data, CRN uses domain adversarial training to build balancing representations of the patient history. At each timestep, CRN constructs a treatment invariant representation which removes the association between patient history and treatment assignments and thus can be reliably used for making counterfactual predictions. On a simulated model of tumour growth, with varying degree of time-dependent confounding, we show how our model achieves lower error in estimating counterfactuals and in choosing the correct treatment and timing of treatment than current state-of-the-art methods.
Introduction
In clinical practice, decision-making is rarely a one-off event. Physicians must continuously evaluate a patient's state to decide which treatment to administer, when to initiate it, and when to cease therapy. While Randomized Controlled Trials (RCTs) provide the most reliable evidence for these decisions, they are often limited in scope, duration, and patient diversity. Consequently, there is a growing interest in leveraging longitudinal observational data, such as Electronic Health Records (EHRs), to estimate treatment effects.
However, observational data is plagued by time-varying confounding. This occurs when patient covariates that influence future treatment assignments are themselves affected by past treatments. For example, a doctor might prescribe a specific medication because a patient’s lab results deteriorated following a previous treatment. If a predictive model is trained directly on this data, it will likely learn the physician's decision-making policy rather than the underlying causal effect of the treatment. Traditional methods like Marginal Structural Models (MSMs) attempt to correct this via Inverse Probability of Treatment Weighting (IPTW), but these often struggle with high-dimensional data and non-linear dependencies.
The Counterfactual Recurrent Network (CRN)
To overcome the limitations of weighting-based methods, we propose the Counterfactual Recurrent Network (CRN). The CRN shifts the paradigm from weighting samples to learning balanced representations of patient history that are invariant to the treatment assigned.
Sequence-to-Sequence Architecture
The CRN utilizes an encoder-decoder architecture designed for multi-step ahead prediction. The encoder processes the longitudinal patient history—including past covariates and treatments—to build a comprehensive state representation. The decoder then uses this representation to predict future outcomes under various hypothetical (counterfactual) treatment sequences.
Adversarial Balancing
The core innovation of CRN is the use of Domain Adversarial Training to handle time-varying confounding. By treating each treatment assignment as a distinct "domain," the model aims to learn a representation $\Phi(\bar{\mathbf{H}}_t)$ that is predictive of the outcome but contains no information about which treatment was actually administered.
This is achieved through a Gradient Reversal Layer (GRL) connected to a treatment classifier. During backpropagation, the GRL reverses the gradients from the treatment classifier, effectively forcing the encoder to "unlearn" the correlations between patient history and treatment assignment. The result is a balanced representation that mimics a randomized trial setting, allowing for unbiased counterfactual estimation.
Experimental Evaluation
We evaluated the CRN using two primary datasets:
- Tumor Growth Simulation: To assess counterfactual accuracy, we utilized a bio-mathematical model of lung cancer growth. This simulation allowed us to generate 10,000 patient trajectories where the ground truth for all possible treatment paths (chemotherapy and radiotherapy) was known. We introduced varying degrees of time-dependent confounding ($\gamma$) to test the model's robustness.
- MIMIC-III: We applied the model to real-world EHR data from the MIMIC-III database, focusing on predicting white blood cell counts for ICU patients receiving antibiotics.
Results and Discussion
The CRN demonstrated superior performance across all benchmarks. In the tumor growth simulation, CRN significantly outperformed existing state-of-the-art methods like Recurrent Marginal Structural Networks (RMSNs).
As shown in the results, the performance gap between CRN and other models widens as the degree of time-dependent confounding ($\gamma$) increases. At high levels of confounding, CRN achieved a 17% improvement in RMSE over RMSNs. Furthermore, an ablation study revealed that removing the adversarial balancing component ($\lambda=0$) led to a 48.1% increase in error, highlighting the critical role of the GRL in achieving unbiased predictions.
Beyond error metrics, CRN proved more effective at clinical decision support. It consistently identified the optimal timing for starting or stopping treatments, leading to better simulated patient outcomes (e.g., smaller final tumor volumes) compared to policies derived from baseline models.
Conclusion and Future Directions
The Counterfactual Recurrent Network provides a robust framework for estimating treatment effects over time by leveraging adversarial representation learning. By explicitly removing the bias inherent in observational medical history, CRN enables more accurate "what-if" analyses for complex treatment plans.
While promising, the model relies on the assumption of "no hidden confounders"—that all variables influencing treatment decisions are captured in the data. Future research will explore extending CRN to handle continuous treatment dosages and further validating its utility in diverse real-world clinical settings where ground-truth counterfactuals are unavailable.
Summary updated