In clinical medicine, the most important questions are often "what if" questions. What if we started chemotherapy today instead of next week? What if we switched from radiotherapy to a combined treatment? Answering these requires estimating counterfactuals—outcomes for treatment paths that were never actually taken.
A team from Oxford, UCLA, and Cambridge recently introduced the Counterfactual Recurrent Network (CRN) at ICLR 2020, a novel architecture designed to peer into these alternative futures using observational data.
TL;DR
The Counterfactual Recurrent Network (CRN) is a sequence-to-sequence model that estimates treatment effects over time by using domain adversarial training to break the link between a patient’s history and the treatments they received. This allows it to outperform existing methods in predicting how a patient will respond to various future treatment plans, even when the data is heavily biased by past medical decisions.
What Problem Does This Paper Address?
Most medical data is "observational" (like Electronic Health Records), not from randomized trials. This creates a massive problem for machine learning: confounding bias.
In the real world, doctors don't assign treatments randomly; they assign them based on the patient's current state. If a doctor gives a potent drug only to the sickest patients, a naive model might conclude the drug causes poor outcomes. In a longitudinal (time-series) setting, this is even trickier because of time-varying confounders: past treatments affect a patient's health, which then influences future treatment decisions.
Existing epidemiological tools like Marginal Structural Models (MSMs) try to fix this by "weighting" data points, but these often suffer from high variance and numerical instability. The authors sought a way to use the power of deep representation learning to "de-bias" patient history directly.
Methods
The CRN uses a sequence-to-sequence (seq-to-seq) architecture consisting of an encoder and a decoder, but with a clever twist:
- The Encoder (Patient History): An RNN processes the patient's history (covariates and past treatments). To remove bias, the authors use Domain Adversarial Training. They attach a "treatment classifier" to the RNN's hidden state.
- The Tug-of-War: Through a Gradient Reversal Layer (GRL), the network is trained to maximize the loss of the treatment classifier while minimizing the loss of the outcome predictor.
- The Result: The network learns a "balancing representation"—a summary of the patient that is useful for predicting health outcomes but contains no information about which treatment the doctor is likely to choose next.
- The Decoder (Future Planning): Once the encoder has a clean, unbiased representation of the patient's state, the decoder uses it to project future outcomes under any hypothetical sequence of treatments. It acts as a "medical simulator," allowing clinicians to test different treatment timings and combinations.
Datasets
- Simulated Tumor Growth: Since we can never observe true counterfactuals in real life (we can't give a patient two different treatments at the same time), the authors used a state-of-the-art bio-mathematical model of lung cancer. This allowed them to know the "ground truth" for every possible treatment path.
- MIMIC-III: A real-world database of ICU patients. They used this to predict how antibiotics affect white blood cell counts over time, validating the model's performance on messy, real-world electronic health records.
Key Insights and Ideas
- Representation over Weighting: Instead of trying to "re-weight" samples (which is what MSMs do), CRN changes the features themselves. By forcing the representation to be treatment-invariant, the model becomes "blind" to the biases of the original doctors' policies.
- Adversarial De-biasing: This is a creative application of domain adaptation. In this context, different treatments are treated as different "domains." If the model can't tell which "domain" (treatment) a patient belongs to based on their history, the representation is balanced.
- Seq-to-Seq for Planning: Most causal models only look one step ahead. By using a decoder, CRN can simulate entire multi-year treatment plans, which is much closer to how clinical decisions are actually made.
Key Results
- Lower Error: On the tumor simulation, CRN achieved significantly lower Root Mean Squared Error (RMSE) in predicting counterfactuals compared to the current state-of-the-art (Recurrent Marginal Structural Networks).
- Handling High Bias: As the "confounding" in the data increased (i.e., as doctors' policies became more rigid), the performance gap between CRN and other models widened. When bias was high, CRN improved counterfactual estimation by 48.1% over a standard RNN.
- Optimal Timing: The model was significantly better at identifying the best time to start or stop a treatment, a critical task in oncology where timing can be as important as the drug itself.
Limitations and Future Work
- The "No Hidden Confounders" Assumption: Like almost all causal inference methods, CRN assumes that we have recorded all the variables that influenced the treatment decision. If a doctor used a piece of information not in the data (e.g., the patient "looked tired"), the model's predictions may still be biased.
- Theoretical Guarantees: While the empirical results are strong, the authors note that more work is needed to provide formal theoretical bounds on the counterfactual error.
- Dosage: The current model handles discrete treatment types (e.g., "Chemo" vs "No Chemo") but doesn't yet account for continuous dosages, which is a natural next step for pharmacological applications.
Paper Citation: Bica, I., Alaa, A. M., Jordon, J., & van der Schaar, M. (2020). Estimating counterfactual treatment outcomes over time through adversarially balanced representations. International Conference on Learning Representations (ICLR).
Summary updated