Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning

Jannik Deuschel, Caleb N. Ellington, Benjamin J. Lengerich, Yingtao Luo, Pascal Friederich, Eric P. Xing

Introduction: The Accuracy-Interpretability Trade-off

In medical AI, a persistent tension exists between predictive accuracy and model interpretability. Human clinicians do not follow static rules; their decisions are dynamic, influenced by a patient’s history, previous treatments, and evolving symptoms. Traditional "interpretable" models, such as decision trees or simple logistic regression, often lack the flexibility to capture these nuances, resulting in subpar performance. Conversely, "black-box" models like Recurrent Neural Networks (RNNs) offer high accuracy but fail to provide the transparency required for clinical trust and safety auditing.

Contextualized Policy Recovery (CPR) addresses this by reframing policy recovery as a multi-task learning problem. Instead of seeking a single "universal policy," CPR treats every patient context as a unique task requiring its own simple, interpretable model.

Methods: A Hybrid Architecture for Dynamic Interpretability

The core innovation of CPR is its hybrid architecture, which combines the strengths of deep learning with the transparency of linear models.

  1. The Context Encoder (Black-Box): An RNN or LSTM processes the patient's history (past symptoms and actions). Rather than predicting the next action directly, it outputs the parameters ($\theta$) for a linear model tailored to the current context.
  2. The Policy Model (Glass-Box): This simple linear map uses the generated parameters to map current observations to an action.
Comparison of CPR with existing policy recovery models

This structure allows clinicians to see the exact weight given to specific variables (e.g., Heart Rate vs. Creatinine) at any given moment. The authors propose two variants: Local CPR, for point-in-time interpretability, and Global CPR, which uses a "telescoping" linear combination to quantify the exact effect of every historical feature on the final decision.

Key Insights and Results

The framework was evaluated on MIMIC-III (antibiotic prescription), ADNI (MRI ordering), and simulated datasets. CPR significantly outperformed state-of-the-art (SOTA) interpretable methods like POETREE and INTERPOLE. In the MIMIC-III task, CPR achieved a +22% AUROC improvement, matching the performance of pure black-box LSTMs.

CPR identifies outlier patients and adapts to demographic differences

Beyond performance, CPR serves as a powerful discovery tool. By clustering the generated linear models, the authors identified distinct "treatment regimes." For instance, for patients already on antibiotics, the model's priorities shifted toward monitoring side effects (like kidney failure) rather than just infection markers. Furthermore, CPR proved more robust in modeling outlier groups, such as younger patients, where universal models often fail.

Discussion and Future Work

The results suggest that the accuracy-interpretability gap is not an inherent law but an artifact of model design. By allowing parameters to be dynamic and context-dependent, CPR achieves the "best of both worlds."

However, limitations remain. The current study focuses on offline policy recovery from historical data; real-time online learning remains to be extensively tested. Additionally, while the model handles binary decisions well, extending it to high-dimensional or continuous action spaces (e.g., drug dosages) is a key area for future research. Finally, while mathematically transparent, the "Global CPR" output may still require specialized visualization tools to be fully accessible to clinicians.

Summary updated

BibTeX
@inproceedings{deuschel2024contextualized,
  author = {Jannik Deuschel and Caleb N. Ellington and Benjamin J. Lengerich and Yingtao Luo and Pascal Friederich and Eric P. Xing},
  title = {{Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=YEQM0asWCH},
}
Figure