This paper, presented at KDD 2020, tackles a common but tricky scenario in medical AI: learning how to treat patients when you have multiple competing goals and a unique kind of "perfect" historical data. Using antibiotic prescription for Urinary Tract Infections (UTIs) as a case study, the authors demonstrate how machine learning can significantly outperform human clinicians by navigating the trade-offs between individual patient health and broader public health concerns.
TL;DR
The authors develop and compare methods for learning treatment policies in settings where historical data reveals the outcomes of all possible treatments, even though these outcomes are unknown at the moment a doctor must act. Their "direct" policy learning approach not only outperforms clinicians but also allows for seamless integration of complex goals, such as knowing when to defer a difficult case back to a human expert.
What Problem Does This Paper Address?
In many medical scenarios, doctors must make an "empiric" decision—prescribing a treatment immediately before definitive test results are available. For UTIs, a lab test can eventually show exactly which antibiotics would have worked for a specific patient. While this information arrives too late for the initial prescription, it creates a goldmine of historical data where we "fully observe" all potential outcomes.
The challenge is two-fold:
- Multi-objective Trade-offs: Doctors aren't just trying to cure the current patient (minimizing Inappropriate Antibiotic Therapy, or IAT). They also need to be "antibiotic stewards," avoiding the overuse of broad-spectrum (2nd-line) drugs that drive long-term population-level resistance.
- Model Misspecification: Traditional "indirect" methods try to predict the outcome of every drug and then pick the best one. However, if the underlying biology is complex but the optimal decision rule is simple, these predictive models might fail to find the best policy.
Methods
The researchers compare three distinct strategies:
- Thresholding (Indirect): Train models to predict the probability of success for each drug. Use a set of thresholds to decide which drugs are "likely effective," then pick the cheapest/narrowest-spectrum option among them.
- Expected Reward Maximization (Indirect): Combine effectiveness and cost into a single "reward" score using a weight ($\omega$) to represent the trade-off. Train models to predict this reward for each drug and pick the maximum.
- Direct Policy Optimization: Instead of predicting outcomes, this method learns the decision rule directly. It optimizes a surrogate loss (multinomial deviance) to find a policy that maximizes the expected reward across the population.
A key advantage of the Direct approach is its ability to handle Learning to Defer. The authors simply add a "defer to doctor" action to the model's choices, incentivizing the model to step aside when it predicts the human clinician is likely to make a better (or equally good) decision.
Datasets
- UTI Dataset: A real-world dataset of 15,806 specimens from two major Boston hospitals (MGH and BWH) collected between 2007 and 2016. It includes patient demographics, medical history, and—crucially—ground-truth resistance profiles for four common antibiotics.
- Synthetic Dataset: A controlled environment where the relationship between patient features and drug success is highly non-linear (complex), but the optimal treatment rule remains a simple linear function. This was used to stress-test the direct vs. indirect approaches.
Key Insights and Ideas
- The Pareto Frontier: There is no single "best" policy; there is a frontier of optimal trade-offs. By varying the reward weight $\omega$, the authors could generate a range of policies, allowing healthcare providers to choose their preferred balance between effectiveness and stewardship.
- Decoupling Complexity: The synthetic experiments proved that direct policy learning is more robust. It can find a simple, effective decision rule even when the underlying biological outcomes are too messy for standard predictive models to capture accurately.
- Smart Deferral: As the model was allowed to defer more cases to doctors, the performance gap on the remaining "decision cohort" actually widened. This suggests the model successfully identifies the specific types of cases where human intuition fails and algorithmic precision is most needed.
Key Results
The ML policies consistently landed "below and to the left" of human performance, meaning they achieved lower IAT rates and lower broad-spectrum usage simultaneously.
- Head-to-Head vs. Doctors: One learned policy reduced the IAT rate by 25% relative to clinicians while slightly reducing broad-spectrum usage.
- Stewardship Win: Another policy virtually eliminated 2nd-line drug usage (reducing it from 33.6% to ~1%) while still reducing the IAT rate by 10% compared to doctors.
- Synthetic Superiority: In complex environments, the direct approach approached Bayes-optimal performance with far fewer samples than the indirect approach.
Limitations and Future Work
- Nonstationarity: Antibiotic resistance patterns change over time (e.g., a drug that worked in 2010 might be useless in 2024). The authors suggest future work should explicitly model these shifting trends.
- Interpretability: While the linear direct policy is somewhat interpretable via its coefficients, more work is needed to make these "black box" suggestions truly transparent for frontline clinicians.
- Data Assumptions: The "fully observed" setting is powerful but specific. Extending these multi-objective direct methods to "partially observed" settings (where we only know the outcome of the drug actually prescribed) remains a major frontier in causal inference.
Summary updated