Abstract
Medical decision-making often involves balancing competing objectives, such as maximizing treatment efficacy while minimizing the use of broad-spectrum antibiotics to combat resistance. This paper investigates treatment policy learning in settings where historical data provides fully observed outcomes for all possible actions—a common scenario in clinical diagnostics like antibiotic susceptibility testing. We compare indirect predictive modeling against direct policy optimization. Our results on a real-world Urinary Tract Infection (UTI) dataset demonstrate that machine learning models can significantly outperform clinicians, reducing inappropriate therapy by over 25% at equivalent stewardship levels. Furthermore, direct policy optimization proves more sample-efficient in complex environments, offering a robust framework for multi-objective clinical decision support.
Introduction
In many medical scenarios, such as treating a Urinary Tract Infection (UTI), doctors face a "goldilocks" dilemma. They must choose between a narrow-spectrum antibiotic (targeted, lower risk of resistance) or a broad-spectrum antibiotic (higher efficacy, but contributes to "superbugs"). While precise lab results take days to process, historical datasets often contain these delayed results for every drug that could have been prescribed. This paper explores how to translate this "perfect hindsight" into real-time decision policies that navigate the trade-off between immediate patient health and long-term antibiotic stewardship.
Methodology
The authors evaluate three distinct approaches to learning a "Pareto frontier" of policies:
- Indirect Thresholding: Models predict the success probability for each drug; the policy selects the narrowest drug meeting a specific threshold.
- Indirect Expected Reward Maximization: Effectiveness and stewardship "costs" are combined into a single weighted reward score.
- Direct Policy Optimization: This method directly optimizes the decision rule using a multinomial deviance loss. By focusing on the decision boundary rather than modeling every nuance of patient health, this approach can be more robust and data-efficient.
Additionally, the authors introduce a Learning to Defer framework, allowing the model to identify cases where clinicians are likely to err and only intervening then to mitigate alert fatigue.
Experimental Results
The models were evaluated on a real-world UTI dataset of 15,806 specimens and a synthetic dataset designed to test decision boundary complexity.
Beating the Experts
All machine learning approaches achieved strictly better performance than clinicians. When matched to the same broad-spectrum usage rate as doctors (33.6%), the ML models reduced the rate of inappropriate therapy (IAT) by over 25% (dropping from 11.9% to ~8.9%).
Stewardship and Efficiency
The models demonstrated the ability to virtually eliminate broad-spectrum drug usage (reducing it to <1%) while still providing better patient outcomes than current clinical practice. Furthermore, synthetic tests confirmed that the Direct approach is significantly more sample-efficient when the underlying biology is complex but the optimal decision boundary is relatively simple.
Discussion and Limitations
A primary limitation is the "fully observed" assumption; many medical fields do not provide counterfactual outcomes for all possible paths. Additionally, antibiotic resistance is nonstationary; future work must address how to update these policies as bacterial resistance patterns evolve over time. Despite these challenges, the study provides a powerful framework for institutionalizing stewardship goals through data-driven policy learning.
Summary updated