TL;DR
This paper introduces APSS, an automated system that helps hospital pharmacists identify inappropriate antimicrobial prescriptions. APSS goes beyond static rule-based systems by incorporating a machine learning module that learns new rules from user feedback, allowing it to adapt to local practices and improve over time.
What Problem Does This Paper Address?
Inappropriate prescribing of antimicrobials (ATM) is a significant global health concern, leading to increased healthcare costs, adverse drug effects, and the rise of antimicrobial resistance. While ATM stewardship programs are crucial for combating this, they require extensive manual review of clinical data by experts, which is often unsustainable due to resource limitations. Existing automated systems rely on predefined knowledge bases (KBs) of rules, but these often generate a high rate of false alerts, hindering their effectiveness. This is partly because it's challenging to capture the complex, experience-driven decision-making of physicians and the unique local practices of different hospitals within static rule sets.
Methods
The core of the system is APSS, an automated prescription surveillance system that utilizes a knowledge base (KB) of rules. What sets APSS apart is its Learning Module, which employs a supervised machine learning algorithm to discover new rules for classifying inappropriate prescriptions. This module combines instance-based learning and rule induction techniques.
Key methodological aspects include:
- Temporal Abstraction: Raw clinical data (like temperature readings) is transformed into meaningful "episodes" representing states that hold over time intervals. This allows the system to capture the temporal nature of ATM prescriptions.
- Instance-Based Learning: The system uses nearest neighbor classification, calculating distances between rules and sequences of patient data.
- Rule Induction: The algorithm learns classification rules, where the antecedent is a conjunction of propositions over time intervals, implying an inappropriate prescription as the consequent.
- Rule Refinement: The learning process iteratively modifies rules to increase the distance between "appropriate" and "inappropriate" classifications, aiming for more accurate and specific rules.
- J-measure: This metric is used to evaluate the quality of learned rules, balancing simplicity and predictive accuracy.
Datasets
The paper describes experiments using two datasets derived from real patient data at the Centre Hospitalier Universitaire de Sherbrooke (CHUS):
- METRO: A smaller dataset focused on metronidazole (an ATM often prescribed orally).
- TAZO: A larger dataset focused on piperacillin-tazobactam (TAZO), the most prescribed intravenous ATM at the center.
These datasets were partitioned into training and test sets. The paper does not introduce new benchmark datasets but uses real-world clinical data to evaluate the learning module.
Key Insights and Ideas
- Learning from False Alerts is Crucial: The high rate of false alerts in traditional systems is a major barrier. APSS's ability to learn from user feedback (e.g., overridden alerts) is key to overcoming this and improving precision.
- Temporal Dynamics are Essential: Antimicrobial prescribing is not a static decision. Capturing the evolution of a patient's condition over time is critical for accurate surveillance, and temporal abstraction is a powerful tool for this.
- Adaptability is Key for Real-World Deployment: Hospitals have unique practices. A system that can learn and adapt to these local nuances (rather than relying on a one-size-fits-all KB) is more likely to be effective and accepted.
- Machine Learning Can Uncover Unexpected Patterns: The learning module can identify clinically relevant patterns of inappropriate prescribing that might not be immediately obvious to human experts, leading to targeted interventions and improved stewardship.
Key Results
- Initial Deployment Success: APSS, even before the learning module, demonstrated significant impact, leading to a reduction in ATM consumption (13.5%) and expenditures (CAD 305,000) during its initial evaluation period.
- High Acceptance Rate: Prescribers and decision-makers showed universal appreciation for APSS, with a high acceptance rate (91%) for pharmacist recommendations.
- Learning Module Performance: The learning module (TIM) extracted accurate and sensitive rules for identifying "early switch therapy" (transitioning from IV to oral ATM).
- TIM vs. Other Algorithms: TIM demonstrated comparable or better recall and accuracy than instance-based learning (IBL) and classification rules (CRL) while being significantly faster than ARM (Association Rule Mining). TIM outperformed CRL by up to two orders of magnitude in speed.
- Rule Relevance: A significant portion (63%) of the rules learned by TIM were found to be clinically relevant by an expert.
Limitations and Future Work
- Imbalanced Data: The paper acknowledges that learning from imbalanced datasets (where inappropriate prescriptions are rare) is a significant challenge that has not been fully characterized for their algorithm. This is a key area for future work.
- Generalization to Other Hospitals: While the learning capability is designed for adaptability, further experimentation is needed before releasing it in deployed versions at new hospitals.
- Exploring Other Temporal Data Mining Techniques: The authors suggest integrating other temporal data mining methods to explore the vast amount of accumulated data and identify more interesting patterns.
- Broader Application: The potential to adapt APSS for monitoring other types of drugs beyond antimicrobials is mentioned as a future direction.
Summary updated