TL;DR
Machine learning (ML) shows significant promise for predicting drug-induced cardiotoxicity, a major hurdle in drug development. This scoping review of 25 studies reveals that while ML models, particularly Support Vector Machines (SVM) and Random Forests (RF), achieve promising predictive performance, challenges remain in data standardization, external validation, and integrating diverse data sources for robust and generalizable predictions.
What Problem Does This Paper Address?
Drug-induced cardiotoxicity is a serious concern, leading to drug withdrawals and posing a significant risk to patient safety. While some ML models have focused on specific mechanisms like hERG channel inhibition, there's a need to predict a broader range of cardiotoxic outcomes. This paper addresses the challenge of systematically reviewing and understanding the current landscape of ML applications for predicting various human drug-induced cardiotoxicity outcomes, identifying trends, and highlighting areas for improvement.
Methods
This study is a scoping review, meaning it systematically searched and synthesized existing literature on ML models for predicting drug-induced cardiotoxicity. The authors employed a structured framework, adhering to PRISMA-ScR guidelines, to identify relevant studies. They searched major databases (PubMed, EMBASE, SCOPUS, Web of Science) using predefined keywords. Data extraction focused on databases used, features, algorithms, and performance metrics, categorized by evaluation type (training, testing, cross-validation, external validation). A quality and risk-of-bias assessment was also conducted for the included studies.
Datasets
The studies reviewed utilized various databases for cardiotoxicity information, with the most common being:
- SIDER (Side Effect Resource): Used in 9 out of 25 studies.
- CredibleMeds (formerly AZCERT): Used in 6 studies.
- DICTrank (Drug-Induced Cardiotoxicity Rank): Used in 3 studies.
- OMOP (Observational Medical Outcomes Partnership): Used in 3 studies.
- Micromedex: Used in 3 studies.
These databases provide information on adverse drug reactions, drug-induced cardiotoxicity, or curated evidence related to specific cardiotoxic events like TdP. No new benchmark datasets were introduced in this review; it focused on synthesizing findings from existing studies.
Key Insights and Ideas
- Broadening the Scope: While hERG inhibition has been a major focus, ML is increasingly being applied to predict a wider array of cardiotoxic outcomes, including arrhythmia, cardiac failure, heart block, hypertension, and myocardial infarction.
- Data Source Limitations: SIDER, a common data source, has limitations including potential labeling bias (treating unmentioned drug-adverse event pairs as negative) and being a static database. Newer resources like DICTrank and DIQTA are emerging to address these issues.
- Feature Diversity: Models utilize a range of features, from molecular descriptors and fingerprints to biological and functional data like target proteins and transcriptomics. However, some studies show strong performance using only molecular features.
- Algorithm Dominance: SVM and RF remain the workhorses for cardiotoxicity prediction, demonstrating robust performance. However, there's a growing interest in ensemble methods and deep learning architectures.
- Performance Variability: While ML models show promise, there's substantial variability in predictive performance across different cardiotoxicity outcomes. Arrhythmia and hypertension tend to show more stable results compared to cardiac failure and general cardiotoxicity.
- The Crucial Role of External Validation: Many studies reported high performance on training or internal validation sets, but external validation is critical for assessing true generalizability. The review highlights that only a subset of studies conducted robust external validation.
Key Results
- 25 Studies Included: The review analyzed 25 studies that met the inclusion criteria.
- Promising Performance Metrics: Several studies reported externally validated Area Under the Receiver Operating Characteristic Curve (AUC-ROC) values above 0.70 and classification accuracy exceeding 0.75 for various cardiotoxicity outcomes. For example, some models achieved AUC-ROC values up to 0.898.
- Algorithm Performance: SVM and RF were frequently used and often achieved strong results. For instance, some RF models achieved accuracy around 0.838 with high sensitivity and specificity.
- Outcome-Specific Performance: Models tended to perform better for arrhythmia and hypertension compared to cardiac failure and general cardiotoxicity, though statistical significance was not always found.
- Quality Concerns: A significant number of studies had limitations in areas like class imbalance handling, external validation rigor, confusion matrix reporting, and code availability, indicating potential risks of overestimating performance.
Limitations and Future Work
Limitations:
- Inconsistent Reporting: Many studies did not report essential performance metrics (e.g., TP, FP, TN, FN), hindering quantitative synthesis and direct comparison.
- Heterogeneity: Significant variation in datasets, feature types, and algorithms made direct comparison of models challenging.
- Data Quality: Most cardiotoxicity resources lack confirmed true negative labels, leading to potential labeling noise and inflated specificity.
- Limited External Validation: A substantial number of studies relied on internal validation or cross-validation, which may not accurately reflect real-world performance.
Future Work:
- Larger, Up-to-Date Datasets: Development of more comprehensive and current datasets is crucial.
- Systematic External Validation: Prioritizing rigorous external validation is essential for robust model assessment.
- Integration of Diverse Features: Incorporating a wider range of biologically meaningful features, including target proteins and transcriptomic data, can improve model generalizability.
- Mechanism-Informed Models: Developing models that incorporate mechanistic insights into drug-induced cardiotoxicity pathways can enhance interpretability and predictive power.
- Personalized Prediction: Future research could explore personalized ML models using EHRs and multi-omic data for individual-level cardiotoxicity risk prediction.
- Regulatory Alignment: Aligning ML validation strategies with regulatory frameworks (e.g., FDA guidance) is important for clinical translation.
Summary updated