Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting

Oleksandr Narykov, Yitan Zhu, Thomas S. Brettin, Yvonne A. Evrard, Alexander Partin, Fangfang Xia, Maulik Shukla, Priyanka Vasanthakumari, James H. Doroshow, Rick L. StevensBriefings in Bioinformatics 2025 · DOI

TL;DR

Drug response prediction (DRP) models struggle with imbalanced datasets, leading to biased predictions. This paper introduces a novel multi-objective optimization (MOO) loss function, MOORLE, that explicitly addresses this imbalance by balancing performance across different drugs. This approach significantly improves the generalization capabilities of deep learning models for drug discovery, particularly in challenging drug-blind scenarios.

What Problem Does This Paper Address?

Predicting how a patient will respond to a specific drug is crucial for personalized medicine and drug discovery. This involves understanding the complex interplay between a patient's genetic makeup (or cell line characteristics) and the drug's molecular properties. Machine learning (ML) models, especially deep learning (DL) models, have shown promise in this area by learning these intricate relationships from large datasets.

However, a significant challenge in drug response prediction (DRP) is data imbalance. This means that some drugs or cell lines are much more represented in the training data than others. This imbalance can lead to models that perform well on common drugs but poorly on rare ones, or models that are biased towards certain types of biological responses. This is particularly problematic in drug discovery, where the goal is to predict responses to novel compounds that are likely to be underrepresented in existing datasets. Existing methods often treat DRP as a single-objective optimization problem, aiming to minimize an overall error metric, which can mask poor performance on specific subgroups.

Methods

The core contribution of this paper is the development of a Multi-Objective Optimization Regularized by Loss Entropy (MOORLE) loss function. This approach reframes DRP as a multi-objective problem, aiming to optimize performance not just overall, but also for individual drugs.

Here's a breakdown of the key methodological aspects:

  • Multi-Objective Optimization (MOO): Instead of a single performance metric (like Mean Squared Error - MSE), MOORLE considers individual drug performance metrics (e.g., R-squared for each drug).
  • Entropy Regularization: To balance the performance across drugs, the paper introduces an entropy-based regularization term. This term encourages the model to have similar levels of error for each drug. The goal is to find a model that lies on the "Pareto front," meaning it offers a good trade-off between individual drug performance and overall performance.
  • MOORLE Loss Function: The proposed loss function combines the average of individual drug MSEs with the entropy regularization term. This allows researchers to control how much emphasis is placed on balancing performance across drugs (via a regularization coefficient, $\alpha$).
  • Model-Agnostic Approach: The MOORLE loss function is designed to be model-agnostic, meaning it can be integrated into various ML algorithms, including classical models and deep learning architectures.
  • Mixed Sampling Strategy: In addition to the novel loss function, the paper also explores a "mixed sampling" strategy for training data. This strategy aims to create more balanced mini-batches during training, further addressing data imbalance.

Datasets

The study utilizes two publicly available DRP datasets:

  • Cancer Therapeutics Response Portal (CTRP): This dataset contains a large number of drug response experiments (254,566) across 812 cell lines and 495 ligands.
  • Cancer Cell Line Encyclopedia (CCLE): This dataset is smaller, with 8950 experiments based on 474 unique cell lines and 24 drugs.

The authors note that CCLE exhibits a "slight imbalance" while CTRP shows a "significant imbalance," making them suitable for evaluating their proposed methods.

Key Insights and Ideas

  • Data imbalance is a critical bottleneck: The paper strongly emphasizes that ignoring data imbalance leads to unreliable DRP models, especially for drug discovery where predicting responses to novel compounds is paramount.
  • MOO offers fine-grained control: By treating each drug as a separate objective, MOO allows for a more nuanced approach to model optimization, moving beyond a single, potentially misleading, aggregate performance metric.
  • Entropy regularization promotes fairness: The entropy-based component of MOORLE is key to ensuring that the model doesn't neglect underrepresented drugs, leading to more equitable performance across the board.
  • Drug-blind evaluation is crucial for drug discovery: The paper highlights the importance of drug-blind splits to simulate real-world virtual screening scenarios, where models must predict responses to drugs not seen during training.
  • Mixed sampling can complement loss function improvements: While the MOORLE loss function is the primary innovation, the mixed sampling strategy also shows promise in further mitigating imbalance issues.

Key Results

The study demonstrates that the MOORLE loss function, particularly when combined with the mixed sampling strategy, significantly improves DRP model performance, especially in drug-blind evaluation scenarios.

  • Improved Generalization: In drug-blind evaluations (virtual screening), the MOORLE loss function consistently led to better performance metrics (e.g., higher R2, lower MSE/MAE) compared to the standard MSE loss, particularly on the imbalanced CTRP dataset. For instance, on the CTRP dataset, MOORLE with sequential sampling achieved a higher R2 (0.482) compared to MSE with sequential sampling (0.287).
  • Reduced Variability: The MOORLE loss function also helped reduce the standard deviation of performance metrics, indicating more robust and consistent predictions.
  • Ablation Study Findings: The ablation study revealed that the MOORLE loss function was the primary driver of performance improvements, especially in drug-blind settings. The mixed sampling strategy showed benefits on the CCLE dataset but was less impactful on the highly imbalanced CTRP dataset.
  • Model-Agnostic Utility: The MOORLE loss function showed utility when integrated into various models, including DeepTTA and GraphDRP, suggesting its broad applicability.

Limitations and Future Work

  • Hyperparameter Tuning: The performance of the MOORLE loss function is dependent on the regularization coefficient ($\alpha$). Finding the optimal $\alpha$ requires nested cross-validation, which is computationally demanding. The authors suggest this as an area for future work, aiming to develop more efficient hyperparameter optimization strategies.
  • Computational Cost: Nested cross-validation for hyperparameter tuning significantly increases computational runtime.
  • Real-world Applicability of Splits: While drug-blind splits approximate virtual screening, the authors acknowledge that performance metrics from these splits may not directly translate to real-world effectiveness due to inherent chemical property variations in compounds across training, validation, and test sets.
  • Future Work: The authors propose extending this work to other imbalance types (e.g., sex, age), exploring different aggregation methods for subgroup losses, and conducting large-scale analyses with more community models and extensive hyperparameter optimization. They also suggest investigating the MOORLE loss function's potential for other DRP applications like drug repurposing and personalized medicine.

Summary updated

BibTeX
@article{narykov2025data,
  author = {Oleksandr Narykov and Yitan Zhu and Thomas S. Brettin and Yvonne A. Evrard and Alexander Partin and Fangfang Xia and Maulik Shukla and Priyanka Vasanthakumari and James H. Doroshow and Rick L. Stevens},
  title = {{Data imbalance in drug response prediction: multi-objective optimization approach in deep learning setting}},
  journal = {Briefings in Bioinformatics},
  year = {2025},
  doi = {10.1093/bib/bbaf134},
}
Figure