Abstract
Drug synergy prediction is a challenging and important task in the treatment of complex diseases including cancer. In this manuscript, we present a novel unified Model, known as BAITSAO, for tasks related to drug synergy prediction with a unified pipeline to handle different datasets. We construct the training datasets for BAITSAO based on the context-enriched embeddings from Large Language Models for the initial representation of drugs and cell lines. After demonstrating the relevance of these embeddings, we pre-train BAITSAO with a large-scale drug synergy database under a multi-task learning framework with rigorous selections of tasks. We demonstrate the superiority of the model architecture and the pre-trained strategies of BAITSAO over other methods through comprehensive benchmark analysis. Moreover, we investigate the sensitivity of BAITSAO and illustrate its unique functions including new drug discoveries, drug combinations-gene interaction, and multi-drug synergy predictions.
Introduction
Treating complex diseases like cancer often requires drug combinations to overcome resistance and reduce toxicity. However, the number of possible combinations is astronomical—a "combinatorial explosion" that makes physical lab testing impossible for every pair, let alone triplets or larger cocktails. Existing computational models face major hurdles, including data silos caused by inconsistent naming conventions, noisy biological features (like gene expression), and poor generalization to unseen drugs or cell lines.
The BAITSAO Framework
To address these challenges, we introduce BAITSAO (a name derived from the Chinese term for collections of herbs and drugs). The model utilizes a three-stage pipeline designed for robustness and scalability:
- Context-Enriched Embeddings: Instead of relying solely on raw chemical structures (SMILES), BAITSAO uses GPT-3.5 to generate detailed text descriptions of drugs and cell lines based on biological prompts. These descriptions are converted into high-dimensional vectors. This "semantic" approach allows the model to capture functional similarities that raw structural data might miss.
- Multi-Task Learning (MTL): BAITSAO is pre-trained to solve multiple related problems simultaneously: predicting synergy scores (regression), identifying synergistic pairs (classification), and predicting single-drug inhibition levels. This shared knowledge base improves the model's overall predictive power.
- Task Selection & Weighting: We developed a "Help-Harm" matrix to ensure tasks complement each other and used Uncertainty Weighting (UW) to automatically balance the importance of different loss functions during training.
Performance and Generalization
BAITSAO was pre-trained on the DrugComb database, containing over 700,000 combinations. In benchmark tests against established models like DeepSynergy and DeepDDs, BAITSAO ranked best in three out of four major metrics (PCC, MSE, and ROC-AUC).
A significant advantage of BAITSAO is its "zero-shot" reasoning capability. Because the model understands the description of a drug rather than just its ID, it can predict the synergy of a drug it has never encountered in training simply by processing its functional description. Furthermore, the architecture is scalable; it can predict the synergy of three or more drugs by averaging their embeddings, a capability validated by experimental growth inhibition data for melanoma treatments.
Biological Insights and Drug Similarity
Our analysis reveals that text-based embeddings from LLMs serve as powerful biological proxies. Drugs with similar clinical functions cluster together in the embedding space, even if their chemical structures differ significantly. This allows BAITSAO to identify potential drug repurposing opportunities and discover similarities between compounds that traditional methods might overlook.
By employing SHAP (SHapley Additive exPlanations), we also identified specific genes in cell lines that drive predicted synergy, providing a bridge between AI predictions and interpretable molecular biology.
Conclusion
BAITSAO represents a significant step toward a foundation model for pharmacology. By leveraging the semantic power of LLMs and a robust multi-task learning framework, it overcomes the limitations of traditional drug synergy models. Future work will focus on incorporating more diverse data types, such as genome-wide association studies (GWAS) and granular single-cell data, to further refine its predictive accuracy for personalized medicine.
Summary updated