Abstract
Drug synergy prediction is a challenging and important task in the treatment of complex diseases including cancer. In this manuscript, we present a unified Model, known as BAITSAO, for tasks related to drug synergy prediction with a unified pipeline to handle different datasets. We construct the training datasets for BAITSAO based on the context-enriched embeddings from Large Language Models for the initial representation of drugs and cell lines. After demonstrating the relevance of these embeddings, we pre-train BAITSAO with a large-scale drug synergy database under a multi-task learning framework with rigorous selections of tasks. We demonstrate the superiority of the model architecture and the pre-trained strategies of BAITSAO over other methods through comprehensive benchmark analysis. Moreover, we investigate the sensitivity of BAITSAO and illustrate its promising functions including drug discoveries, drug combinations-gene interaction, and multi-drug synergy predictions.
Introduction
In pharmacology, the concept of "1 + 1 > 2"—where a combination of drugs is more effective than the sum of its individual parts—is the ultimate goal of therapy. Identifying these synergistic drug pairs is critical for treating complex diseases like cancer and HIV, as they can enhance efficacy while reducing side effects and drug resistance. However, the experimental search space is astronomical, making it impossible to test every possible combination in a lab.
Existing machine learning models for synergy prediction face significant hurdles, including data heterogeneity across different naming conventions, poor generalization to "unseen" drugs, and a reliance on structural features (like SMILES strings) that often miss the broader biological context of how a drug functions in a living system.
Methods: LLM-Enriched Embeddings and Multi-Task Learning
To address these challenges, we introduce BAITSAO, a unified computational framework that treats drug synergy analysis as a multi-modal, multi-task problem.
Context-Enriched Embeddings
Instead of relying solely on chemical structures, BAITSAO leverages Large Language Models (LLMs) to generate detailed textual descriptions of drugs (e.g., mechanism of action, indications) and cell lines. These descriptions are converted into high-dimensional, context-enriched embeddings. This approach allows the model to "understand" the biological role and functional context of the inputs, acting as a bridge between chemistry and biology.
Multi-Task Learning (MTL) Framework
BAITSAO is trained simultaneously on four related tasks: predicting continuous Loewe synergy scores, binary synergy classification, and individual drug inhibition levels. The architecture utilizes data-specific layers that feed into shared layers, followed by task-specific output layers. To balance these objectives, the model employs a learnable uncertainty weighting mechanism that prevents any single task from dominating the training process.
Key Results and Insights
BAITSAO demonstrates state-of-the-art performance across major benchmarks, including DrugComb, DeepSynergy, and MARSY. Key findings include:
- Performance Boost: The transition from single-task to multi-task learning improved classification accuracy by up to 9.57%.
- Semantic Accuracy: LLM-generated embeddings showed high correlation (PCC ≥ 0.76) with curated databases like DrugBank, proving they are biologically grounded.
- Zero-Shot Capability: Because it uses semantic embeddings, BAITSAO can perform "zero-shot" inference for novel drugs simply by generating a text description for them.
- Explainability: Using SHAP (SHapley Additive exPlanations), we identified specific genes driving synergy predictions, providing a "white-box" look into the model's decision-making.
- Multi-Drug Synergy: The model successfully extended predictions to "tri-drug" combinations, identifying synergistic triplets supported by independent experimental literature.
Discussion and Future Directions
While BAITSAO excels at generalization, it still faces a "cold start" problem for entirely new synthetic compounds with no existing literature. Future work will focus on integrating real-time, patient-specific genomic data to move toward true personalized medicine. Additionally, as LLMs evolve, version tracking of embeddings will be essential to ensure the reproducibility of AI-driven drug discovery pipelines.
Summary updated