Contrastive Learning Drug Response Models from Natural Language Supervision

Kun Li, Xiuwen Gong, Jia Wu, Wenbin Hu, Mengran Li, Dawei Zhao

Abstract

Drug Response Prediction (DRP) is a cornerstone of phenotypic drug discovery, yet traditional regression-based models often struggle with zero-shot generalization. This failure stems from "disordered" internal representations where samples with distinct labels overlap in high-dimensional space. We propose CLDR (Contrastive Learning Drug Response), a framework that treats numerical IC50 labels as natural language descriptions. By aligning drug-cell features with text-based "captions" and a common-sense numerical knowledge graph, CLDR creates more distinct, ordered representations. Our framework consistently improves performance across various state-of-the-art architectures, with gains ranging from 7.8% to 31.4%, particularly in challenging scenarios involving unseen compounds.

The Challenge of Disordered Representations

Historically, DRP has been treated as a straightforward regression task, aiming to predict the IC50 value—the concentration required to inhibit 50% of biological activity. However, standard regression models often produce messy internal representations. In high-dimensional space, samples with different labels often aggregate into disordered clusters.

Visualization of feature distributions: Traditional methods lead to overlapping clusters, while CLDR produces scattered, ordered representations that are easier to differentiate.

As shown in the visualization above, traditional methods result in overlapping clusters that make different responses indistinguishable. This lack of structure makes it difficult for models to generalize to zero-shot conditions—situations where the model must predict the response for a drug it has never seen during training. In these cases, Pearson correlation coefficients can plummet significantly.

The CLDR Framework

CLDR moves away from pure regression toward a multimodal contrastive learning strategy. The architecture consists of several key components designed to provide richer supervision than a single scalar value.

The CLDR architecture: Drug and cell features are fused and aligned with text-based captions through contrastive learning, supported by a numerical knowledge graph.

Key Components:

  1. The Fusion Encoder: This module takes drug data (SMILES strings) and cell line genomics, fusing them into a feature vector. CLDR is flexible and can be wrapped around existing DRP architectures like GNNs or Transformers.
  2. The Caption Encoder: Instead of a raw number, the model converts IC50 values into natural language prompts (e.g., "The drug response value... is zero point five one three").
  3. Common-sense Numerical Knowledge Graph (CN-KG): To prevent "number blindness," the authors introduced a CN-KG. This graph explicitly defines mathematical relationships (e.g., "1 is less than 2"), constraining the text encoder to preserve the natural order and continuity of numbers.
  4. Contrastive Pre-training: The model maximizes similarity between drug-cell features and their corresponding text descriptions. This "scatters" the features in a way that reflects their actual response values.

Key Results and Insights

The framework was validated using the GDSC2 dataset, containing over 150,000 samples. By forcing the model to "describe" results in a high-dimensional anchor space, it learns a more discriminative feature space.

  • Performance Gains: Average performance increases ranged from 7.8% to 31.4% across six state-of-the-art models.
  • Zero-Shot Robustness: In the most difficult settings, CLDR-enhanced models showed massive improvements in Pearson and Spearman correlations.
  • Drug Screening Accuracy: In simulated screening tests, CLDR achieved a hit rate of ~32% on the first try, outperforming traditional methods by roughly 10%.

Limitations and Future Work

While effective, the model's performance peaks at a decimal precision of 0.01; higher granularity introduces stability trade-offs. Additionally, the inclusion of a text encoder and knowledge graph increases the parameter count. Future research will explore more complex prompt engineering and the integration of larger biological knowledge graphs to further enrich the supervision provided by the text.

Summary updated

BibTeX
@inproceedings{li2024contrastive,
  author = {Kun Li and Xiuwen Gong and Jia Wu and Wenbin Hu and Mengran Li and Dawei Zhao},
  title = {{Contrastive Learning Drug Response Models from Natural Language Supervision}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure