TL;DR
This paper introduces scOTM, a novel deep learning framework that accurately predicts how individual cells will respond to perturbations (like drugs) using large language models (LLMs) and optimal transport. scOTM excels at generalizing to new cell types and handling unpaired data, offering a more flexible and biologically informed approach than previous methods.
What Problem Does This Paper Address?
Understanding how cells respond to external stimuli, such as drugs or cytokines, is crucial for advancing medicine, from deciphering disease mechanisms to predicting drug efficacy and side effects. While single-cell RNA sequencing (scRNA-seq) provides detailed snapshots of cellular states, capturing dynamic responses to perturbations at scale is challenging and expensive. Existing computational models often struggle with:
- Unpaired Data: They frequently lack direct one-to-one matches between control and perturbed cells, making it hard to learn precise response mappings.
- Rigid Assumptions: Many models force latent cell representations into a standard normal distribution, limiting their ability to capture complex biological variations.
- Cell-Type Specificity: Naive averaging strategies can overlook the unique responses of different cell types and the heterogeneity within them.
- Generalization: Models often perform poorly when applied to cell types they haven't seen during training.
This paper aims to overcome these limitations by developing a framework that can predict perturbation responses from unpaired data and generalize effectively to unseen cell types.
Methods
scOTM is a generative model built upon a Variational Autoencoder (VAE) architecture, enhanced with two key innovations:
- Large Language Model (LLM) Embeddings: scOTM integrates prior biological knowledge by using LLMs.
- Single-cell LLMs (e.g., scGPT): These are used to generate embeddings for the cellular transcriptional profiles, providing a biologically informed initialization.
-
Molecular LLMs (e.g., ChemBERTa, ESM2): These are used to generate embeddings for the perturbations themselves (e.g., drugs), capturing their semantic meaning. These embeddings are combined with the raw gene expression data before being fed into the encoder.
-
Maximum Mean Discrepancy (MMD) Regularization: Instead of the standard KL divergence, scOTM uses MMD to align the latent space distribution with a standard normal prior. MMD is a non-parametric method that compares distributions without needing explicit probability density functions, allowing for more flexible latent space modeling.
- Optimal Transport (OT): To model the correspondence between control and perturbed cell states, scOTM employs OT. This allows for a principled and interpretable mapping between the distributions of control and stimulated cells in the latent space, effectively capturing the shifts in gene expression. This is particularly useful for unpaired data.
The framework consists of a training phase (encoding, MMD regularization, and decoding) and a prediction phase (using OT to align latent representations and then decoding to predict gene expression).
Datasets
The study utilizes publicly available datasets for evaluation:
- Human Peripheral Blood Mononuclear Cell (PBMC) dataset: This dataset, from Kang et al., was used for evaluating generalization across different cell types.
- Open Problems-Single-Cell Perturbations competition dataset (Kaggle): This dataset, specifically the subset treated with belinostat and DMSO controls, was used for a comprehensive evaluation of the model's performance.
Key Insights and Ideas
- LLMs as Biological Priors: Integrating LLM-derived embeddings for both cells and perturbations significantly enhances the model's ability to capture biological context and semantic shifts, leading to better predictive performance.
- MMD for Flexibility: Moving away from rigid KL divergence to MMD for latent space regularization allows for more expressive and biologically relevant embeddings, especially in complex systems.
- Optimal Transport for Unpaired Data: OT provides a robust mechanism to learn correspondences between control and perturbed states even when direct cell-to-cell pairings are unavailable, a common challenge in perturbation studies.
- Generalization is Key: The framework is explicitly designed and evaluated for its ability to generalize to unseen cell types, a critical requirement for real-world applications.
Key Results
- Superior Prediction Accuracy: scOTM consistently outperformed state-of-the-art methods (scPRAM, scGen, scVIDR, trVAE, CellOT) in predicting whole-transcriptome responses across seven cell types, achieving an average R² of 94.05% for gene expression mean and 77.5% for variance.
- Accurate DEG Identification: scOTM identified a substantially higher number of differentially expressed genes (DEGs) that overlapped with ground truth, particularly in challenging cell populations like dendritic cells and CD14+ monocytes.
- Robust Generalization: The model demonstrated strong generalization capabilities to unseen cell types.
- Interpretability: Visualizations of scOTM's learned latent space showed clear separation between cell types and perturbation conditions, indicating that the model captures biologically meaningful patterns.
- Data Efficiency: scOTM maintained strong predictive performance even when trained on as little as 10% of the data, highlighting its robustness and scalability.
Limitations and Future Work
- LLM Bias: The reliance on LLMs trained predominantly on human single-cell data might limit applicability to non-human species.
- Computational Resources: Scaling scOTM to very large atlases (tens of millions of cells) may require substantial computational resources and optimization strategies.
- Future Directions:
- Integrating batch-effect correction modules for enhanced cross-dataset generalization.
- Extending the framework to multimodal single-cell data (e.g., ATAC-seq, CITE-seq) for a more comprehensive view of cellular states.
Summary updated