Predicting cellular responses to perturbation across diverse contexts with State

Abhinav Adduri, Dhruv Gautam, Beatrice Bevilacqua, Alishba Imran, Reena Shah, Mohsen Naghipourfar, Noam Teyssier, Rajesh Ilango, Sanjay Nagaraj, Chiara Ricci-Tam, Chris Carpenter, Vishvak Subramanyam, Aidan Winters, Mingze Dong, Sravya Tirukkovalur, Jeremy A. Sullivan, Brian S. Plosky, Basak Eraslan, Nicholas D. Youngblut, Jure Leskovec, Luke A. Gilbert, Silvana Konermann, Patrick D. Hsu, Alexander Dobin, Donald S. Burke, Hani Goodarzi

Abstract

Cellular responses to perturbations are a cornerstone for understanding biological mechanisms and selecting drug targets. While machine learning models offer tremendous potential for predicting perturbation effects, they currently struggle to generalize to unobserved cellular contexts. Here, we introduce STATE, a transformer model that predicts perturbation effects while accounting for cellular heterogeneity within and across experiments. STATE predicts perturbation effects across sets of cells and is trained using gene expression data from over 100 million perturbed cells. STATE improved discrimination of effects on large datasets by more than 30% and identified differentially expressed genes across genetic, signaling and chemical perturbations with significantly improved accuracy. Using its cell embedding trained on observational data from 167 million cells, STATE identified strong perturbations in novel cellular contexts where no perturbations were observed during training. We further introduce Cell-Eval, a comprehensive evaluation framework that highlights STATE’s ability to detect cell type-specific perturbation responses, such as cell survival. Overall, the performance and flexibility of STATE sets the stage for scaling the development of virtual cell models.

Introduction

In drug discovery and genetics, predicting how a specific "perturbation"—such as a novel drug or a CRISPR gene knockout—will affect a cell remains a fundamental challenge. Cells are inherently diverse; a lung cell may react to a therapeutic agent quite differently than a heart cell. Creating "virtual cell" models requires overcoming two major hurdles: biological heterogeneity and technical noise. Because single-cell sequencing is destructive, we cannot observe the "before and after" of the same individual cell. Furthermore, data from disparate laboratories often contain "batch effects" that obscure true biological signals. Current deep learning models frequently fail to outperform simple linear baselines when generalizing to new cellular contexts.

Methodology: The STATE Framework

The authors propose STATE, a multi-scale transformer architecture designed to navigate the complex space of cell states. STATE consists of two complementary modules:

  1. State Embedding (SE) Model: A foundation model pretrained on 167 million human cells to learn a universal representation of cellular states. It employs a "dual-axis" loss—predicting expression across genes within a cell and across cells for a single gene—to filter technical noise and capture true biological variation.
  2. State Transition (ST) Model: A transformer that operates on sets of cells rather than individuals. By modeling populations as distributions, ST accounts for the inherent diversity of the starting state. It utilizes Maximum Mean Discrepancy (MMD) as a loss function to ensure predicted perturbed populations match the statistical distributions of real-world observations.
Overview of the STATE training pipeline, from learning across heterogeneous datasets to simulating perturbation experiments.

The study leverages an unprecedented scale of data, including over 100 million perturbed cells across 70 contexts. Key benchmarks include the Tahoe-100M dataset for chemical perturbations and the Replogle-Nadig dataset for genome-scale genetic perturbations. The authors also introduced Cell-Eval, a framework focusing on biologically meaningful metrics like Differentially Expressed (DE) gene overlap and perturbation effect size.

Key Insights: Population-Level Modeling

A central innovation in STATE is the shift from individual-cell prediction to population-level modeling. The transformer's self-attention mechanism allows the model to "see" the heterogeneity within a group, leading to more realistic and diverse predicted responses. Furthermore, the authors provide a mathematical proof connecting STATE to Optimal Transport (OT) theory, showing it learns the most efficient "path" from an unperturbed to a perturbed state without rigid mathematical assumptions.

Results: Performance and Scaling

STATE demonstrates significant improvements over existing state-of-the-art models like scGPT and CPA:

  • Superior Discrimination: STATE improved the ability to distinguish between different perturbation effects by over 50%.
  • DE Gene Accuracy: It identified true differentially expressed genes with over 2-fold accuracy across genetic, signaling, and chemical datasets.
  • Phenotype Prediction: STATE-predicted gene expression profiles were used to predict cell survival (fitness) with a Pearson correlation of 0.52, doubling the performance of baseline models.
The impact of cell set size on validation loss, illustrating the scaling laws where larger sets lead to significantly improved prediction performance.

The model also exhibits clear scaling laws: as the size of the cell sets used during training increases, the validation loss drops significantly. This suggests that the model's performance will continue to improve with even larger datasets and set sizes.

Discussion and Future Work

While STATE generalizes well to new cell lines within known tissues, future work will explore its performance on entirely held-out datasets with no related cell contexts. Additionally, extending the model to predict combinatorial effects (e.g., drug synergies) and incorporating other data modalities like proteomics or cell morphology are logical next steps toward a holistic virtual cell model.

Summary updated

Abstract

Abstract Cellular responses to perturbations are a cornerstone for understanding biological mechanisms and selecting drug targets. While machine learning models offer tremendous potential for predicting perturbation effects, they currently struggle to generalize to unobserved cellular contexts. Here, we introduce S tate , a transformer model that predicts perturbation effects while accounting for cellular heterogeneity within and across experiments. S tate predicts perturbation effects across sets of cells and is trained using gene expression data from over 100 million perturbed cells. S tate improved discrimination of effects on large datasets by more than 30% and identified differentially expressed genes across genetic, signaling and chemical perturbations with significantly improved accuracy. Using its cell embedding trained on observational data from 167 million cells, S tate identified strong perturbations in novel cellular contexts where no perturbations were observed during training. We further introduce Cell-Eval, a comprehensive evaluation framework that highlights S tate ’s ability to detect cell type-specific perturbation responses, such as cell survival. Overall, the performance and flexibility of S tate sets the stage for scaling the development of virtual cell models.
BibTeX
@inproceedings{adduri2025predicting,
  author = {Abhinav Adduri and Dhruv Gautam and Beatrice Bevilacqua and Alishba Imran and Reena Shah and Mohsen Naghipourfar and Noam Teyssier and Rajesh Ilango and Sanjay Nagaraj and Chiara Ricci-Tam and Chris Carpenter and Vishvak Subramanyam and Aidan Winters and Mingze Dong and Sravya Tirukkovalur and Jeremy A. Sullivan and Brian S. Plosky and Basak Eraslan and Nicholas D. Youngblut and Jure Leskovec and Luke A. Gilbert and Silvana Konermann and Patrick D. Hsu and Alexander Dobin and Donald S. Burke and Hani Goodarzi and Hani Goodarzi},
  title = {{Predicting cellular responses to perturbation across diverse contexts with State}},
  booktitle = {FM4LS @ NeurIPS},
  year = {2025},
  url = {https://openreview.net/forum?id=QOMkmXVyiv},
}
Figure