Stack: In-Context Learning of Single-Cell Biology

Mingze Dong, Abhinav Adduri, Dhruv Gautam, Christopher Carpenter, Rohan Shah, Chiara Ricci-Tam, Yuval Kluger, Dave P. Burke, Yusuf Husein Roohani2026 · DOI

TL;DR

This paper introduces STACK, a novel foundation model for single-cell biology that leverages the context of neighboring cells to generate more accurate and informative representations. STACK enables "in-context learning," allowing it to predict cellular responses to unseen conditions without retraining, and has been used to create Perturb Sapiens, a comprehensive atlas of perturbed human cells.

What Problem Does This Paper Address?

Single-cell RNA sequencing (scRNA-seq) generates vast amounts of data, promising deep insights into cellular diversity. However, existing computational models often treat cells in isolation, overlooking crucial contextual information that can improve measurement precision. This limitation hinders their ability to generalize to new biological conditions or tasks without extensive retraining. Specifically, current foundation models struggle with "zero-shot" learning (performing well on unseen data) and accurately predicting the effects of perturbations or variations in cellular states.

Methods

STACK employs a tabular transformer architecture that processes cells within their context. Key innovations include:

  • Gene Module Tokens: Instead of processing individual genes, STACK tokenizes gene expression into "gene module tokens," representing coherent subsets of gene variability.
  • Tabular Attention: This architecture uses multi-head attention (MHA) layers that operate both intra-cellularly (within a cell's tokens) and inter-cellularly (across tokens of cells within the same context).
  • Masked Gene Reconstruction: During pre-training, STACK learns by reconstructing masked gene expression values, leveraging the context of other cells in the set.
  • In-Context Learning (ICL): After pre-training, STACK is fine-tuned to perform "cell prompting." This allows it to predict how query cells would behave under a condition specified by "prompt" cells, enabling zero-shot prediction of perturbation effects or donor-specific states. This is achieved through a self-distillation procedure and a novel post-training objective.

Datasets

  • Pre-training: STACK was pre-trained on scBaseCount, a massive dataset of 149 million human single cells.
  • Post-training (ICL): For in-context learning, STACK was post-trained on a curated dataset of 55 million cells from CELLxGENE and the Parse PBMC 10M dataset, which includes cytokine perturbations.
  • Evaluation: The model was evaluated on numerous observational and perturbational datasets, including Kidney atlas, Brain atlas, Lymph node atlas, Lung atlas, Tabula Sapiens, OpenProblems (drug perturbations), Tahoe-100M, Parse-PBMC, and X-Atlas (Xaira).
  • Perturb Sapiens: This paper introduces Perturb Sapiens, a novel whole-organism atlas of perturbed cells generated using STACK, spanning 28 tissues, 40 cell classes, and 201 perturbations.

Key Insights and Ideas

  • Context is King: The paper strongly demonstrates that considering cells within their biological context significantly improves representation learning and downstream task performance, especially in zero-shot scenarios.
  • In-Context Learning for Biology: STACK pioneers in-context learning for single-cell biology, enabling models to adapt to new conditions and tasks at inference time without retraining, akin to large language models.
  • Perturb Sapiens as a Resource: The creation of Perturb Sapiens highlights STACK's capability to generate comprehensive, high-resolution atlases of cellular responses, providing a valuable resource for biological research.
  • Generalization Beyond Supervision: STACK's ability to generalize to unseen cell types, tissues, and perturbations without explicit labels or fine-tuning is a major step towards truly universal biological foundation models.

Key Results

  • Superior Zero-Shot Performance: STACK consistently outperforms baseline models (including zero-shot, fine-tuned, and trained-from-scratch models) on various downstream tasks, achieving state-of-the-art results in 28 out of 31 perturbation prediction benchmarks.
  • Enhanced Embeddings: STACK embeddings show substantial improvements in cell type classification and integration tasks, outperforming alternatives by significant margins (e.g., +139.4% in integration performance on Tabula Sapiens).
  • Accurate Perturbation Prediction: STACK accurately predicts perturbation effects, even in novel cell types and across different datasets, outperforming existing methods by approximately 100% on large-scale datasets.
  • Perturb Sapiens Atlas: The generated Perturb Sapiens atlas, covering 28 tissues and 201 perturbations, reveals realistic cellular responses and has been validated against in vitro data.

Limitations and Future Work

  • Human-Centric Model: The current STACK model is trained exclusively on human data. Extending it to multi-species applications would require modifications to the tokenization procedure to handle gene misalignment.
  • Rare Cell Types and Weak Effects: Model calibration for rare cell types and subtle perturbation effects remains an area for improvement.
  • Signaling Cascades: The complex, time-dependent nature of signaling cascades might complicate the interpretation of STACK predictions, especially for secondary perturbation effects.
  • Data Curation: While post-training focused on in vivo cell types, more sophisticated data curation and alignment schemes could further enhance generalization across diverse experimental settings.

Summary updated

Abstract

Single-cell transcriptomics offers the promise of measuring the diversity of cellular phenotypes across species, diseases, and other biological conditions. Recently, foundation models have emerged to identify this variation, yet most methods represent each cell independently, despite technical limitations that reduce measurement precision at the single-cell level. Here, we present Stack, a foundation model trained on 149 million uniformly preprocessed human single cells that leverages tabular attention to generate representations for each cell informed by the cells in its context. Stack offers substantial improvements for downstream tasks in the zero-shot setting compared to baselines, whether they are zero-shot, fine-tuned, or trained from scratch on the target dataset. Stack can perform in-context learning from unlabeled cells representing arbitrary conditions, such as a chemical perturbation or a different donor, and predict the effect of those conditions on a target cell population without requiring data-specific fine-tuning. We apply Stack to generate <i>Perturb Sapiens</i>, the first human whole-organism atlas of perturbed cells, spanning 28 tissues, 40 cell classes, and 201 perturbations. We validated subsets of <i>Perturb Sapiens</i> using <i>in vitro</i> stimulation profiles. Overall, Stack presents a new modeling framework where cells themselves act as guiding examples at inference time, unlocking general-purpose in-context learning capabilities for single-cell biology.
BibTeX
@misc{dong2026stack,
  author = {Mingze Dong and Abhinav Adduri and Dhruv Gautam and Christopher Carpenter and Rohan Shah and Chiara Ricci-Tam and Yuval Kluger and Dave P. Burke and Yusuf Husein Roohani},
  title = {{Stack: In-Context Learning of Single-Cell Biology}},
  year = {2026},
  doi = {10.64898/2026.01.09.698608},
}
Figure