OCCAM: Improved Single-Cell Representation Learning and Perturbation Effect Prediction

Ihab Bendidi, Kian Kenyon-Dean, Alisandra K. Denton, Auguste Genovesio, Emmanuel NoutahiarXiv 2026 · arXiv

TL;DR

Occam is a streamlined multilayer perceptron (MLP) framework that outperforms complex transformer-based "foundation models" in learning single-cell representations and predicting genetic perturbation effects. By integrating denoising-based learning with biological priors like protein sequences and gene networks, it sets a new bar for generalizing to unseen cell types and complex multi-gene interactions.

What Problem Does This Paper Address?

Understanding how cells react to genetic changes is fundamental to drug discovery and basic biology, but the sheer number of possible experiments—thousands of genes across hundreds of cell types—makes exhaustive testing impossible. Researchers need computational models that can generalize: if we know how a gene affects a liver cell, can we predict its effect on a T-cell? Can we predict what happens when we knock out two genes at once?

Current state-of-the-art models often rely on massive Transformer architectures. However, these models are computationally expensive and don't always outperform simple linear baselines. The authors of Occam set out to prove that a well-designed, simpler architecture can achieve superior results by better leveraging biological context and robust training objectives.

Methods

Occam (named after Occam's Razor, favoring simplicity) uses a two-stage approach based on Multilayer Perceptrons (MLPs):

  1. Representation Learning via Denoising: The model first learns to "understand" a cell's state by reconstructing gene expression profiles. It uses a novel "swap-in denoising" task where some gene values are replaced with those from another cell, forcing the encoder to learn robust gene-gene relationships to recover the original profile.
  2. Perturbation-Effect Prediction: To predict the result of a genetic intervention, Occam combines the learned cell state with external biological knowledge. It uses gene interaction networks (GO, STRING), text descriptions (GenePT), and protein sequences (ESM2) to characterize the target gene.
  3. FiLM Modulation: These external features are fused and used to modulate the cell's internal representation through Feature-wise Linear Modulation (FiLM), effectively "shifting" the cell's state to its predicted perturbed state.
  4. Compositional Logic: For double perturbations, Occam applies its single-gene predictor twice in a sequence and averages the results from both possible orders, allowing it to capture complex, non-linear interactions between genes.
Occam's two-stage training pipeline: representation learning through swap-in denoising (left) and perturbation-effect prediction using biological priors and FiLM modulation (right).

Datasets

  • DiverseRNA: A massive pretraining corpus of ~1.4 million RNA-seq profiles from nine different datasets.
  • RHJ & Norman: Large-scale CRISPR perturbation screens (interference and activation) used for training and testing generalization.
  • Virtual Cell Challenge H1: A human embryonic stem-cell dataset used for "zero-shot" evaluation in entirely unseen biological contexts.
  • scIB Benchmarks: Standard datasets (Pancreas, PBMC) used to evaluate how well the model integrates data across different batches while preserving cell-type identity.

Key Insights and Ideas

  • The Power of Simplicity: The most striking finding is that simple MLPs can beat complex Transformers in this domain. This suggests that for gene expression data, the right training objective (like denoising) and biological priors are more important than architectural complexity.
  • Biological Priors are Essential: By including protein sequences and gene networks, Occam gains a "biological intuition" that allows it to predict effects for genes it hasn't seen in specific cell types before.
  • Better than "Sum of Parts": Most simple models predict double perturbations by just adding the effects of two single perturbations. Occam's sequential prediction approach captures the "synergy" (or lack thereof) between genes much more accurately.
  • Denoising as a Foundation: The "swap-in" denoising task proved exceptionally good at creating representations that are both robust to technical noise (batch effects) and sensitive to biological changes (perturbations).

Key Results

  • State-of-the-Art Representations: Occam achieved the highest aggregate scores for both perturbation information and cell-type preservation, outperforming models like scGPT and scVI.
  • Superior Generalization: In "few-shot" tests (predicting a known perturbation in a new cell line), Occam significantly outperformed four major retrained competitors, including TxPert and STATE.
  • Double Perturbation Accuracy: Occam reduced expression error in double-perturbation scenarios compared to both complex models and the standard "sum of singles" baseline.
  • Zero-Shot Performance: In completely held-out cell lines and assays, Occam remained the most effective model at distinguishing between different perturbations without any fine-tuning.
Table 1: Cell-representation quality by dataset and overall. Trained methods in the first two groups share the DiverseRNA corpus; public checkpoints use much larger corpora. Bold and under- line mark the best and second-best mean, respectively, in each column across all groups.

Limitations and Future Work

  • Fixed Gene Set: The current model uses a fixed vocabulary of genes. Applying it to datasets with different gene sets requires matching gene IDs and zero-filling missing data.
  • Population-Level Focus: While the model takes individual cells as input, its evaluations primarily focus on average population responses rather than predicting the unique trajectory of a single specific cell.
  • The "Expression Level" Hurdle: Like all current models, Occam still struggles to predict absolute expression levels with high accuracy in entirely new biological contexts; predicting the relative change (direction of change) remains much easier.
  • Future Directions: The authors suggest expanding the framework to handle more than two simultaneous perturbations and moving beyond genetic interventions to predict the effects of chemical drugs.

Summary updated

Abstract

Predicting the effects of genetic perturbations requires models that generalize beyond observed cell contexts and gene combinations. We introduce Occam, a multilayer perceptron (MLP) framework for representation learning and perturbation-effect prediction. Occam combines cell representations learned through denoising with gene interaction, text, and protein-sequence features. It achieves the highest aggregate representation scores for perturbation information and cell-type preservation with batch integration. In cross-context few-shot prediction, a perturbation and a cell context are each observed during training, but never together. Occam better captures the measured response patterns for these unobserved combinations than four retrained competitors or direct transfer of the same perturbation's measured effects from other training cell lines. Without fine-tuning, Occam best distinguishes perturbations among the evaluated models and simple baselines in cell lines held out from predictor training, across seen and unseen assays. Predicting expression levels accurately in these settings remains challenging. For double perturbations, which target two genes simultaneously, Occam composes learned single-gene responses to better capture the combined response and reduce expression error compared with competing models and summed single-gene effects. These gains extend to both strongly additive and weakly additive responses. Together, these results support simple MLPs for cell representation learning and perturbation-effect prediction across contexts and gene combinations.
BibTeX
@misc{bendidi2026occam,
  author = {Ihab Bendidi and Kian Kenyon-Dean and Alisandra K. Denton and Auguste Genovesio and Emmanuel Noutahi},
  title = {{OCCAM: Improved Single-Cell Representation Learning and Perturbation Effect Prediction}},
  year = {2026},
  eprint = {2611.21959},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21959},
}
Figure