Choosing Perturb-seq Experiments by Uncertainty Over Regulons, Not Genes

Large CRISPR screens read out thousands of genes in single cells after switching individual genes up or down. These screens are expensive, so a natural goal is to run them adaptively. A model of the cell's regulatory wiring picks the next gene to perturb, the screen runs, the model updates, and the loop repeats. In simulation this works well. Active causal discovery, meaning algorithms that choose interventions to pin down a cause-and-effect graph, beats random choice by large margins on synthetic graphs. On real single-cell screens it mostly does not. Standard uncertainty-driven selection often loses to choosing perturbations at random. Our hypothesis is that the failure comes from where the uncertainty is measured, not from the idea of choosing experiments. Per-gene readouts are dominated by dropout and sampling noise, so a model's disagreement about individual gene responses mostly tracks noise, not unresolved wiring.

We propose to measure uncertainty, and choose experiments, in the space of regulons. A regulon is a transcription factor together with the set of genes it controls. Averaging a perturbation's effect over a regulon's many targets shrinks the noise roughly in proportion to the regulon's size, which is the logic biologists already use to infer transcription factor activity. We build a small, mostly linear module-level causal model and estimate its uncertainty by resampling cells. We then select perturbations by expected information about the transcription-factor-to-module graph. The central test is a matched comparison: the same model and the same acquisition rule, scored in gene space versus module space, run on retrospective replays of public Perturb-seq screens. If module-space scoring does not beat gene-space scoring and random selection, the claim is wrong.

Core Claim and Positioning

The claim is narrow. Choosing perturbations to reduce uncertainty over a regulon-level causal graph recovers that graph with fewer experiments than random choice, gene-level information-based choice, or expression-prediction uncertainty. The reason is that regulon pooling turns a noise-dominated acquisition signal into a structure-dominated one.

The idea is not a new acquisition rule. The rule is essentially mutual information between an experiment's outcome and the graph, as in CBED. CBED selects both which variable to intervene on and at what dose, using a Bayesian graph posterior. It was validated on synthetic graphs and the simulated DREAM benchmark, and posterior inference stalls beyond 50 to 100 variables. Licence adds cheap simulators as lower-fidelity experiments. CORE learns a reinforcement-learning policy, but only up to 10 variables. All three treat individual measured variables as graph nodes, and none touches real single-cell data. We reuse their information-gain logic but change the variables it is computed over.

Two papers supply the motivating evidence. LLM-guided Perturb-seq acquisition runs budgeted active learning on the Replogle K562 and RPE1 screens. It finds that expression-uncertainty selection (the BALD criterion) underperforms random selection in several settings, and that LLM re-ranking helps but is volatile across cell lines. Its target is expression error, not graph structure. PerturbCausal shows that good expression prediction does not imply causal recovery. Deep virtual cell models inflate effect sizes about 23-fold, none beats a sparse linear ElasticNet comparator on causal-recovery F1, and even the reference graphs inferred from real screens are not robustly above chance against TRRUST.

Two other papers supply method rather than motivation. PerturbODE shows that regulation under perturbation can be fit through a low-dimensional gene-to-module bottleneck at the scale of the TF Atlas. It does so entirely offline, with no experiment selection. metaTF builds regulons from curated databases (TRRUST, ChEA, KnockTF) and scores factor activity by enrichment across targets.

Together these papers both constrain and suggest the design. PerturbCausal's linear result argues for a simple, mostly linear model. PerturbODE and metaTF suggest the module space in which to measure uncertainty.

Model and Acquisition

We summarize each perturbation $p$ of a transcription factor $t$ at strength $u_p$ by its pseudobulk mean shift $\delta_p \in \mathbb{R}^G$ over $G$ genes, relative to control cells. The shift is modeled through $K$ modules:

$$m_p = M m_p + W_{:,t}\,u_p, \qquad \delta_p = A\,m_p + \varepsilon_p .$$

The terms are as follows. $m_p \in \mathbb{R}^K$ is the module-activity shift. $M$ is a sparse $K\times K$ module-to-module matrix; feedback cycles are allowed as long as $(I-M)$ is invertible. $W$ is a sparse $K\times T$ transcription-factor-to-module matrix. $A$ is a sparse $G\times K$ module-to-gene loading. $\varepsilon_p$ is noise.

This is a steady-state, linearized analogue of the PerturbODE bottleneck. Keeping it linear is deliberate, given PerturbCausal. Modules are initialized from metaTF-style curated regulons and refined on control cells plus the seed perturbations. The quantity to be discovered is the nonzero pattern of $W$ and $M$.

For transcription factors not yet perturbed, $W$ is constrained only by observational co-variation between factor activity and module activity in control cells. That co-variation leaves the direction and the directness of edges ambiguous. Intervening on the factor resolves this ambiguity.

Uncertainty comes from an ensemble refit on cell-level bootstrap resamples and on sparsity-path draws. For a candidate $t$, each ensemble member predicts $m_p$. Under a Gaussian approximation, the information score is

$$s(t) = \tfrac12 \sum_{k=1}^{K} \log\!\left(1 + \frac{\mathrm{Var}_{\text{ens}}[(m_p)_k]}{\sigma_k^2}\right).$$

Here $\sigma_k^2$ is the noise variance of module $k$'s measured shift, estimated from cell resampling. Because each module averages over many targets, $\sigma_k^2$ is far smaller than per-gene noise. The gene-space version of the score replaces $m_p$ with $\delta_p$ and $\sigma_k^2$ with per-gene noise $\sigma_g^2$.

Batches are drawn by sampling candidates in proportion to their score, following Soft-CBED, so that one batch does not collapse onto a single uncertain region. On SERGIO, where perturbation strength can be set freely, $u_p$ is also chosen, following CBED's finding that dose carries information. On the real screens, strength is fixed by the assay.

Research Program

The work has three parts.

Diagnostic study. On each real screen, we compute both gene-space and module-space scores for every unperturbed candidate. We then reveal the candidate's actual data and measure the realized reduction in posterior edge entropy. The question is which score correlates with what is actually learned. This directly tests the explanation for why BALD loses to random.

Retrospective active loops. We run active loops on Replogle K562 and RPE1, Norman 2019, and the TF Atlas 812-gene subset. "Running an experiment" means revealing a held-out perturbation's cells. The protocol starts from 20 seed perturbations, acquires batches of 20, and stops at a budget of 200, matching the LLM-guided study.

Synthetic studies with known truth. We repeat the loops on SERGIO, which has realistic dropout and known graphs, and on DREAM. These let us check the method against the ground truth that real data lacks, and let us place it next to CBED and Licence on their own benchmark.

Evaluation on real data must respect PerturbCausal's warning that reference graphs are noisy. The primary real-data metric is therefore convergence to the module graph estimated from the full screen. We report AUPRC over edges at each budget, normalized by the split-half agreement of that full-screen graph. That agreement is the best any method can hope for, and we report it explicitly.

Secondary metrics are AUPRC against ChIP-Atlas binding for TF Atlas factors, which follows PerturbODE's external check, and TRRUST overlap, reported with its chance level. On SERGIO and DREAM we report AUPRC and Structural Hamming Distance, meaning the number of edge additions, deletions, and reversals needed to match the true graph, against the true graph.

Baselines

The naive reference point is uniform random selection of perturbations under the same budget and batch size. It is fed to the identical module-level model. Here random is a serious competitor, not a straw man: the LLM-guided study calls it the "Final Boss" baseline because uncertainty methods lost to it. Beating it means a statistically reliable gain in normalized AUPRC across the area under the budget curve, over at least ten seeds, on most pathways or screens. Our results will name the screens and pathways where we fail to beat it.

The strongest graph-oriented competitor is gene-level information-based design in the style of CBED. Full CBED cannot run on hundreds of genes, so we implement its acquisition over a bootstrap ensemble of sparse linear gene-level graphs. These are ElasticNet regressions, matching the comparator that held its own in PerturbCausal. The ensemble is given the same data, the same refit budget, and the same Soft-CBED batching. We run original CBED and Licence only on DREAM, where they were designed to operate. For reference, CBED reports recovering DREAM graphs "up to 4 times faster than competing methods," and Licence reports an approximately 27.7% Structural Hamming Distance improvement on Erdős-Rényi graphs. For gene-level information-based design on real Perturb-seq graph recovery, no reported result was found on Lacuna, so we will establish that number ourselves.

The strongest real-data active learner is the LLM-guided two-stage pipeline: deep-ensemble uncertainty filtering followed by LLM re-ranking. That study reports the lowest expression-error area under the curve on all five K562 pathways and wins on 2 of 4 RPE1 pathways. No graph-recovery number is reported. We run its acquisition order and feed the selected perturbations to our module model, so that only the choice of experiments differs. We also include its BALD component alone.

The key ablation is the same module model with gene-space scoring. It isolates the claim that the space in which uncertainty is measured matters. A second ablation keeps module-space scoring but uses random gene groupings of matched sizes. It tests whether curated regulon biology matters, or whether any averaging over groups suffices.

Why not just…?

Why not just pick perturbations at random? This is the main threat. The LLM-guided study shows that it often wins on real screens. If random matches module-space scoring at every budget, we will report that the active component adds nothing at these scales. The diagnostic correlation study will then still say why, by showing whether any score predicts realized learning. Synthetic evidence that random loses badly, such as CORE's Structural Hamming Distance of 1.26 versus 8.33 for random, does not transfer by assumption. That is exactly the gap this proposal tests.

Why not just run CBED-style information gain on genes with a sparse linear model? This is the matched ablation, and it is the experiment the proposal stands or falls on. Our prediction is that, at the noise levels seen in single-cell data, gene-level variance among ensemble members tracks per-gene sampling noise. Its scores should therefore correlate weakly with realized edge-entropy reduction. If the gene-level scores correlate as well as the module-level ones and the budget curves overlap, modules are unnecessary, and we drop them.

Why not just perturb the predicted master regulators first? Ranking transcription factors by the size of their curated regulon, or by VIPER-style activity variance, is a cheap heuristic suggested by the master-regulator view of Califano et al.. We include it as a baseline. We expect it to help early and stall later, because it never targets ambiguous edges among smaller modules. If it matches our method throughout, the honest conclusion is that regulon size, not uncertainty, drives the gain.

Why not just simulate perturbations with a virtual cell model and skip real experiments? PerturbCausal already shows that this route fails for graph recovery, with inflated effects that calibration does not repair. We therefore do not use simulators as cheap experiments.

Decisive Checks

Three results would settle the question. First, in the diagnostic study, the correlation between module-space score and realized information should clearly exceed that of the gene-space score on at least the K562 and TF Atlas screens. Second, the module-scored loop should reach a given fraction of the full-screen graph's achievable AUPRC with fewer perturbations than random, gene-level information gain, and LLM-guided selection. It should also hold that advantage on ChIP-Atlas AUPRC. Third, on SERGIO, where the truth is known and noise can be dialed, the advantage of module-space over gene-space scoring should grow as dropout increases. It should vanish when noise is low, since that is when gene-level estimates become reliable. If it does not vanish, the pooling explanation is wrong even if the method works.