Repression Upper Limits for Single-Cell CRISPR Screens

When a gene is turned down by a repressor, such as a guide-directed CRISPR interference (CRISPRi) system, a small non-coding RNA, or a protein that blocks transcription, the effect has to be read out by counting a handful of RNA molecules per cell. Many of those counts are zero simply because capture failed, not because the gene is silent. Current methods for learning gene networks from perturbation screens make a binary call: an edge "gene A represses gene B" is either found or not. A missing call is then treated as evidence of no effect. Particle and astroparticle physicists, who face the same scarcity of counts, never do this. They report an upper limit ("any effect is smaller than x at 95% confidence"), and they state in advance how large a signal their data could have seen. This proposal brings that practice to single-cell repression screens. For every perturbation–gene pair we compute how much repression the data could have detected, given cell count, sequencing depth and baseline expression. We then ask how much of what benchmarks call "no effect" is really "too weak to see."

The practical payoff is diagnostic as well as methodological. The large CausalBench study found that most network-inference methods stop improving after seeing about half the available cells (A large-scale benchmark for network inference from single-cell perturbation data). That plateau has two readings. Either the algorithms waste information, or the remaining true effects sit below a counting floor that more cells of the same depth barely move. A per-pair sensitivity map tells the two apart. It also turns the benchmark's statistical negatives, which feed its False Omission Rate (the share of predicted non-edges that truly have no effect), into something interpretable.

Core Claim

We claim that a count-level likelihood for each perturbation–gene pair separates three outcomes that current pipelines merge: detected repression, an excluded effect, and an undetectable effect. The likelihood is a negative binomial model with per-cell size factors and an optional mixture term for cells that responded versus cells that did not. The three outcomes mean, respectively, an upper limit that rules out meaningful repression, and an upper limit too loose to say anything. We further claim that the expected sensitivity is close to analytic and explains much of the CausalBench data-utilization plateau. For a gene with mean count $\mu$ per cell, negative-binomial dispersion $\alpha$, and $N$ perturbed cells, the smallest detectable log-fold repression at significance level $z$ is roughly

$$ \delta_{\min} \approx z \Big/ \sqrt{N \, \frac{\mu}{1+\alpha\mu}} , $$

where the fraction is the per-cell Fisher information for a log-fold change under the negative binomial. This gives the familiar $1/\sqrt{N}$ Poisson scaling when $\mu$ is small. When $\mu$ is large, the per-cell information caps at $1/\alpha$, so extra sequencing depth stops helping and only more cells do.

The prediction is falsifiable. If the plateau is a counting floor, the number of pairs crossing into the detectable region should flatten in step with method performance as cells are subsampled. If it is an algorithmic ceiling, the detectable set should keep growing while methods stall.

The mixture term matters for the biology. CRISPRi knockdown and many natural repressors act in a switch-like, partly penetrant way. Thermodynamic models of enhancers show how cooperative binding and short-range repression produce sharp thresholds (Thermodynamics-Based Models of Transcriptional Regulation by Enhancers). A mean shift cannot tell "half the cells fully silenced" from "all cells dampened by half." A mixture likelihood estimates the responding fraction $f$ and the per-cell repression factor $r$ separately, and it reports when the data cannot separate them.

Relation to Prior Work

CausalBench and its challenge supply the setting and the motivating gap (CausalBench; The CausalBench challenge). They benchmark network inference on CRISPR perturbation screens of K562 and RPE1 cells using Wasserstein distances and False Omission Rate, and they report that simple statistics beat heavy causal models. That evaluation never asks whether a missed edge was detectable at all, which is exactly what we add. The cross-domain autoencoder of Learning Cross-Domain Representations for Transferable Drug Perturbations predicts perturbed expression well, with mean $R^2$ of 0.81 across all genes and 0.62 on differentially expressed genes for sci-Plex3. But it optimizes continuous reconstruction and does not state detection limits. Its weaker score on differentially expressed genes is consistent with weak effects being hard to resolve, though it does not prove it.

The statistical method comes from physics. Unbinned likelihood fitting evaluates each event against a continuous model instead of a histogram (Unbinned Maximum Likelihood Estimation for Signal Extraction), and extended likelihoods also fit absolute yields (J. Lee). Rare-event searches such as EXO-200 quote expected sensitivity alongside observed limits.

The physics literature also constrains the design. It warns that roughly 2σ signals often dissolve with more data (Branching fractions for psi(2S) -> gamma eta' and gamma eta), and that unbinned fits with a misspecified background will fit fluctuations as signal. For that reason the proposal estimates dispersion empirically from control cells and validates calibration by injecting known signals, rather than trusting the model's own error bars.

The weak, degenerate target signatures of piRNA-guided repression (Computational Analysis of Mouse piRNA Sequence and Biogenesis) and the subtle somatic repression seen in placenta (Identification and characterization of Piwi-interacting RNAs in human placentas of preeclampsia) motivate why weak repression is the regime that matters. We do not test those systems here. Multi-step cascade falsification in the style of Formalizing and Falsifying Causal Pathways of Rare Events is a natural follow-on but is out of scope.

Research Program

We will build a single tool with three outputs.

The first output is a per-pair profile likelihood. For each perturbation and readout gene, we fit raw counts in perturbed and control cells with a negative binomial model whose mean includes each cell's library size. A repression parameter enters either uniformly (one log-fold change) or as a two-component mixture ($f$, $r$). From the profile likelihood ratio we produce a point estimate, a significance, and a 95% upper limit on repression. The model works on raw counts with no binning or normalization-then-summarizing step.

The second output is an expected-sensitivity map. For every pair we compute the median expected upper limit under the no-effect hypothesis, using an "Asimov" dataset: the expected counts with no fluctuation. The map is checked against the analytic formula above. It sorts each pair into one of three regions: detectable at a chosen effect size, marginal, or blind.

The third output is a sensitivity-aware evaluation. We recompute CausalBench's statistical metrics, Wasserstein distance and False Omission Rate, restricted to the detectable region. We also report what fraction of each method's predicted non-edges fall in the blind region, where a non-call carries no information.

Calibration uses two internal controls that act like the calibration sources physicists place in a detector. The first is real: each CRISPRi guide's own target gene is a known repression. We can therefore ask whether on-target knockdown is detected where the map says it should be. The second is synthetic: binomial thinning of control-cell counts. We keep each molecule of gene $g$ with probability $r$ in a fraction $f$ of cells, which injects a known repression on top of genuine measurement noise. Binomial thinning is the exact operation of capturing fewer molecules under Poisson sampling, so the injected signal inherits realistic noise. It does not reproduce perturbation-induced changes in dispersion, and we will report that caveat.

Baselines

The naive reference point is MeanDifference, the CausalBench baseline that ranks pairs by the absolute difference in mean expression between perturbed and control cells. It is cheap, reported at 0.04 hours of runtime (A large-scale benchmark for network inference from single-cell perturbation data), and it beat PC, GES, NOTEARS and DCDI there. We run it on identical cells and splits. Our method beats it if, on injected signals at fixed false positive rate, it recovers more weak repressions. It should also produce upper limits whose coverage is near nominal, which MeanDifference cannot produce at all.

The strongest existing competitor is BetterBoost from the CausalBench challenge. It augments tree-based GRNBoost with Kolmogorov–Smirnov tests on perturbation-induced distribution shifts, and it ranked at the top of the challenge (The CausalBench challenge). It is the strongest threat because its KS test already looks beyond the mean. The Guanlab LightGBM entry will be run as a second strong competitor. No reported result for BetterBoost's or Guanlab's Wasserstein or False Omission Rate values was found on Lacuna; the file states that edge-level numbers appear only in figures. We will therefore rerun both from their challenge configurations on the same K562 and RPE1 data, with the same cell subsamples and the same compute. Each method gets one tuning sweep of equal size chosen on held-out perturbations. Beating BetterBoost means higher recall of injected repressions in the marginal sensitivity band at matched false positive rate. It also means a smaller fraction of predicted non-edges falling in regions our map marks as detectable but showing a significant observed effect.

The key ablation is a plain negative-binomial log-fold-change test with no mixture term and no sensitivity map. It isolates whether gains come from the count likelihood itself, which is standard in genomics, or from the two ingredients we add: separating responding fraction from depth of repression, and treating non-detections as upper limits.

Why not just…?

Why not just use MeanDifference, which already wins? For ranking strong edges it may be nearly as good, and we expect it to be. Its failure is specific. A mean is close to zero for low-copy genes, so a large relative repression of a rarely expressed gene produces a tiny absolute difference. A mean also cannot tell partial penetrance from uniform dampening. The injection experiment measures both failures directly, by varying $\mu$, $f$ and $r$ under controlled conditions.

Why not just run a standard negative-binomial differential-expression test, or BetterBoost's KS test? This is the real threat. If our gains in edge ranking disappear against the plain negative-binomial ablation, we will say so. The contribution then rests on the sensitivity map, the upper limits and the evaluation correction, not on detection power. Those outputs are not produced by any per-pair test as normally used. A p-value above threshold does not say whether the pair was testable in the first place. The KS test additionally discards the count structure that tells us depth stops helping once $\mu$ exceeds roughly $1/\alpha$.

Why not just subsample cells and plot each method's performance curve, as CausalBench already did? That empirical curve shows a plateau but cannot assign a cause. Our map adds an independent, method-free prediction: the number of pairs that become detectable at each subsample size. Comparing the two curves is what separates a data floor from an algorithm ceiling.

Proposed Experiments

All experiments use the CausalBench K562 data (about 162,000 cells, 622 interventions) and RPE1 data (about 162,000 cells, 383 interventions).

The first experiment checks calibration. On injected repressions spanning a grid of $\mu$, $f$ and $r$, the 95% upper limits should cover the true value in roughly 95% of trials. The predicted detection boundary should match observed recovery within sampling error. On real data, on-target knockdown should be detected wherever the map predicts sensitivity, and misses outside that region should be rare.

The second experiment tests the plateau. At 10, 25, 50, 75 and 100 percent of cells, we track two things side by side. One is the count of pairs entering the detectable region at a fixed repression size, for example a 30% reduction. The other is the Wasserstein and False Omission Rate performance of MeanDifference, BetterBoost and Guanlab. Flattening of both supports a counting floor. Continued growth in the detectable set while methods stall supports an algorithmic ceiling, and we would report how many detectable pairs each method leaves unexploited.

The third experiment revisits the benchmark. We report how method rankings change when False Omission Rate is computed only over detectable pairs. We also report what share of each method's non-edges were blind.

The fourth experiment tests the mixture term. On injected data, we measure whether $f$ and $r$ are recovered separately, and how many cells are needed before they are. On real data, we report the distribution of on-target knockdown penetrance across guides. This is a direct, testable readout of how switch-like CRISPRi repression actually is.

A negative outcome would still be useful. If the plain negative-binomial ablation matches the full model on every check, the field would learn that the plateau and the strength of simple baselines come down to counting statistics that any count test already captures.

References

  1. A large-scale benchmark for network inference from single-cell perturbation data
  2. Thermodynamics-Based Models of Transcriptional Regulation by Enhancers: The Roles of Synergistic Activation, Cooperative Binding and Short-Range Repression
  3. CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data
  4. The CausalBench challenge: A machine learning contest for gene network inference from single-cell perturbation data
  5. Learning Cross-Domain Representations for Transferable Drug Perturbations on Single-Cell Transcriptional Responses
  6. Update of the search for the neutrinoless decay<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mrow><mml:mover><mml:mrow><mml:mi>τ</mml:mi></mml:mrow><mml:mrow><mml:mo>→</mml:mo></mml:mrow></mml:mover></mml:mrow></mml:mrow><mml:mi>μ</mml:mi><mml:mi>γ</mml:mi></mml:math>
  7. Search for Neutrinoless Double-<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mi>β</mml:mi></mml:mrow></mml:math>Decay with the Complete EXO-200 Dataset
  8. Branching fractions for<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mi>ψ</mml:mi><mml:mn/><mml:mo>(</mml:mo><mml:mn>2</mml:mn><mml:mi>S</mml:mi><mml:mo>)</mml:mo><mml:mn/><mml:mo>→</mml:mo><mml:mi>γ</mml:mi><mml:mrow><mml:msup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mo>′</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:math>and γη
  9. Computational analysis of mouse piRNA sequence and biogenesis
  10. Identification and characterization of Piwi-interacting RNAs in human placentas of preeclampsia
  11. Formalizing and Falsifying Causal Pathways of Rare Events