When the numbers inside a neural network are rounded onto a coarse grid, the network becomes strangely hard to nudge: an input perturbation smaller than the rounding step gets erased in the forward pass, because everything within a grid cell maps to the same value. This "quantization-induced robustness" is well documented for image classifiers — Sanchari Sen showed that low-precision networks absorb small perturbations that defeat full-precision ones, and vector-quantization defenses push this much further by snapping whole feature patches to nearest neighbors in a learned discrete codebook. Meanwhile, an unrelated field has an robustness problem of its own: models that predict how a cell's gene expression changes after a drug or gene knockout collapse when tested on perturbations unlike their training set. PertEval-scFM found that single-cell foundation models trained on tens of millions of cells barely beat a mean predictor under such shifts, while a small specialized graph model won easily. Nobody has tried the obvious transplant: put a discrete codebook between the cell and the predictor, so that biological noise too small to matter falls between codewords and disappears.
The application regime we target is inhibition of the TGF-β/Smad pathway, the signaling cascade that drives fibrosis and epithelial–mesenchymal transition (EMT). This regime is ideal for two reasons. First, the pathway's noise structure is unusually well characterized: live-cell imaging and stochastic modeling by Kolbe, Hexemer, et al. showed that single cells show unpredictable bursts of Smad2 nuclear entry driven by stochastic receptor internalization, and a companion line of ensemble modeling established that the pathway senses analog fold-changes rather than switching on and off. That gives us a principled, data-grounded way to set how coarse the quantization should be: coarse enough to erase sub-threshold noise, fine enough to preserve the fold-change signal the pathway actually uses. Second, the pathway has a sharply characterized discrete outcome — an irreversible EMT tipping point between 1 and 2 hours after TGF-β exposure, located by dynamic network biomarker analysis in SMAD7 and SERPINE1 as novel dynamic network biomarkers — which lets us test whether the geometry of a learned codebook independently rediscovers that boundary.
Core Claim
A vector-quantization bottleneck, applied to gene-module patches of a cell's expression profile rather than to image pixels, will substantially improve out-of-distribution prediction of transcriptional responses to TGF-β/Smad perturbations, relative to the same predictor with a continuous latent space of matched capacity. The mechanism is the one demonstrated for vision: perturbations smaller than the distance to the nearest codeword boundary are projected back to the clean centroid. The closest existing work establishes each half of this claim separately but never together. DEFEAT inserts exactly this kind of codebook into CLIP image embeddings, lifting robustness on EuroSAT from 11.21% to 54.65% — but touches only vision-language data. Adversarial defenses via vector quantization builds the rate-distortion theory and shows that patch-based vector quantization (swRD) reaches 73.67% robust accuracy under AutoAttack on CIFAR-10 where scalar rounding reaches only 13.92% — but only on pixel grids. PertEval-scFM documents the out-of-distribution failure of continuous-latent perturbation models — but proposes no defense and models perturbations only by zeroing out gene tokens. Our proposal is the first to place the discrete-latent defense inside perturbation-response predictors, to calibrate its resolution against measured pathway dynamics, and to read the resulting codebook geometry as a model of cell-fate discretization.
Research Program
The build has three components. The first is the module itself: partition the expressed genome into gene-module patches (pathway annotations, including the 37-gene dynamic network biomarker module identified in the SMAD7/SERPINE1 study), encode each patch to a latent vector $z_g$, and snap it to the nearest codeword in a shared codebook $\mathcal{C}$:
$$\hat{z}_g = \arg\min_{c \in \mathcal{C}} \|z_g - c\|_2.$$
A perturbation $\delta$ to the cell's profile is ablated whenever $\|\delta\|$ is smaller than half the distance from $z_g$ to the boundary of its Voronoi cell — the direct analogue of Sen's sub-step-size absorption, now in a learned space. Training uses the standard straight-through estimator and codebook-commitment remedies from the VQ-VAE literature; the module is deliberately architecture-agnostic so it can be inserted into both a simple multi-layer perceptron and the graph-based predictor GEARS.
The second component is calibration. The central risk — flagged honestly below — is that discretization destroys exactly the graded fold-change information the pathway uses, since ensemble modeling of TGF-β/Smad shows it is an analog sensor, not a digital switch. We therefore set the effective quantization step using Kolbe et al.'s CIR-process simulator, which reproduces single-cell Smad2 burst dynamics and runs at 2.2 seconds per path (versus 210 seconds for τ-leaping), and which, when calibrated on 100 pM data, predicted responses at 5 and 25 pM without retraining. By feeding simulated trajectories through candidate codebooks, we can measure directly where burst height, duration, and frequency survive quantization while measurement noise is erased, and pick the codebook resolution that separates the two. We expect performance versus codebook size to be non-monotonic — an intermediate resolution should win, echoing the non-monotonic precision–robustness relationship Mathias Lechner and colleagues found in quantized networks.
The third component is certification. Following the pattern of ARQ, which obtained provable robustness guarantees for mixed-precision networks under randomized smoothing (matching full-precision certified robustness on ResNet-50 with 1.5% of the bit operations), we assign codewords stochastically during inference and certify a radius within which the predicted post-perturbation profile is provably unchanged under bounded measurement noise. This matters because it converts "the prediction didn't move" from an anecdote into a guarantee, sidestepping the gradient-masking objection that Elias B. Khalil's combinatorial attacks raised for quantized networks.
Proposed Experiments
The performance evaluation runs on three benchmarks from the file. First, the Norman 2019 CRISPR perturbation dataset under the SPECTRA graph splits of PertEval-scFM, reporting AUSPC (lower is better — the file reports GEARS at approximately 0.008 and scGPT at approximately 0.045, with scGPT barely beating the mean baseline) and top-20 differentially expressed gene metrics. Second, held-out perturbations from the Replogle 2022 genome-scale CRISPRi screens (over 2.5 million cells, roughly 10,000 perturbations in K562 and RPE1), with a dedicated split over TGF-β pathway targets — TGFBR1, SMAD2, SMAD3, SMAD4, SMAD7 — reporting E-distance and false omission rate as in CausalBench, and respecting the scPerturb reliability thresholds of at least 300 cells per perturbation and 1,000 UMI counts per cell. Third, cross-cell-type transfer on the OP3 benchmark of chemical perturbation responses (146 compounds, over 300,000 PBMCs from 3 donors), reporting MRRMSE and cosine similarity.
Two mechanistic checks complement the benchmarks. On GSE17708, the A549 EMT time course with eight timepoints from 0 to 24 hours, we track the population latent trajectory through the codebook and test whether the crossing of a codeword boundary falls between 1 and 2 hours post-induction, the tipping point window independently established by the dynamic network biomarker analysis of SMAD7 and SERPINE1. And on the Kolbe simulator outputs at 0, 2.5, 5, 25, and 100 pM TGF-β, we quantify exactly which burst statistics survive quantization and which noise is ablated, producing the calibration curve that fixes codebook resolution for the live-data experiments.
Baselines
The do-nothing reference point is the mean baseline: predict the population-average unperturbed profile for every perturbation. PertEval-scFM reports that scGPT's AUSPC of roughly 0.045 on Norman barely outperformed this baseline, but the file gives no numerical value for the mean baseline itself — no reported result found on Lacuna for the mean baseline's AUSPC — so we will re-run it on our splits. The strongest existing competitor is GEARS, the graph neural network over gene-gene interactions, which PertEval-scFM reports at an AUSPC of approximately 0.008 on the Norman dataset, the best figure in the file. We will also run a multi-layer perceptron on raw gene expression, since the same benchmark found MLPs matched or beat all foundation models — a result that makes the MLP a genuine contender rather than a strawman. Fairness is enforced by giving every method the same training data, the same SPECTRA splits, the same tuning budget (the same number of random-search configurations and training FLOPs), and five seeds. Our method is evaluated as the identical architectures (MLP and GEARS) with the codebook inserted, so any gain is attributable to discretization and not to extra capacity. Beating the baseline means an AUSPC below approximately 0.008 with non-overlapping seed intervals on the Norman splits, plus improvement on the top-20 DEG metrics where PertEval found all models struggle. Two ablations isolate the key ingredient: an equal-dimensional continuous bottleneck (no snapping) and scalar per-gene rounding (no patching), mirroring the scalar-versus-vector comparison in Adversarial defenses via vector quantization, where scalar rounding reached 13.92% robust accuracy and patch-based vector quantization reached 73.67% on CIFAR-10.
Why not just…?
Why not just use a continuous denoising autoencoder or a smaller continuous latent? This is the most serious alternative, and we do not argue around it — the continuous-bottleneck ablation is designed to settle it. The only direct evidence in the file favoring discretization over continuous filtering comes from vision: scalar continuous discretization reached 13.92% robust accuracy under AutoAttack while vector quantization reached 73.67% in Dong and Mao's work, and DEFEAT found continuous CLIP embeddings fundamentally unable to resist small feature nudges where discrete codebooks raised robustness from 11.21% to 54.65% on EuroSAT. That is suggestive, not decisive, for gene expression. If the matched continuous bottleneck matches our discrete one on the Norman and Replogle splits, the core claim is falsified and we will report it as such.
Why not just build a mechanistic ODE/SDE model of the pathway? Deterministic ensemble models with log-normal parameter sampling already reproduce single-cell trajectories across more than one million MCF10A data points, per the TGF-β/Smad modeling review. But these models predict nuclear Smad dynamics, not genome-wide transcriptional responses to arbitrary knockouts, and they suffer documented parameter non-identifiability; Kolbe et al.'s stochastic model is kinetic and stops at the signaling-to-transcription gap. We therefore reuse the stochastic model rather than compete with it — it supplies the calibration data for codebook resolution — and we evaluate on the perturbation benchmarks where mechanistic models make no predictions at all. CausalBench adds a caution that also applies here: methods imposing acyclic structure failed to beat observational baselines on real interventional data, which is why our predictor keeps the graph prior of GEARS rather than a causal-DAG assumption.
Why not just fine-tune the foundation models? PertEval-scFM already shows this falls short: fine-tuned foundation models decayed under out-of-distribution graph shifts, scaling from 5 million to 33 million pre-training cells did not help, and contextual alignment with the target task mattered more than size. Our change is orthogonal to pre-training scale — it alters the geometry of the representation — and we test it inside the strongest small model (GEARS) precisely so that a win cannot be attributed to foundation-model capacity. The honest residual risk is that GEARS's graph prior already absorbs the same noise our codebook would filter; if the codebook adds nothing on top of GEARS, that is the result, and the vision-style defense simply does not transfer to expression data.
Decisive Checks
Three outcomes would decide the proposal. The continuous-versus-discrete ablation on matched capacity is the primary test of the mechanism. The codebook-size sweep is the primary test of the calibration story: a U-shaped curve with an interior optimum, consistent with Lechner's non-monotonic precision findings, supports the claim that resolution must sit between noise and signal; monotone degradation toward the continuous limit would instead support the analog-sensor objection. And the GSE17708 boundary-crossing check is the primary test of the mechanistic reading: if the learned codebook's decision cliff aligns with the independently measured 1–2 hour EMT tipping point of SMAD7 and SERPINE1, then quantization is not merely a defensive trick borrowed from vision — it is a usable formal language for how analog Smad accumulation becomes a discrete cell fate.
References
- PertEval-scFM: Benchmarking Single-Cell Foundation Models for Perturbation Effect Prediction
- Data-based stochastic modeling reveals sources of activity bursts in single-cell TGF-$\beta$ signaling
- Modelling cellular signalling variability based on single-cell data: the TGFb/SMAD signaling pathway
- SMAD7 and SERPINE1 as novel dynamic network biomarkers detect and regulate the tipping point of TGF-beta induced EMT
- Discrete Latent Features Ablate Adversarial Attack: A Robust Prompt Tuning Framework for VLMs
- Adversarial Defenses Via Vector Quantization
- ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs
- CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data
- scPerturb: Harmonized Single-Cell Perturbation Data
- A benchmark for prediction of transcriptomic responses to chemical perturbations across cell types