Labs that image cells or sequence tissue must pick a neural network design, and the designs that score best on their own data often fail on the next day's plates. The failure comes from technical batch effects: variation introduced by reagent lots, microscope drift and plate position that can be larger than the biology being measured. Checking whether a design survives new batches usually means training it many times across held-out batches, which is exactly the kind of compute that academic labs lack. Zero-cost proxies are scores computed from a randomly initialized network using one or two minibatches, and they already let researchers rank architectures in minutes rather than GPU-days. They have only been tested on clean, well-mixed benchmarks such as CIFAR and ImageNet. We propose a proxy that asks a different question of an untrained network. At initialization, does the network's gradient respond more to differences between biological treatments within a batch, or more to differences between batches? Negative-control wells carry no treatment signal, so they supply a direct measurement of the second quantity.
The claim is testable and could fail in informative ways. Existing proxies might already predict cross-batch accuracy. Architecture might not matter once test-time normalization is applied. Or random-weight features might do the job without any gradients. We therefore build the missing ground truth first: a tabular benchmark of architectures trained and evaluated on held-out experimental batches. Only then do we compare the batch-contrast proxy against every simpler alternative. The whole pipeline, apart from the one-time benchmark construction, is designed to run on a single GPU.
Core Claim
Current proxies treat all variation in per-sample gradients as a single kind of "noise." ZiCo (ZiCo) is an example. It scores a network by the ratio of mean to standard deviation of absolute gradients across samples. The authors derive this from convergence and generalization arguments under i.i.d. sampling, and report a Kendall's τ of 0.61 on NATSBench-TSS. In multi-batch screening data, gradient variation has two sources that should be scored in opposite directions. Variation between treatments within a batch is signal. Variation between batches is a shortcut the network will learn. Negative controls estimate the batch component with no biology mixed in. The proposed score, for layer $l$, is
$$S = \sum_{l} \log \frac{P_l}{B_l},$$
where $P_l$ is the variance, across treatment classes within the same batch, of class-mean absolute gradients at initialization (averaged over batches), and $B_l$ is the variance across batches of the mean absolute gradient on negative-control samples. When controls are absent, $B_l$ falls back to the variance across batches of batch-mean gradients over all samples. The claim is that $S$ ranks architectures by accuracy on unseen batches, and by the size of their seen-to-unseen gap, better than generic proxies do. It should also be less tied to parameter count.
Research Program
Three findings motivate the work. The RxRx1 authors (RxRx1) show that a supervised DenseNet161 drops from 91.1% accuracy on seen batches to 75.1% on unseen ones, so batch shortcuts are real and costly. Svatko et al. (Deep Learning for BioImaging) find that untrained ViTs reach about 0.19 Recall@5% on RxRx3-core, against about 0.20 for pretrained foundation models, and that early-layer textures and plate artifacts drive much of the apparent performance. They treat this as a diagnosis of the benchmarks and do not turn it into a way to score architectures. We take their observation, that initialization already reveals what a network is sensitive to, and use it to build a proxy.
The method borrows two ideas from existing work. From CS-ARM-BN (Stabilizing In-Context Multi-Source Domain Adaptation), which adapts normalization statistics using a plate's negative controls, we take controls as a clean reference for technical drift. From ZiCo we take the per-layer log-ratio over gradient statistics. The proxy for adversarial robustness (Zero-cost Proxy for Adversarial Robustness Evaluation) is the closest precedent for scoring robustness rather than clean accuracy at initialization. It does this through NTK eigenvalues and a finite-difference estimate of input-space curvature, but it targets norm-bounded attacks on natural images and has a documented bias against skip connections. Our proxy targets a physical, structured shift instead. We check it for the same skip-connection bias, since Svatko et al. report representational collapse in ResNets.
Two negative results constrain the design. NAS-Bench-Suite-Zero (NAS-Bench-Suite-Zero) found that many proxies mostly track model size and needed explicit de-biasing. We therefore report partial correlations controlling for parameter count for every proxy. The RxRx1 authors found that gradient reversal hid batch identity (2.3% batch accuracy) without by itself fixing unseen-batch accuracy. This warns that "batch-invisible" is not the same as "batch-robust," so we score the ratio against unseen-batch accuracy directly rather than against how hard it is to decode the batch.
What we build is a ground-truth benchmark. We take a cell-based search space in the style of NATSBench, with input stems widened to six fluorescence channels. We sample several hundred architectures that also span ViT-like and residual designs. Each is trained with a short fixed schedule on downsampled RxRx1 images using leave-batches-out splits. We record seen-batch accuracy, unseen-batch accuracy, and unseen-batch accuracy after AdaBN (re-estimating normalization statistics on the target batch). A second, smaller set is trained on JUMP-CP sources and evaluated with CS-ARM-BN, where plate controls are part of the protocol. For RxRx3-core, which has no comparable training split, we use frozen random-weight embeddings as an external retrieval check, following the setup Svatko et al. used.
The funding-bias literature is a constraint on this program, not a subject of study. On Tables with Numbers, with Numbers and Irresponsible AI document that compute cost concentrates this work in corporate labs, and both offer no technical countermeasure. We respond in two ways. The expensive step, the benchmark, is paid once and released as a table, so other labs can test future proxies without retraining anything. And every proxy is timed on a single consumer-grade GPU. The benchmark also lets us address one tension in the file directly. RxRx3-core reports that the 1.86B-parameter Phenom-2 leads on drug-target average precision at 0.307, while untrained networks nearly match pretrained ones on recall. Within our search space, we can measure whether added capacity buys unseen-batch accuracy or mainly seen-batch memorization.
Baselines
The naive reference points are parameter count and FLOPs. On NATSBench-TSS, ZiCo's authors report that #Params reaches Kendall's τ 0.57, above Synflow at 0.54. On batched data, size may predict seen-batch accuracy well, which is why it is the first thing the proposal must beat on unseen batches. We also report random ranking, and seen-batch validation accuracy after full training. The latter is the "oracle-but-wrong-target" reference. If trained seen-batch accuracy already ranks architectures well on unseen batches, the problem is small.
The strongest existing proxies are ZiCo and the XGBoost ensemble over 13 proxies from NAS-Bench-Suite-Zero, which the authors report improves Spearman correlation by up to 42%. We also include nwot, which that paper found the most robust single proxy. All proxies see the identical two minibatches drawn across the same source batches, the same initialization seeds (five per architecture), and the same hardware. The ensemble is fit with the same number of labeled architectures that any learned variant of our proxy would see.
Svatko et al.'s random-weight features supply the most dangerous simple baseline. Here we train a linear probe on untrained-network features from source batches and evaluate it on held-out batches, and use that accuracy as the score. This costs a few minutes and needs no gradient statistics.
A low-fidelity training baseline trains each architecture for one epoch and ranks by held-out-batch accuracy.
Three ablations isolate the key ingredient. The first replaces $B_l$ with generic across-sample gradient variance, which recovers a ZiCo-like score. The second estimates $B_l$ from all samples rather than controls. The third shuffles batch labels before computing $B_l$. These tell us whether separating batch variance from other variance matters, and whether controls matter.
No reported result was found on Lacuna for any zero-cost proxy ranked against unseen-batch accuracy on RxRx1, JUMP-CP or RxRx3-core, so the numbers from clean benchmarks cited above are context rather than targets. Beating a baseline means a higher Kendall's τ with post-AdaBN unseen-batch accuracy, with the difference exceeding the 95% bootstrap interval across architectures and seeds. It must also hold after partialling out parameter count and appear on both RxRx1 and JUMP-CP.
Why not just…?
Why not just use ZiCo or the proxy ensemble as they are? They may work, and this is the first thing we test. ZiCo's derivation assumes i.i.d. sampling, and its denominator counts batch drift as ordinary noise. Nothing in the file shows how it behaves across batches. Ablation one decides this question. If a ZiCo-like score with generic variance ranks unseen-batch accuracy as well as $S$, the batch-contrast idea adds nothing and we will report that.
Why not just use an off-the-shelf backbone with AdaBN or CS-ARM-BN and skip architecture choice? This is the most serious threat. AdaBN reaches 95.6% on unseen RxRx1 batches, and CS-ARM-BN reaches 0.930 ± 0.019 on JUMP-CP against a 0.939 ± 0.005 in-domain baseline. If adaptation flattens the differences between architectures, a proxy has little to select. We therefore make the post-adaptation spread across architectures a primary measurement. We report the gap between the best and median architecture after AdaBN and CS-ARM-BN, and target the proxy at post-adaptation accuracy rather than raw accuracy. We also test the regime where CS-ARM-BN's own paper shows AdaBN and TENT collapse: target batches of size one without controls, and severe label shift. If the spread after adaptation is within seed noise everywhere, the proposal's value reduces to the benchmark and the negative result.
Why not just probe random-weight features, as Svatko et al. effectively did? This could match $S$, because both read out what an untrained network is sensitive to. The gradient version has one possible advantage. It reflects how training will move the weights, not only what the initial features contain, and that may matter more for trained cross-batch accuracy than for frozen retrieval. The head-to-head comparison on the benchmark settles it. If the feature probe ties, we recommend it as the simpler tool and report the gradient score as unnecessary.
Why not just train each candidate for one epoch? Short training is often strong, but early training is when networks pick up the easy, low-level cues Svatko et al. identify. So one-epoch held-out accuracy may reward shortcut-prone designs. We compare it directly and report its cost in wall-clock time next to the proxies'.
Decisive Checks
The first check establishes whether the premise holds. We compute Kendall's τ between seen-batch and unseen-batch accuracy across the trained architectures. If it is near one, batch robustness is not a separate ranking problem and ordinary proxies suffice.
The second check compares proxies. We rank all proxies and baselines by τ against post-AdaBN unseen-batch accuracy on RxRx1 and post-CS-ARM-BN accuracy on JUMP-CP. We also rank them against the seen-to-unseen gap, partialled for parameter count.
The third check tests whether controls matter: we compare control-based and all-sample versions of $B_l$ on JUMP-CP.
The fourth check tests transfer. A search guided by $S$ within the space should select architectures whose trained unseen-batch accuracy beats the ZiCo-selected and largest-model picks at matched FLOPs. We also check whether RxRx3-core retrieval with frozen random-weight embeddings moves in the same direction.
The fifth check is for bias. We inspect whether $S$ systematically penalizes residual designs, as proxy $R$ does, or favors ViT-like ones in a way that tracks Svatko et al.'s finding that ViTs resist collapse.
A clean win on the first three checks justifies the proxy. A loss on the second check with a large post-adaptation spread still gives the field its first honest ground truth for batch-aware architecture ranking.
References
- ZiCo: Zero-shot NAS via inverse Coefficient of Variation on Gradients
- RxRx1: A Dataset for Evaluating Experimental Batch Correction Methods
- Deep Learning for BioImaging: What Are We Really Learning?
- Stabilizing In-Context Multi-Source Domain Adaptation for Biomedical Images Through Controls
- Zero-cost Proxy for Adversarial Robustness Evaluation
- NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies
- On Tables with Numbers, with Numbers
- Irresponsible AI: big tech's influence on AI research and associated impacts