Bits per Variant: Measuring What Variant Effect Models Must Carry

Deep models that predict whether a genetic variant causes disease turn a tiny input into a very large internal representation. A single missense change (one amino acid swapped for another) is a few bytes of text, but a protein transformer turns it into a vector of a thousand or more numbers at every residue and every layer. Researchers who split neural networks between a phone and a server call this "intermediate feature expansion," and they have built tools that ask a sharp question: how many bits of an internal representation must survive for the downstream task to still work? We propose to point those tools at variant effect predictors. The aim is not to ship genomic models over wireless links. It is to measure, layer by layer and gene by gene, how much information about a variant is actually needed to predict its effect in each kind of laboratory assay and to support a clinical classification.

The answer matters for the central unsolved problem in clinical genetics. More than 90% of missense variants in about 4,000 disease genes are still "variants of uncertain significance" (VUS), meaning no one can say whether they cause disease (Tejura et al.). Part of the difficulty is that variants break proteins in different ways: some destabilize the fold, some disrupt a binding interface while leaving abundance intact (Nielsen et al.; Tejura et al.). Today's predictors compress all of this into a single pathogenicity score. A rate–distortion measurement tells us whether that one number is enough, or whether a few more bits per variant, learned from a frozen model's middle layers, would separate mechanisms that a scalar blurs together.

Core Claim

We claim that a small, discrete code of a few tens of bits per variant, read out of the intermediate layers of a frozen protein model through a learned bottleneck, keeps more of what matters for variant interpretation than the model's final scalar score. Specifically, it should better predict held-out multiplexed assay measurements for each mechanism separately and support gene-specific clinical calibration with no loss in agreement with expert labels. We further claim that the bit rate this requires will differ by assay mechanism and by layer, and that this curve will tell us whether protein transformers show the rising-entropy-with-depth behavior already seen in language and vision transformers. If the scalar score matches the code at equal rate, that is a clean negative result: current predictors are already near the useful compression limit, and remaining VUS will not be resolved by richer model readouts.

Why the Obvious Bridge Is the Wrong One

The file finds no prior work connecting split inference and variant interpretation. The obvious combination is to split a variant effect predictor between a sequencing site and a cloud server and compress what crosses the network. A skeptic defeats that at once. The raw input, a variant string plus a protein sequence of a few kilobytes, is far smaller than any intermediate tensor, so sending it directly beats any codec. The privacy argument is also weak, because intermediate features are known to be invertible by generative attacks (Li et al.; Lei et al.), and a feature that lets a server classify a variant probably reveals the variant. We therefore drop deployment. We keep the measurement machinery of split computing and use it as a scientific instrument on the variant interpretation problem.

Research Program

The method reuses three pieces of split-computing work. From Datta et al. and its companion A Low-Complexity Approach to Rate-Distortion Optimized Variable Bit-Rate Compression for Split DNN Computing we take the idea of freezing the backbone and training only a tiny bottleneck at the chosen layer, with the loss $L = L_t + \alpha L_r$. Here $L_t$ is the task loss, $L_r$ is a differentiable estimate of the bit rate, and $\alpha$ sets the trade-off. Freezing matters here because it means each rate point is a probe of the same fixed model, not a retrained one. From Rate-Distortion Optimization for Transformer Inference we take the learned hyperprior entropy model (a small network that predicts the probability of each code symbol so it can be encoded compactly) and the idea of "usable information," meaning information a bounded decoder can actually extract. That paper also reports the "Transformer Trait": in GPT-2 the bitrate needed to keep accuracy rises from about 148 to about 492 bits per token when the split moves from layer 3 to layer 9. From DT-UFC we take the histogram-equalizing transform that fixes very peaky activation distributions before entropy coding. We include it only as an ablation, because we do not yet know whether protein model activations are peaky.

The backbones are a frozen open protein language model, a transformer trained on protein sequences, and MVP (Qi et al.). MVP is a residual convolutional predictor built on engineered genomic features, and it serves as the convolutional counterpart for the depth-versus-rate comparison. For each variant we form the difference between mutant and wild-type representations at the mutated position and a small window around it. We pass that difference through a bottleneck of roughly 0.3M parameters and quantize the result into a discrete code whose length in bits we can count exactly.

Two task heads sit on the decoded code. The first predicts each assay type as a separate output. These are abundance (VAMP-seq), cellular fitness (saturation genome editing), and interaction (split yeast two-hybrid), drawn from MaveDB (MaveDB 2024) and the integrated assay corpus of Tejura et al.. The second predicts a continuous probability of pathogenicity using the flat-bottomed, $\epsilon$-insensitive loss of Cao et al., which forgives errors inside a clinical category and penalizes errors that cross category boundaries. That loss is where the file's third gap, asymmetric clinical cost inside a rate objective, gets tested directly.

Sweeping $\alpha$ and the tapped layer produces rate–performance curves for each mechanism and each gene. The scientific outputs are three. First, the minimum bits per variant needed to reach a fixed fraction of full-representation performance, per mechanism. Second, whether that minimum rises or falls with depth in the protein transformer compared with MVP. Third, whether scores computed from the codes survive gene-specific calibration of the ExCALIBR type, which converts continuous scores into clinical evidence points (Tejura et al.). All splits are by gene, so no gene contributes to both training and evaluation of a code.

Baselines

The naive reference point is the frozen model's own final pathogenicity score, stored at 16 bits per variant, or any existing scalar predictor such as AlphaMissense (Livesey et al.). This is also the most dangerous competitor, because it is already an extremely compressed code that experts use. We fit the same per-assay regression heads on top of it, with the same gene-held-out folds and the same hyperparameter search budget as the learned codes. Beating it means a code at no more than 64 bits gives higher per-assay Spearman correlation on held-out genes for at least the interaction and abundance mechanisms, with no drop in ClinVar area under the curve.

The second baseline is an uncompressed probe. The same heads are trained on the full-precision intermediate representation, which is thousands of 32-bit numbers. This sets the performance ceiling that defines "fraction of full performance," and it tells us how much the bottleneck costs.

The third baseline is the natural cheap compressor: principal components of the representation, uniformly quantized to the same bit budget. It uses the same data, and the number of components and the bits per component are tuned under the same search budget. The learned code must dominate this curve to justify being learned.

The fourth baseline is lossless Deflate on the full representation. de Andrade et al. report that their learned codec reached equal task performance at 21.84% of Deflate's bitrate on language models. We expect a similar or larger gap here but will report it rather than assume it.

The strongest existing clinical system is kernelized weakly supervised regression on MutPred2 features (Cao et al.). On the CAGI ENIGMA set of 430 BRCA1/2 variants it reports a binary AUC of 0.97, up from 0.72 for logistic regression, and an ordinal multi-class AUC of 0.80. We will reproduce this setting and require our code-based head to match 0.97 binary AUC within confidence intervals at a stated bit budget. For the calibration outcome, the reference is Tejura et al.'s resolution of 75% (12,135 of 16,115) of ClinVar VUS in 40 genes with under 1% discordance. We will rerun that calibration on code-derived scores and count how many VUS reach evidence thresholds at the same discordance ceiling. For per-mechanism MAVE prediction under a rate constraint, no reported result was found on Lacuna.

Two ablations isolate the key ingredient. The first replaces the task-aware rate loss with plain reconstruction error at matched rate. This tests whether task awareness, and not compression alone, keeps mechanism information. The second replaces the $\epsilon$-insensitive clinical loss with ordinary cross-entropy, to see whether asymmetric clinical cost changes which bits are kept.

Why not just…?

Why not just send the raw variant and run the model in the cloud? For deployment, one should, and that is exactly why this proposal does not claim a deployment benefit. The raw input is smaller than any feature. The bits-per-variant question is about what the model must know, not about network links, so this alternative answers a different question and does not threaten ours.

Why not just use the existing scalar pathogenicity score? This is the real threat. The scalar is highly compressed and already calibrated in practice. The motivation for doing better is mechanistic diversity: Nielsen et al. show that Lynch syndrome variants mostly act through destabilization, while Tejura et al. document variants with normal abundance but broken interactions, and one number cannot separate these. That motivation does not prove anything, though. The first baseline comparison settles the question. If the scalar ties the code on held-out-gene interaction assays, we report that current scores already carry the useful information.

Why not just fit linear probes or principal components on the embeddings? Probing without a rate constraint tells us whether information is present somewhere in thousands of dimensions. It does not tell us how much is needed, which is what distinguishes mechanisms and layers. Quantized principal components are included at matched rate as a baseline. If they match the learned bottleneck, the result reduces to "the information is linear and low-rank," which is still useful, and the learned codec is unnecessary.

Why not just run the full model locally on hospital hardware? For inference this is fine and irrelevant to our claim. It does not produce a compact per-variant object that can be precomputed for every possible missense variant in clinical genes, shared the way MaveDB shares measurements, and recalibrated per gene as new assay data arrive.

Proposed Experiments

The first decisive check tests the Transformer Trait on protein models. We plot the bits per variant needed to hold 95% of full-representation Spearman correlation against layer, for the protein transformer and for MVP. A rising curve for the transformer and a falling one for MVP would extend de Andrade et al. to biology. A flat or falling transformer curve would show that biological transformers behave differently from language models.

The second check is the scalar-versus-code test on gene-held-out MaveDB and Tejura assay data, reported separately for abundance, fitness and interaction. The core claim stands or falls on the interaction and abundance results.

The third check is clinical. We apply gene-specific calibration to code-derived pathogenicity scores on ClinVar and BRCA Exchange (Rehm et al.), and report the fraction of VUS reaching evidence thresholds, discordance with expert labels, and binary and ordinal AUC on CAGI ENIGMA against the 0.97 and 0.80 references. We also measure how calibration thresholds shift as the rate drops, which addresses the file's fifth gap about compression noise and discordance.

The fourth check examines activation statistics. We measure how peaky the protein model's activations are, and whether the DT-UFC transform changes rate at matched performance. A null effect here would simply narrow the method.

The final deliverable is a precomputed low-bit code for every possible missense variant in the 40 calibrated genes, released together with its rate–performance curves.