Phylogenetic Shrinkage for Sparse Reservoir Slices

Machine learning models can now read a virus's genome and guess whether it can infect humans, which is exactly the capability pandemic preparedness needs: most animal viruses will never be tested in a lab, so a sequence-only screen is the only scalable triage. But the screen fails precisely where it matters most. In Hidden challenges in evaluating spillover risk of zoonotic viruses using machine learning models, Kawasaki et al. fine-tuned genomic language models — text-style models trained on DNA instead of words — on 140,638 viral sequences, and the resulting BERT-infect model earned the highest PR-AUC (a precision-recall summary) in 18 of 26 viral families, yet scored an F1 of 0.000 on SARS-CoV-2-related sarbecoviruses and H5 influenza A: the two lineages that actually caused the deepest recent emergencies. Aggregate success hid total failure on the specific slices that count.

I argue this is an intersectional data sparsity problem in disguise, and that the fairness literature has already built the missing machinery without ever pointing it at biology. Slice a pathogen corpus by host clade × viral clade × geography × time and you get the same combinatorial explosion studied in algorithmic fairness, where fine-grained subgroups empty out and models trained on the population average quietly do worse on the sparse pockets than no model at all. The proposal here is to treat each reservoir–virus–region cell as a group sitting on a biological tree, share statistical strength across that tree with structured shrinkage, stop treating "never observed infecting humans" as a confident negative in barely sampled cells, and audit the result with size-adaptive credible intervals instead of point estimates.

Core Claim

The claim has three parts. First, the sarbecovirus and H5 failures are not random bad luck but a predictable consequence of training with an unweighted loss over intersections whose sample support is radically uneven — the same mechanism the fairness literature calls worsenalization. Second, a group-structured prior whose covariance follows phylogenetic distance — hosts that share recent ancestry, and viruses that share recent common ancestors, should have similar infectivity adjustments — will recover part of the lost worst-slice performance that neither aggregate training nor flat per-group reweighting can, because it borrows evidence from relatives rather than amplifying noise within the sparse cell itself. Third, the evaluation protocol matters as much as the model: on slices with a handful of sequences, a point F1 is meaningless, and only interval-valued, size-adaptive audits can say whether a model is actually better than chance on the slices that matter.

Prior Work and What It Leaves Open

Four lines of work converge on this gap without closing it. The motivation comes from Hidden challenges, which documents the aggregate-versus-slice failure and the censoring problem — sequences labeled "animal-infecting" may simply never have had a documented human exposure opportunity — but treats both as generic "hidden challenges," without formalizing them as intersectional sparsity or trying any variance-aware remedy. The method comes from Structured Embedding Models for Grouped Data, whose S-EFE models learn group-specific embeddings that shrink toward a shared mean through Gaussian hierarchical priors, beating isolated per-group models on held-out likelihood across ArXiv, Senate-speech, and grocery data; but its groups are unstructured categorical bins, and its authors explicitly flag that it ignores relational topology between groups — which is exactly what host and viral phylogenies provide. The auditing tool comes from Size-adaptive Hypothesis Testing for Fairness, whose SAFT procedure switches between a large-sample Wald test and Dirichlet-multinomial Bayesian credible intervals and proves a "no-power zone" where even a 100% observed disparity is statistically indistinguishable from noise; it has only ever been run on tabular demographic data. And a design constraint comes from Doubly-Regressing Approach for Subgroup Fairness, whose DRAF algorithm caps combinatorial blowup by enforcing fairness only on subgroups above a support threshold γ — a sensible engineering choice that is unacceptable here, since in reservoir surveillance the subgroups below the threshold are the ones most likely to harbor the next spillover. Two further results bound what I should expect: On the Epistemic Limits of Personalized Prediction proves minimax limits on auditing fine-grained subgroups (for 19 or more binary attributes, even a global-population sample cannot reliably verify a 1% personalization gain), which tempers claims about what sparse-slice evaluation can certify; and Bounding and Approximating Intersectional Fairness through Marginal Fairness merges correlated attributes into statistical blocks using total correlation, a spirit similar to my tree-based merging but driven by statistical association in the data rather than external biological structure. FairGrad supplies the strongest cheap competitor — per-group Lagrangian reweighting of gradients — and also a warning, since it proves some parity objectives cannot be reached with positive weights alone. Nothing in the retrieved literature applies any of this to pathogen sequences; the closest adjacent work, Fair Few-shot Learning with Auxiliary Sets and the transfer-learning survey Transfer Learning from Pretrained Protein Language Models, borrow statistical mass across tasks or exploit pretraining, but neither conditions on intersectional structure or evaluates slice-wise failure.

Research Program

The build is a fine-tuned genomic language model (starting from the DNABERT and ViBE backbones used by BERT-infect, with its 250-base-pair fragments and 4-mer tokens) augmented with a per-slice adjustment vector $\delta_g$, where a slice $g$ is a cell in host clade × viral clade × region. The prior is where the biology enters: instead of S-EFE's exchangeable shrinkage toward a global mean, $\delta_g$ is drawn from a Gaussian whose covariance decays with phylogenetic distance,

$$\delta_g \sim \mathcal{N}(0,\,K), \qquad K_{g,g'} = \rho_h\big(h(g),h(g')\big)\,\rho_v\big(v(g),v(g')\big),$$

where $\rho_h$ is the fraction of shared branch length between the two hosts on the host tree (1 for the same species, near 0 for distant relatives), and $\rho_v$ is the same for the viral tree. A slice with five sequences then inherits a prior anchored by its well-sampled relatives rather than by the global average, and it can still deviate when its own data disagrees.

The second ingredient addresses censoring. A sequence labeled animal-only from a host–region cell that has seen almost no surveillance is a weak negative, not a strong one. I replace the unweighted cross-entropy with

$$\mathcal{L} = -\sum_{\text{human-infecting}} \log p(y{=}1 \mid x;\delta_g) \;-\; \sum_{\text{animal-only}} \pi_g \log p(y{=}0 \mid x;\delta_g), \qquad \pi_g = \frac{e_g}{e_g + c},$$

where $e_g$ is a sampling-effort score for cell $g$ (counts of sequences and exposure records in the cell, computable from repository metadata) and $c$ is a constant; negatives from unsampled cells get $\pi_g$ near zero, so they nudge rather than dominate. The bias of public repositories makes this concrete: the Zoonosis database consolidates over 4,500 samples across more than 60 countries, but its coverage is wildly lopsided — Brucella alone accounts for 2,515 experiments versus 21 for M. tuberculosis — so effort scores computed from such metadata directly encode which negatives deserve distrust.

The third ingredient is the audit, not the model. Every reported result will carry a slice-wise credible interval in the SAFT style: Dirichlet-multinomial intervals for small slices, Wald-type intervals for large ones, with the switch point set by the interval procedure rather than an arbitrary cutoff. Headline metrics will be worst-supported-slice F1 and PR-AUC, never aggregates alone.

Baselines

The do-nothing reference is the unweighted fine-tune of the genomic language models exactly as specified in Hidden challenges: same 140,638 sequences, same 26 families, same temporal split (training on data up to 2017, testing on 2018 onward), same 250-base-pair fragments and 4-mer tokenization. That paper reports the highest PR-AUC in 18 of 26 families for BERT-infect and an F1 of 0.000 on SARS-CoV-2-related sarbecoviruses and H5 influenza A; no reported result found on Lacuna for a numeric aggregate PR-AUC value, so the comparison on aggregate performance will be qualitative (families won) plus our own reproduced numbers. The strongest existing competitor is FairGrad-style Lagrangian reweighting transplanted from FairGrad: per-slice loss weights updated from constraint duals during the same fine-tuning, given the identical backbone, data, splits, tuning budget, and compute, since that method scales cheaply (its authors report the fair ResNet18 training cost only 0.42 seconds more per epoch than unconstrained training) and directly targets slice-level error rates without dropping any slice. Beating it means a higher lower confidence bound on worst-supported-slice F1 on the temporal test split, with aggregate family-level PR-AUC not falling outside the unweighted model's credible band on more than a small minority of families. DRAF from Doubly-Regressing Approach for Subgroup Fairness will also be run with its support threshold γ, but it is expected to lose by construction on worst-slice metrics because its own doctrine excludes the sparsest slices from protection. Two ablations isolate the key ingredients: the same architecture with an exchangeable Gaussian prior (S-EFE-style shrinkage toward the global mean, no tree covariance), which isolates the phylogenetic structure; and the same model with $\pi_g$ fixed to 1, which isolates the censoring-aware negative weighting.

Why not just…?

Why not just evaluate at the viral-family level, where data is plentiful? Because that is precisely the blind spot already demonstrated: Hidden challenges shows a model that wins at the family level while scoring 0.000 on the two most consequential lineages, and both Doubly-Regressing Approach for Subgroup Fairness (fairness gerrymandering) and On the Epistemic Limits of Personalized Prediction (worsenalization) show that population-level metrics can coexist with worst-group failure. The whole point of the proposal is that the aggregate number is the wrong statistic.

Why not just drop slices with fewer than N samples, like DRAF does? Fixed-threshold filtering is the standard engineering answer, but it deletes exactly the cells of highest concern: rare wildlife hosts and marginal livestock are where unmonitored spillover originates, as in the Eld's deer survey in Prevalence and molecular characterization of Enterocytozoon bieneusi in endangered Eld's deer, where the zoonotic ITS genotypes that matter have counts of $n=19$, $n=10$, $n=5$, and fewer. Size-adaptive Hypothesis Testing for Fairness further shows fixed thresholds are statistically brittle near the boundary. Dropping is not an option here; shrinking and borrowing is the alternative.

Why not just oversample or re-weight rare slices? This is the real threat, and I will not argue around it: inverse-frequency reweighting is cheap, strong, and will be run head-to-head as the FairGrad baseline. Two pieces of evidence say it should fall short. FairGrad proves some parity objectives are unreachable with positive weights alone, and Doubly-Regressing Approach for Subgroup Fairness shows that forcing strict equality on tiny groups overfits their noise. More specifically, reweighting amplifies censored negatives — it up-weights exactly the under-sampled cells where "animal-only" labels are least trustworthy — whereas the $\pi_g$ term down-weights them, and the $\pi_g = 1$ ablation will show how much of any gain comes from this alone. If plain reweighting matches tree-structured shrinkage on worst-slice F1 at equal aggregate cost, that is a finding I will report: it would mean the phylogeny carries no signal beyond group identity, and the proposal's value would reduce to the censoring fix and the audit protocol.

Proposed Experiments and Decisive Checks

The main experiment uses the 140,638-sequence, 26-family benchmark of Hidden challenges with its temporal split, which is the honest test: models train on the world as known up to 2017 and are judged on what emerged after. Headline numbers are worst-supported-slice F1 and PR-AUC over host × viral-clade × region slices, each reported with SAFT-style credible intervals; the sarbecovirus and H5 influenza slices are the designated hard cases, where the reference score is 0.000. Success at the model level is a positive lower confidence bound on F1 for those slices, beating the FairGrad baseline's bound. Success at the audit level is calibration: on well-supported slices where repeated subsampling gives ground truth, the 90% intervals must cover true slice performance at least 90% of the time — if they do not, the audit itself is unsound and the paper says so. A field demonstration applies the audit machinery to small-cohort surveillance data, using the Eld's deer dataset (217 animals, overall prevalence 17.5%, per-genotype counts as small as 5) to show concretely which genotype-level claims are and are not statistically defensible — an illustration that fixed-threshold reporting would either discard or overclaim. Finally, the two ablations settle the mechanism: if tree-covariance shrinkage beats exchangeable shrinkage, phylogenetic borrowing earns its place; if not, the honest conclusion is that the fix lies in censoring-aware labels and interval-valued evaluation, which is itself a usable contribution. I am explicit about the ceiling: given the temporal split, slices absent from training can only be reached through their relatives, and the minimax limits of On the Epistemic Limits of Personalized Prediction caution against promising certified performance on slices with near-zero support. The proposal's falsifiable claim is the relative one: tree-structured borrowing, censored negatives, and size-adaptive audits together reduce worst-slice failure relative to every simpler alternative at matched aggregate performance — or the experiments tell us which of those three ingredients is doing the work and which is not.