Deep networks for genetic risk prediction beat the standard additive polygenic risk score inside the cohort they were trained on. The additive score simply sums each variant's allele count times a small weight. The open question is whether these networks gain because they capture epistasis, meaning gene–gene interactions where one variant's effect depends on another's, or because they memorize cohort-specific quirks. One suspect is linkage disequilibrium (LD): the local correlation between nearby variants that differs between ancestries and genotyping arrays. A model that looks at a window of neighboring variants can learn "interactions" that are really this cohort's LD pattern, and those patterns will not carry over to a new population. Interactions between distant regions of the genome are more likely to reflect biology. We therefore propose a risk model with two parts. A deliberately low-capacity local encoder summarizes each LD block into a single token, and a shared interaction module lets every block token attend to every other, across chromosomes. At transfer time only the local encoders are re-adapted to the new cohort.
The claim is falsifiable, and it is new in how it divides the labor. Existing work widens the model's view to capture more interactions. We instead treat the local view as the place where cohort-specific signal lives, and the long-range view as the part expected to transfer. If the claim holds, freezing the interaction module and adapting only the block encoders should keep most of the within-cohort gain on an ancestrally different external cohort. It should also do better than retraining everything. If the claim fails, the non-additive gains reported for genomic transformers are mostly not transferable epistasis. That would be an important negative finding in its own right.
Motivation from Prior Work
Three lines of work motivate the gap. Inferring genotype-phenotype maps using attention models shows that self-attention over 1,000 yeast loci captures high-order epistasis with about 25,000 parameters, against roughly 500,000 for explicit pairwise models, and transfers across 18 environments. Its full attention does not scale to human panels, and it tests environmental shift rather than ancestry shift. HEMERA scales attention to 378,866 SNPs with a Linformer, a low-rank approximation that makes attention cost linear, and reports AUC 0.9932 within a single matched Million Veteran Program cohort. For a trait whose heritability is modest, that figure is more plausibly a shortcut than germline epistasis. HEMERA also never tests transfer. MetaGeno restricts attention to within each chromosome and reaches AUROC 0.809 for ischemic stroke on UK Biobank, against 0.650 for a traditional PRS and 0.778 for LightGBM. It validates on All of Us, but no AoU number is available in our material ("no reported result found on Lacuna for MetaGeno on All of Us"). It notes that ancestry composition affects performance. Its design also cannot represent any interaction between chromosomes.
Two results show that epistasis hides from marginal screens and so constrain the design. The marginal epistasis test shows strongly interacting variants often have negligible marginal signal. Complete compositional epistasis detection shows XOR-like interactions have zero marginal slope. Together they rule out pre-filtering by marginal p-value, which the Alzheimer's feedforward model relies on. That model reaches auROC 0.84 against 0.70 for wPRS.
The method borrows from two sources:
- From cross-dataset generalization, it takes the idea of splitting a representation into a domain-specific branch and an invariant branch. The two-branch anti-spoofing model factors out a camera fingerprint in this way. Here the "fingerprint" is local LD structure. We adapt that branch rather than discard it, because it also carries real additive signal.
- From the receptive-field literature, it takes the lesson of CCNet and Guanbin Li et al.: global context should be routed through a cheap intermediate step rather than by stacking local layers. Our intermediate step is the LD-block token.
With a few thousand block tokens, dense attention becomes affordable. We therefore do not need criss-cross factorization at the block level, and we say so plainly rather than bolt it on.
Core Claim
Partition the genome into LD blocks $b = 1, \dots, B$, with $B$ in the low thousands. Let $x_b$ be the genotypes in block $b$. The model is
$$\hat{y} = g_\theta\big(h_{\phi_1}(x_1), \dots, h_{\phi_B}(x_B)\big)$$
The terms are defined as follows:
- $h_{\phi_b}$ is the local encoder for block $b$. It is restricted to a within-block additive term plus a small rank-limited haplotype term.
- $g_\theta$ is a transformer over block tokens with full cross-chromosome attention.
- Ancestry principal components enter both the model and all baselines as covariates.
The claim has three parts:
- Moving from a source cohort to an ancestrally different target mainly degrades $h_\phi$.
- Keeping $\theta$ fixed and re-fitting only $\phi$ recovers most of the source-cohort advantage over PRS. The re-fit uses either a small labeled target sample or label-free re-standardization of block tokens with target genotype statistics.
- The retained advantage comes from the inter-block, non-additive part of $g_\theta$.
Research Program
We will first build the model and a decomposition diagnostic. We fit the best additive model to the network's own predictions on held-out individuals. The variance left over is the model's non-additive component, which we split into within-chromosome and between-chromosome parts by masking attention. The key quantity is how much of this non-additive component survives the move from UK Biobank to All of Us.
Second, we run a semi-synthetic ground-truth study. We use real UK Biobank and All of Us genotypes, so the real LD differences between ancestries are kept, and simulate phenotypes with planted additive effects plus AND and XOR interactions. The planted pairs are placed both within and across chromosomes and chosen to have near-zero marginal signal. This tells us which architectures recover which interactions, and whether they still hold up when the ancestry changes.
Third, we run a real-trait study:
- Ischemic stroke, matched to MetaGeno's setup, trained on UK Biobank European participants and evaluated on All of Us.
- Left ventricular mass, as a UK Biobank-internal trait with a validated interacting pair. The cardiac hypertrophy study confirmed non-additivity of CCDC141×IGF1R experimentally, with an 8.5% cell-size decrease on double silencing. We test whether our interaction module assigns this pair non-additive credit and whether that credit is stable across ancestry subsets.
The yeast segregant data from the attention-model paper serves as a small, clean check. Across its environment shift, the blocked model should match full per-locus attention; the file reports no numeric $R^2$ for the full-attention model.
Baselines
The naive reference point is an additive PRS with ancestry principal components, fitted on the same SNPs. MetaGeno reports this PRS at 0.650 AUROC on UK Biobank stroke. We re-fit it ourselves for the transfer comparison.
LightGBM on the same SNPs is the strongest non-deep competitor, reported at 0.778 by MetaGeno. Tree ensembles find interactions without any architecture choices. It gets the same features, splits and hyperparameter-search budget as our model.
The strongest existing method is MetaGeno's chromosome-wise transformer, reported at 0.809 AUROC. We reimplement it on the same SNP set, splits and GPU-hour budget. We compare against it both at its reported within-UK Biobank number and on All of Us, where we must generate the number ourselves. A HEMERA-style Linformer over all SNPs is included because it represents "just widen the receptive field to the whole genome." Its in-cohort score is expected to be high; its transfer score is the question.
Two ablations isolate the key ingredient. The first gives the local encoder full capacity, meaning an unrestricted per-block network. The second fine-tunes the whole model on the target rather than only $\phi$. A third ablation limits $g_\theta$ to within-chromosome attention, which tests whether cross-chromosome routing matters.
Beating MetaGeno means two things. First, a higher All of Us AUROC at matched labeled-target budget, with a bootstrap confidence interval that excludes zero difference. Second, a smaller source-to-target drop in the non-additive variance component.
Why not just…?
Why not just use MetaGeno or a whole-genome Linformer and fine-tune on the target cohort? This is the most serious threat. If full fine-tuning on a few thousand labeled All of Us individuals matches our local-only adaptation, then our split adds nothing beyond the extra parameters it adapts. The full fine-tuning ablation settles this at several target sample sizes. We expect our method to win mainly in the low-label regime, where full fine-tuning overfits. If it does not win there, we drop the claim.
Why not just use gradient-boosted trees, which already capture interactions? LightGBM trails MetaGeno within UK Biobank by 0.778 to 0.809. It has no explicit mechanism to separate local tagging from long-range interaction, so its splits can latch onto LD-specific proxies. The semi-synthetic study measures this directly: if trees recover the planted cross-chromosome XOR pairs and transfer as well as our model, the architecture is unnecessary.
Why not just add pairwise terms found by lo-siRF or marginal-epistasis screening to a PRS? Explicit screening is interpretable and experimentally validated in the cardiac hypertrophy study. However, testing every pair explodes combinatorially, and screens that filter on marginal signal miss XOR-like pairs. We include "PRS plus screened pairs" in the semi-synthetic study. If it recovers the same planted interactions at lower cost, the honest conclusion is that the neural interaction module is only worth using when interactions are higher-order or diffuse.
Decisive Checks
Three negative controls guard against the HEMERA-style shortcut:
- Train on permuted phenotypes; AUROC should be near chance.
- Predict ancestry principal components from block tokens after adaptation; this should fail.
- Test whether the non-additive component shrinks when array-batch indicators are regressed out.
The central test is whether freezing $g_\theta$ and adapting only $\phi$ keeps the UK Biobank non-additive variance and AUROC gain over PRS on All of Us better than every baseline, with cross-chromosome attention accounting for part of it. If the retained non-additive component falls to near zero for all models, including ours, we report that transferable epistatic gain is not supported at current sample sizes.
References
- Inferring genotype-phenotype maps using attention models
- HEMERA: A Human-Explainable Transformer Model for Estimating Lung Cancer Risk using GWAS Data
- MetaGeno: a chromosome-wise multi-task genomic framework for ischaemic stroke risk prediction
- Detecting epistasis with the marginal epistasis test in genetic mapping studies of quantitative traits
- The complete compositional epistasis detection in genome-wide association studies
- Deep learning-based polygenic risk analysis for Alzheimer’s disease prediction
- Camera Invariant Feature Learning for Generalized Face Anti-Spoofing
- CCNet: Criss-Cross Attention for Semantic Segmentation
- Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
- Epistasis regulates genetic control of cardiac hypertrophy