When a genome-wide association study (GWAS) finds a region linked to a disease, it usually cannot say which DNA variant is responsible. Neighboring variants are inherited together, so dozens of them show nearly identical statistical signal. Statistical fine-mapping, the step that assigns each variant a probability of being causal, breaks these ties with prior beliefs about which variants are likely to matter. Today those priors come from coarse hand-made labels such as "coding," "in an open-chromatin peak," or "in a KEGG pathway." Separately, genomic language models (sequence models pre-trained on DNA rather than text) now produce rich per-variant representations. These come either from evolutionary alignments, as in GPN-Star, or from human variation, as in UKBioBERT. These representations are evaluated on trait-agnostic tasks like pathogenicity or constraint. They are never used to set the prior inside a fine-mapping model.
We propose to learn, for each trait, a small map from frozen language-model variant embeddings to a prior causal probability. The map is trained only by maximizing how well the fine-mapping model explains GWAS summary statistics across many traits, with the map's weights shared through a low-rank trait factorization. The bet is that the embedding contains several separable kinds of biology, such as protein disruption, promoter grammar, and enhancer motifs. Different traits weight these differently, and a single conservation score cannot express that difference. The bet would be falsified if adding the language model's scalar score as one more annotation column performs as well, and the experiments are built around that comparison.
Core Claim
Evolutionary language models measure purifying selection, which is trait-agnostic. Many late-onset or environment-shaped risk alleles sit in regions that selection tolerates. This is the tension the file identifies between GPN-Star-style constraint and the trait-specific enrichments found by LDpred-funct. Our claim is that a trait-specific linear readout of a frozen high-dimensional embedding, fitted through the fine-mapping likelihood itself, recovers trait-relevant functional directions that neither a constraint scalar nor categorical annotations capture. The result should be smaller credible sets and causal calls that replicate better in other ancestries.
Concretely, let $e_j$ be the frozen embedding difference (alternate minus reference allele) for variant $j$. Let $u_t$ be a learned low-dimensional vector for trait $t$, and let $W$ be a shared projection matrix. The prior probability that variant $j$ is causal for trait $t$ is
$$\pi_{tj} = \sigma\big((W u_t)^\top e_j + b_t\big),$$
where $\sigma$ is the logistic function and $b_t$ is a trait-specific offset. The parameters $(W, u_t, b_t)$ are chosen to maximize
$$\sum_{t}\sum_{\ell} \log p\big(\hat z_{t\ell} \mid R_\ell, \pi_{t\cdot}\big),$$
the summed log marginal likelihood of the observed association z-scores $\hat z_{t\ell}$ at locus $\ell$, given the locus's reference correlation (LD) matrix $R_\ell$. The likelihood comes from a sum-of-single-effects fine-mapping model on summary statistics, the same model family SparsePro builds on. The shared $W$ lets traits with few loci borrow strength. The trait-specific $u_t$ lets type 2 diabetes and schizophrenia read different directions of the same embedding.
Relation to Prior Work
Two papers motivate the gap. SparsePro already learns annotation enrichment weights from summary statistics, but only over a handful of categorical annotations, selected with a G-test. We keep its summary-statistic-only philosophy and replace those categories with a continuous embedding that is learned across traits. Multiple instance fine-mapping (MIFM) is the closest existing work. It trains a convolutional sequence model from scratch on more than 13,000 GWAS, treating associated linkage blocks as weakly labeled "bags" and max-pooling over them, and reports a 17% relative improvement in African-ancestry polygenic score performance. We differ in three ways. We use the proper fine-mapping likelihood with LD instead of bag labels and max-pooling, which avoids MIFM's assumption that every associated block contains a detectable sequence-disrupting variant. We reuse pre-trained transformer embeddings rather than a 512-bp convolutional model. And we produce explicitly trait-specific priors rather than one trait-pooled score.
Three papers supply the representations. GPN-Star provides alignment-aware constraint embeddings, LOL-EVE provides promoter-specific embeddings that handle insertions and deletions, and UKBioBERT provides embeddings aware of human variation. None of the three connects to fine-mapping or to polygenic risk machinery.
Two results constrain the design. UKBioBERT's "fog of multi-gene training" shows that pushing pathway structure into the language model itself did not help, so we keep the language model frozen and put trait specificity in a light readout. PRSbils needed individual-level validation data to fit group weights and saw inconsistent gains in the Korean KoGES cohort. That is a warning that annotation weights can overfit one ancestry, so we test transfer across ancestries directly. SSLPRS names functional-annotation extension as open work. Our priors could later feed its spike-and-slab mixture, but that is downstream of this proposal.
Research Program
We first precompute embedding differences for roughly ten million common and low-frequency variants, using each of three frozen models: GPN-Star at the mammal and primate timescales, LOL-EVE for variants in promoters, and UKBioBERT. We reduce each to a few hundred dimensions with principal components fitted on held-out chromosomes only.
We then fit the prior map on publicly available GWAS summary statistics for several dozen UK Biobank traits, using European LD reference panels. All variants on even chromosomes are held out from fitting. Fitting alternates two steps. In the first, we run fine-mapping per locus under the current priors. In the second, we take gradient steps on $(W, u_t, b_t)$ using the expected causal assignments, which is an empirical-Bayes loop analogous to SparsePro's but with a neural prior.
Finally, we analyze what each trait vector $u_t$ reads. We project trait directions onto known annotation categories and onto motif-disruption scores. This tests whether, for example, late-onset traits load less on deep-vertebrate constraint, which would directly address the evolutionary-versus-trait tension.
Baselines
The naive reference point is a flat prior: summary-statistic fine-mapping with SuSiE and uniform causal probabilities. Any method should beat this in credible-set size. On SparsePro's simulations, SuSiE reaches a locus-level area under the precision-recall curve (AUPRC) of 0.87, against 0.98 for SparsePro+ (SparsePro).
The strongest annotation-based competitor is SparsePro itself, run with its standard categorical annotations (DNase hypersensitivity, conservation, nonsynonymous status, and tissue histone marks). It is the strongest because it already learns trait-specific weights from summary statistics alone, which is exactly our setting. We run it on the identical loci, LD panels, and held-out chromosome split. We count a win only if we beat its 0.98 simulated AUPRC on simulations not generated from our own embeddings, and outperform it on the real-data replication metrics below.
The strongest sequence-model competitor is MIFM, run as released on the same traits, with its variant scores used as a fine-mapping prior. On the cross-ancestry polygenic score task, its reported 17% relative improvement in African-ancestry cohorts (MIFM) is the bar.
For polygenic prediction, LDpred-funct is the reference. It reports an average prediction $R^2$ of 0.144 across 21 UK Biobank traits and a 4.6% relative gain over SBayesR (LDpred-funct). We would need to exceed it on the same traits when our priors are plugged into the same shrinkage model.
The key ablation replaces the embedding $e_j$ with the language model's single zero-shot score, appended as one annotation. A second ablation removes trait specificity by tying all $u_t$ to a shared vector. Every arm gets the same hyperparameter search budget and the same loci.
Why not just…?
Why not just add the GPN-Star or LOL-EVE score as one more annotation in SparsePro? This is the real threat. A single conservation score may carry most of the useful signal, and if so our method adds complexity for nothing. The scalar-score ablation settles it directly. If the scalar matches the full embedding on held-out chromosomes and on cross-ancestry replication, we will report that adding a scalar annotation is sufficient and drop the embedding claim. The case for the embedding rests on traits whose causal variants fall in regions that selection tolerates. We will therefore report the comparison separately for late-onset traits such as Alzheimer's disease and for developmental or early-onset traits.
Why not just use MIFM, or retrain it with a transformer backbone? MIFM's bag labels assume each associated block contains a variant visible within 512 bp. It also pools all traits into one score, so it cannot express that a variant matters for lipids but not for height. Our trait-tied ablation is effectively the pooled version of our own method; if it matches the full model, MIFM-style pooling is adequate. Retraining a transformer end to end on 13,000 GWAS costs far more compute than fitting a frozen readout, and UKBioBERT's multi-gene result suggests pushing trait structure into the backbone is fragile.
Why not just use richer categorical annotations, such as the full baseline-LD set? LDpred-funct and PRSbils show that curated categories do help. However, they cannot represent sequence context, for example which motif an allele disrupts. We include a baseline-LD-annotated SparsePro arm, so if categories suffice, the result will show it.
Proposed Experiments
The decisive checks avoid circular ground truth, because no experimental causal labels exist at scale.
The first check is held-out-chromosome likelihood. The fitted priors should raise the marginal likelihood of summary statistics on even chromosomes relative to SparsePro and the scalar ablation.
The second check is cross-ancestry replication. We fine-map in European summary statistics and ask how often the top-probability variant in each credible set is also the lead signal in independent East Asian or African-ancestry GWAS, where LD differs. This is the setting where XMAP (XMAP) and Cai et al. 2023 show European LD misleads. Better priors should replicate more often, because they rely on sequence rather than LD.
The third check is external molecular validation. We use LOL-EVE's causal-eQTL benchmark, where LOL-EVE reports a normalized AUPRC of 1.49 and Cohen's d of 0.31 (LOL-EVE), and ask whether our expression-trait priors rank causal eQTLs above that. For GPN-Star's ClinVar and OMIM non-coding comparisons, no reported result was found on Lacuna for GPN-Star's numeric scores.
The fourth check is polygenic prediction. We plug the priors into LDpred-funct-style shrinkage and compare $R^2$ on UK Biobank traits and in non-European cohorts against the 0.144 average and MIFM's 17% bar.
The fifth check is a trait-specificity test. The full model should beat the trait-tied ablation most on traits that selection plausibly tolerates, and its trait vectors should load less on deep-vertebrate constraint for those traits. If neither difference appears, the central claim fails.
References
- Predicting functional constraints across evolutionary timescales with phylogeny-informed genomic language models
- Pre-Training Genomic Language Model with Variants for Better Modeling Functional Genomics
- Incorporating functional priors improves polygenic prediction accuracy in UK Biobank and 23andMe data sets
- SparsePro: An efficient fine-mapping method integrating summary statistics and functional annotations
- Multiple instance fine-mapping: predicting causal regulatory variants with a deep sequence model
- A Genomic Language Model for Zero-Shot Prediction of Promoter Variant Effects
- Incorporating functional annotation with bilevel continuous shrinkage for polygenic risk prediction
- Sparse polygenic risk score inference with the spike-and-slab LASSO
- XMAP: Cross-population fine-mapping by leveraging genetic diversity and accounting for confounding bias