Training Sequence Models on Bulk Tissue by Modeling the Cell Mixture

Most measurements of how DNA variants change gene expression come from bulk tissue: a chunk of brain or blood is ground up and every cell's RNA is counted together. The cell mix differs from donor to donor. One person's sample is 30% microglia, another's 10%. That variation is usually treated as a nuisance. It is also information. If a variant changes expression only in microglia, its apparent effect in bulk should grow and shrink with the microglia fraction. We propose to exploit this directly. We fine-tune a DNA sequence-to-expression model, a neural network that reads raw genome sequence and predicts gene activity, with one output per cell type. We train it on thousands of genotyped bulk tissue samples. The model's cell-type-specific predictions are combined through the physical mixture equation, weighted by each donor's estimated cell proportions, before they are compared to the observed bulk read counts.

The payoff is a variant-effect predictor that learns cell-type resolution from bulk cohorts, which are roughly an order of magnitude larger than single-cell genetic cohorts. Because it predicts from sequence, it also scores rare or unseen variants that association methods cannot test. Two supervision signals are combined. The first is total read counts across genotypes. The second is allele-specific reads: within a donor who carries two different versions of a variant, the ratio of reads from each copy is an internally controlled measurement, and in a mixture that ratio is itself a proportion-weighted blend of per-cell-type ratios.

Core Claim

Current sequence models inherit the confounding in their training data. Models such as AlphaGenome, which reads 1 Mb of sequence and predicts 5,930 human tracks across 11 assay types, are trained largely on bulk tracks, so a predicted "brain" effect blends cell-intrinsic regulation with cell abundance. The AlphaGenome authors themselves list cell-type- and condition-specific variant effects as open.

Statistical methods that do resolve cell types from bulk share a different limitation: they estimate one effect per observed variant and never read sequence. CSeQTL jointly models total counts (negative binomial) and allele-specific counts (beta-binomial) with genotype-by-proportion interactions. SPLITR/deQTL deconvolves 3,387 brain samples and reports 4,757 cell-type-specific eQTLs. EPIC-unmix infers per-donor cell-type expression via an empirical Bayes update of single-cell priors.

Our claim is that the count-level mixture likelihood from CSeQTL can serve as the training loss of a sequence model. Doing so has two effects. It shares statistical strength across all variants through sequence. It also yields cell-type-resolved effects that are more accurate for weakly identified and rare cell types than a two-stage pipeline of "estimate cell-type eQTLs, then train on them." The closest partial precedent is Count Bridges, which predicts cell-type expression from sequence on single-cell data and separately deconvolves bulk. It does not use genotype-dependent or allele-specific counts, and it does not train a sequence model through the bulk mixture.

Research Program

For donor $i$, gene $g$ and cell type $k$, let $w_{ik}$ be the estimated cell fraction, $c_k$ the relative mRNA content per cell, $\mu_{gk}$ a learned cell-type baseline, $s_i$ the library size, and $f_\theta(h,k)$ the model's log-expression output for haplotype sequence $h$ in cell type $k$. The content term $c_k$ matters: SPLITR found neurons carry about 4× the mRNA of glia. Expected bulk counts are then summed on the linear scale:

$$\mathbb{E}[y_{ig}] = s_i \sum_k w_{ik}\, c_k\, \mu_{gk}\, \tfrac12\big(e^{f_\theta(h^{(1)}_{ig},k)} + e^{f_\theta(h^{(2)}_{ig},k)}\big),$$

Here $h^{(1)}, h^{(2)}$ are the donor's two phased haplotypes around the gene. We fit this with a negative binomial likelihood.

For heterozygous donors, the expected fraction of allele-specific reads from haplotype 1 is

$$\pi_{ig} = \frac{\sum_k w_{ik} c_k \mu_{gk} e^{f_\theta(h^{(1)}_{ig},k)}}{\sum_k w_{ik} c_k \mu_{gk}\big(e^{f_\theta(h^{(1)}_{ig},k)} + e^{f_\theta(h^{(2)}_{ig},k)}\big)},$$

We fit this with a beta-binomial likelihood.

Summing on the linear scale is deliberate. The file describes a "transformation trap": fitting the mixture on log counts inflates false positives, which CSeQTL identifies as the source of "leakage" of abundant-cell eQTLs into rare cell types. The per-cell-type variant effect is the difference $f_\theta(\text{alt},k) - f_\theta(\text{ref},k)$.

Cell proportions come from a SPLITR-style, platform-corrected deconvolution and are treated as fixed inputs, with perturbation analyses described below.

To keep compute tractable, the model has a frozen pretrained trunk (Enformer- or AlphaGenome-class) and a trainable cell-type head plus low-rank adapters. Trunk embeddings are computed once per unique haplotype window, and unique haplotypes are far fewer than donors.

Before training, we compute a per-cell-type identifiability score from the donor-by-donor covariance of $w_{ik}$. Cell types whose fractions are too collinear to separate, for example closely related neuronal subtypes, are merged. We state in advance that a type can only be resolved if its abundance actually varies across donors.

Data come from three places. The first is GTEx blood and brain, including allele-specific counts (Lonsdale et al.). The second is ROSMAP/Mayo/MSBB brain, where snRNA-seq references support deconvolution. The third is OneK1K, 982 donors with single-cell RNA-seq across 14 immune types, which serves as a semi-synthetic ground truth.

Baselines

The naive reference point is the pretrained sequence model's variant score on the matching bulk tissue track, applied identically to every cell type. This is what practitioners use today. On GTEx eQTL sign prediction, AlphaGenome reports an improvement from 75% to 80% AUROC (AlphaGenome). A stronger version of doing nothing uses the pretrained model's closest cell-type or cell-line track where one exists.

The strongest existing competitor is a two-stage pipeline. CSeQTL is run with the same deconvolved proportions to estimate per-cell-type eQTL effects, and the same frozen trunk and head are then fine-tuned on those estimates, weighted by their standard errors. This is the most natural objection to our approach: it uses the identical count likelihood, only decoupled from the sequence model. A second two-stage variant regresses on EPIC-unmix per-donor cell-type profiles instead. Both use the same donors, the same train/test chromosome split, the same trunk, and the same hyperparameter search budget as our model.

CSeQTL alone, with no sequence model, is also reported on variants it can test. It reports over 90% directional consistency with independent single-cell studies (CSeQTL).

Three ablations isolate the key ingredient. The first shuffles proportions across donors, removing the composition signal while keeping everything else. The second replaces the linear-scale mixture with a log-scale additive mixture, reproducing the transformation trap. The third drops the allele-specific term.

Beating the two-stage pipeline means higher correlation with held-out single-cell eQTL effects, averaged over cell types and specifically for the lowest-abundance identifiable types. The comparison must be on held-out chromosomes, and also on variants too rare in the cohort for CSeQTL to test. On OneK1K we also report an oracle ceiling: the same model trained directly on true per-cell-type pseudobulk.

Why not just…?

Why not just train on single-cell eQTL data like OneK1K directly? Where large single-cell genetic cohorts exist, that is the ceiling, and we measure our gap to it. Such cohorts are mostly blood, around a thousand donors, and far thinner for brain and other solid tissues. Bulk cohorts like GTEx and ROSMAP are larger and carry allele-specific reads. The question is whether bulk supervision gets close to the single-cell ceiling. If it cannot on OneK1K pseudobulk, the approach has no reason to exist in tissues without single-cell cohorts.

Why not just deconvolve first with CSeQTL or EPIC-unmix, then fine-tune on the estimates? This is the real threat. Our argument is specific. Per-variant estimates in rare cell types are noisy and heavily shrunk; EPIC-unmix, for example, is restricted to high-agreement marker genes and degrades on lowly expressed genes. End-to-end training instead lets every heterozygous donor's allele-specific reads constrain shared sequence parameters. If the two-stage baseline matches us on rare types and held-out variants, we will report that it suffices and that joint training is unnecessary.

Why not just weight the pretrained model's cell-line tracks by estimated proportions, as GRAM does with TF expression? GRAM weights DeepBind TF-binding changes by TF expression and reaches 0.72 AUROC on GM12878 MPRA data. That expression weight is itself composition-confounded when it is taken from bulk tissue. Cell lines also often miss the primary cell states of interest, such as microglia. The "closest cell-type track" baseline tests this route directly.

Why not just use Count Bridges? Count Bridges outperformed a fine-tuned Enformer on OneK1K sequence-to-expression prediction, but the file reports no number. It has not been trained on genotype-dependent or allele-specific counts, or evaluated on variant effects. Its per-cell-type outputs could be dropped into our mixture likelihood as an alternative head, and we would treat it as a compatible component rather than a rival.

Proposed Experiments

The first experiment uses OneK1K as semi-synthetic truth. We build pseudobulk samples by summing each donor's cells, so the proportions and the true per-cell-type genotype effects are both known. We train only on the pseudobulk. We then score per-cell-type effects against single-cell eQTL estimates on held-out chromosomes, stratified by cell-type abundance and variant frequency. This test is limited to total counts, because 3′ single-cell data yield few allele-specific reads; we say so explicitly. The decisive check is whether our model beats the two-stage pipeline for cell types under about 5% abundance and for rare variants. It must also not beat the shuffled-proportion ablation by chance.

The second experiment trains on GTEx blood and brain bulk with allele-specific counts. We evaluate against purified-cell BLUEPRINT profiles, already used for validation in CSeQTL, and against brain single-nucleus eQTLs. The target is sign concordance and effect-size correlation. We also report GTEx eQTL sign AUROC against AlphaGenome's reported 80% as a sanity check that bulk-level accuracy is not sacrificed.

The third experiment tests disease-variant prioritization on TraitGym, with 1,140 complex-trait causal variants each matched to 9 decoys. The file reports CADD at 0.85 AUPRC on the Mendelian subset. It reports no complex-trait AUPRC value, only that Enformer and Borzoi lead on complex non-disease traits and that models struggle on enhancer variants. We therefore rerun the pretrained backbone ourselves as the reference. We test whether taking the maximum over cell types of our cell-resolved scores raises AUPRC on distal complex-trait variants. The comparison includes a positive control: the AP3B2 microglia eQTL that EPIC-unmix found hidden in bulk should emerge as microglia-specific.

The final experiment checks robustness. Proportions and mRNA content factors are perturbed by realistic deconvolution error, and we report how much each cell type's effect estimates drift. A resolved cell type is only claimed where these estimates are stable.