Budgeted Metabolomics: Choosing Whom to Profile to Recover Missing Heritability

A large share of the heritability of complex traits is still unexplained by the genetic variants that studies have found. Part of the gap sits in rare variants whose effects only become visible through intermediate molecular measurements, such as the levels of blood and urine metabolites. Those measurements are expensive. Many biobanks have exome sequence for hundreds of thousands of people, but they can afford untargeted metabolomics for only a fraction of them. This proposal treats the choice of whom to profile as a sequential decision problem under a fixed, short budget.

Two facts make the decision unusually well posed. Everyone's genotype is known before any metabolite is measured. And a single person who carries rare damaging variants in several candidate genes is informative about many gene–metabolite hypotheses at once. We propose a batched allocation rule built on the pessimism principle from structured bandits: each round, profile the people best placed to rule out the most threatening unresolved hypothesis. We pair the rule with a finite-budget lower bound that says, gene by gene, whether any design could have detected the effect.

The practical output is twofold. First, a method that should recover most full-cohort discoveries at a small fraction of the profiling cost. Second, a way to split missing heritability into a part that a better design can recover and a part that no design can recover at a given budget.

Core Claim

Finite-time regret analysis studies how much a sequential learner loses over an explicit, short horizon rather than in the limit. Its central lesson, from Garivier et al., is that loss first grows linearly with the horizon, and only later settles into the efficient logarithmic regime. Dani et al. show that how long the linear phase lasts grows as the gap between hypotheses shrinks.

Rare-variant metabolite studies sit squarely in that unfavorable regime: effects are modest, carriers are few, and there are only a handful of acquisition batches. Our claim is that two things change the picture.

The first is known genotypes. Because genotypes act as fully observed contexts, the problem becomes a design problem with a known information structure, close to the "informative arm" setting in which Jun and Zhang show pessimistic allocation escapes regret that grows with the number of arms.

The second is the lower bound. Garivier's KL-divergence inequality, applied per gene, gives the minimum number of profiled carriers needed to tell an effect above a threshold apart from no effect. KL divergence here is simply a measure of how distinguishable two hypotheses are from data. When that minimum exceeds the number of carriers who actually exist in the cohort, the effect's heritability is missing for information reasons, not design reasons.

Where This Sits

The motivating evidence comes from Scherer et al.. They paired exome sequencing with untargeted metabolomics (1,294 plasma and 1,396 urine metabolites) and found 235 gene–metabolite associations driven by rare damaging heterozygous variants. They profiled the whole cohort in one static batch, so their study says nothing about how many profiles those discoveries required.

Pathway and network aggregation (Luo et al. build module-level risk scores from protein interaction networks and brain imaging) is in the same spirit. It pools weak signals, but it does so on fixed cohorts and never decides what to measure next. We reuse its pooling idea as a prior, not as a static pipeline.

On the sequential side, two papers apply bandits to health and resource allocation:

  • BCoR (Liang et al.) uses hierarchical Bayesian Thompson sampling to schedule interventions.
  • The thresholding reformulation of Xu et al. shows that short-horizon performance hinges on deciding whether a quantity crosses a threshold, not on estimating it exactly.

Both optimize cumulative reward over demographic covariates. Neither handles high-dimensional genotypes or a discovery objective. We borrow Xu et al.'s thresholding view and BCoR's sharing of information across related units.

Takeno and Iwazaki prove that Gaussian-process Thompson sampling has a polynomial, not logarithmic, failure-probability tail. That result constrains our design: a one-shot study cannot afford a method that occasionally fails badly.

Finally, CROP's guarantees hold only for finite hypothesis classes. That is the open problem we address, because thresholding each gene–metabolite effect reduces each test to a binary hypothesis.

Research Program

The model treats each candidate gene $g$ and metabolite $m$ as having a rare-variant burden effect $\theta_{gm}$. Each person $i$ contributes a known burden vector $x_i$ (damaging-variant counts per gene) and, once profiled, a metabolite vector $y_i$. The discovery target is the set of pairs with $|\theta_{gm}| \ge \gamma$, where $\gamma$ is a clinically or statistically meaningful threshold.

A hierarchical prior shares strength across metabolites in the same biochemical pathway and across genes in the same interaction-network module, in the spirit of BCoR and MPRS-NetWAS.

The allocation rule works in rounds of fixed batch size, matching how metabolomics is actually run. Each round has three steps:

  1. Form the current confidence set over threshold decisions.
  2. Find the pessimistic hypothesis: the unresolved assignment that would most change the final discovery set if it were true.
  3. Choose the batch of unprofiled people whose burden vectors maximize the expected KL information against that hypothesis, with a diminishing-returns penalty so the batch does not collapse onto one gene.

People carrying variants in several unresolved genes are naturally favored.

The theoretical work has two parts.

The first is a per-pair lower bound on the expected number of profiled carriers $n_{gm}$ that any rule needs to make the threshold decision with error at most $\delta$. It follows Garivier's inequality:

$$ n_{gm} \;\ge\; \frac{\ln(1/(2.4\,\delta))}{\mathrm{KL}(\gamma, 0)}, $$

where $\mathrm{KL}(\gamma,0)$ is the per-carrier divergence between an effect of size $\gamma$ and a null effect under the residual noise model. When $n_{gm}$ exceeds the number of carriers $c_g$ of gene $g$ in the cohort, the pair is flagged as information-limited.

The second part is an upper bound on missed discoveries after budget $B$ for the pessimistic rule. It should depend on an effective number of informative carriers rather than on the number of candidate pairs, analogous to CROP's effective arm count. The bound is proven for the thresholded, finite-class case and checked empirically beyond it.

The heritability readout is the variance in each metabolite, and in downstream traits, explained by the discovered burden effects. It is reported alongside the variance attributable to pairs flagged as information-limited.

Baselines

The naive reference is uniform random subsampling. At budget $B$, profile $B$ people drawn at random, then run the same burden association test and FDR control as Scherer et al. on that subset. Any adaptive method has to beat this to justify itself.

The strongest realistic competitor is static carrier-enriched sampling. It profiles carriers of rare damaging variants first, ordered by the number of candidate genes each person carries, and fills any remaining budget at random. Geneticists would reach for this design by default, and it already exploits the main structure our method uses. It receives the same budget, the same batch schedule, and the same downstream test.

A second strong competitor is a one-shot optimal design. It solves a single information-maximizing selection over burden vectors using the same hierarchical model, but makes no adaptive updates. This isolates whether sequential acquisition matters.

A third competitor is batched Thompson sampling under the same model, the BCoR-style choice. It isolates pessimism against optimism and randomized exploration.

Ablations remove the network and pathway prior, and replace the pessimistic target with an optimistic upper-confidence target.

All methods get identical data access, the same prior hyperparameters fit on a held-out gene set, and equal tuning effort.

We found no reported result on Lacuna for subsampled or adaptive recall in rare-variant metabolomics. The only reported anchor is the full-cohort ceiling of 235 associations from Scherer et al. We will count it a win if the pessimistic rule reaches at least 1.3 times the recall of carrier-enriched sampling at a 10–25% budget with matched empirical FDR, or reaches 50% recall with materially fewer profiles.

Why not just…?

Why not just profile all carriers of rare damaging variants? This is the real threat. When the carrier pool is smaller than the budget, carrier enrichment is close to optimal, and adaptivity adds little. The proposal earns its keep only when carriers outnumber the budget, or when most carriers turn out to be uninformative after a first batch. Our experiments vary the budget across that crossover point and report it honestly. If carrier enrichment matches us everywhere, the remaining contribution is the information-limited decomposition, which still stands on its own.

Why not just use extreme sampling on polygenic scores or phenotypes? Extreme sampling enriches for common-variant signal. The target here is rare burden effects, and selecting on phenotype can bias effect estimates for exactly those genes. We include it as an additional baseline. We expect it to help detect common-variant metabolite effects, but not the rare-variant associations that make up the 235.

Why not just use Thompson sampling or Bayesian optimization, which already work in health allocation? Takeno and Iwazaki prove that Thompson sampling's failure probability decays only polynomially. A single funded study has one draw, so tail failures matter. Jun and Zhang show optimistic rules over-explore in informative-arm structures. The Thompson-sampling baseline tests this directly; we will report the lower quantiles of recall, not just the mean.

Why not just impute metabolites from genotype with existing prediction models? Imputation models are trained on associations that are already known. They therefore cannot reveal new rare-variant effects, which are precisely the missing heritability in question.

Proposed Experiments

The primary experiment is a retrospective replay on a fully profiled exome-plus-metabolomics cohort of the kind used by Scherer et al. Access is the main practical risk.

The procedure hides all metabolite values and reveals them only for people a method selects, in batches. Methods are scored by recall of the full-cohort association set at budgets of 5%, 10%, 25%, and 50%, under matched empirical FDR. Repeated random seeds give recall distributions.

A second replay uses ADNI. It treats amygdala grey-matter density as the costly measurement and scores recovery of the four network modules found by Luo et al.. Semi-synthetic panels built from 1000 Genomes genotypes, with planted burden effects of controlled size and carrier frequency, test the theory under known ground truth.

The decisive check is whether the lower bound predicts failure. Pairs flagged as information-limited at a given budget should be missed by every method, including carrier enrichment. Pairs flagged as reachable should be recovered by the pessimistic rule well above chance.

If the flagged set does not separate empirical successes from failures, the decomposition of missing heritability into design-limited and information-limited parts is not credible, and we will say so.