Genetic risk models are starting to use rare mutations, meaning DNA changes carried by fewer than one in a hundred people. These models help most for the few people who carry a damaging mutation, and those are also the people for whom we have the least evidence. A standard uncertainty wrapper can report "90% of intervals contain the true value" and still be wrong for nearly every carrier, because carriers are too rare to move the average. We propose to calibrate uncertainty the same way geneticists already gain statistical power: by grouping rare variants into biologically defined units (a gene, a functional class of mutations within a gene, a pathway) and treating carriers within a unit as comparable. The model then gives a calibrated interval for a carrier when the unit holds enough comparable calibration carriers. When it does not, the model declines to predict, and that refusal comes from a precise count rather than a heuristic threshold.
The approach also addresses a second failure. When a model built on European-ancestry participants is applied to people of other ancestries, many of their rare variants never appear in the calibration data. For those variants, reweighting individual variants cannot correct for the shift, because there is no overlap to reweight. Once variants are aggregated into functional units, the units overlap across ancestries, so shift correction becomes possible again. Our claim is that aggregation, a tool built for association testing, is also the right place to define who counts as exchangeable when calibrating and abstaining.
Core Claim
Conformal prediction takes any model's predictions and returns intervals that contain the truth at a chosen rate $1-\alpha$. It does this by ranking held-out "calibration" errors. The guarantee requires exchangeability: the test person must be statistically interchangeable with the calibration people. Split conformal run within a subgroup (often called Mondrian conformal) gives the guarantee within that subgroup, but only if the subgroup has enough calibration members. With $n_s$ calibration people in stratum $s$, the conformal quantile is finite only when
$$ \lceil (1-\alpha)(n_s+1) \rceil \le n_s \quad\Longleftrightarrow\quad n_s \ge \frac{1-\alpha}{\alpha}, $$
so at $\alpha = 0.1$ a stratum needs at least nine calibration carriers before any finite interval is possible. This follows directly from the order-statistic definition of the split-conformal quantile.
For rare variants, strata defined per variant almost never reach that count. Strata defined per pathway reach it easily but mix variants with opposite effects. We propose a hierarchy of strata taken from rare-variant aggregation. The levels run from the variant, to gene × functional-annotation bin (for example AlphaMissense score tertile or protein-truncating status), to gene, to pathway, and finally to "any qualifying carrier." Each test person is assigned to the finest level whose stratum has at least $m$ calibration carriers. If even the coarsest admissible level is too sparse, or if pooling fails a heterogeneity check, the person is deferred.
The rule that assigns a stratum uses only genotypes and annotations, never phenotype labels. Choosing the level therefore does not break within-stratum exchangeability. It is a covariate-defined partition, unlike the label-dependent selection that Conformalized Multiple Testing after Data-dependent Selection shows inflates error. The one label-dependent choice is whether a unit may be pooled. That choice is made on a separate split with a dispersion-style test on residuals. The test is the variance-component statistic behind SKAT, applied to calibration residuals rather than raw phenotypes.
The result is an explicit trade-off. Coarser pooling gives more coverage of carriers, finer pooling gives more faithful exchangeability, and abstention absorbs whatever the data cannot support.
Where This Sits
Three prior results motivate the proposal. Rare variant effect estimation and polygenic risk prediction (RareEffect) estimates per-variant effects with shrinkage and shows that adding rare-variant scores to common-variant polygenic scores raises lipid-level $R^2$ from about 0.54–0.58 to about 0.62–0.66 in the top and bottom 0.5% of the population. It outputs only point estimates, with no intervals and no abstention. That gain is concentrated in exactly the carriers we care about, and we supply the uncertainty layer it lacks.
Act or Defer: Error-Controlled Decision Policies for EHR Foundation Models (STRATCP) combines class-stratified conformal p-values with deferral on health-record data. It reports that standard conformal prediction had roughly 10% selected coverage on positive pancreatic-cancer calls while meeting its marginal target. The authors name calibration scarcity in rare strata as the open bottleneck. Their strata are outcome classes. Ours are genotype-defined units whose granularity adapts to sample size. Genotype strata are the actual source of scarcity in genetic risk, and STRATCP has no mechanism for borrowing across related strata.
How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? shows that conformal and other intervals fall to 55–70% coverage under real shift. This constrains our design: we cannot assume European-calibrated intervals transfer across ancestries.
Two lines of work supply methods. Weighted conformal prediction with learned density ratios, as in Robust Conformal Volume Estimation in 3D Medical Images (a logistic classifier on latent features), and clipped ratio fitting with coverage inflation, as in Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts, both assume the test distribution overlaps the calibration distribution. We apply them within aggregation strata, where overlap across ancestries is plausible, instead of on raw genotypes, where it is not.
The burden-versus-dispersion literature also constrains which strata may be pooled. A gene-based burden analysis found nothing for entorhinal thickness, whereas SKAT-based binning found FANCC (Knowledge-driven binning approach for rare variant association analysis: application to neuroimaging biomarkers in Alzheimer's disease), and burden testing failed on SLC12A3 because effects cancel (Rare variant effect estimation and polygenic risk prediction). Together these show that carriers of different variants in one gene are often not exchangeable. We treat this as the central empirical risk. It is the reason the pooling test exists, rather than something we assume away.
Research Program
The base predictor is fixed. It is RareEffect-style rare-variant scores plus a common-variant polygenic score, fit on a training split. The nonconformity score is either the absolute residual or the conformalized-quantile score from a quantile head. The work has four parts.
The first part builds the stratum hierarchy. Units come from gene boundaries, AlphaMissense and protein-truncating annotations as used in Mapping rare protein-coding variants on multi-organ imaging traits, and KEGG and Reactome pathways following Pathway-based joint effects analysis of rare genetic variants using Genetic Analysis Workshop 17 exon sequence data. A person carrying qualifying variants in several genes is assigned by the most extreme contributing gene score. Ties go to the coarser common stratum, and all assignments are fixed before labels are seen.
The second part is the pooling test. On a dedicated split, each candidate merge of child strata into a parent is tested for residual heterogeneity. We use a burden-type statistic for shifts in mean residual and a SKAT-type variance-component statistic for bidirectional shifts. Merges that fail are forbidden at deployment.
The third part is ancestry transfer. Calibration is on European-ancestry participants and testing is on held-out participants of other ancestries. Within each admissible stratum, we reweight calibration scores by clipped density ratios on ancestry principal components and stratum-level features, following Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts. A test person whose variant is novel to the calibration set is therefore still covered through their functional bin.
The fourth part is theory. We will state the within-stratum guarantee under the assumption of exchangeability within the chosen stratum. We will then bound how coverage degrades when that assumption holds only approximately, using the between-child residual distribution distance that the pooling test estimates. This bound makes precise what the test protects against.
Baselines
The naive reference is standard split conformal on the same fixed predictor, calibrated on the whole calibration pool. It will meet marginal coverage almost by construction. We expect it to fail on carriers, as STRATCP's pancreatic-cancer result on health records suggests, and we report its carrier-conditional coverage to show whether this happens here. No reported result found on Lacuna for conformal coverage of rare-variant genetic risk models, so this baseline's carrier numbers will be new measurements, not reproductions.
The strongest point-prediction reference is RareEffect plus polygenic score, whose reported gain is lipid $R^2 \approx 0.62$–$0.66$ versus $0.54$–$0.58$ in the top and bottom 0.5% (Rare variant effect estimation and polygenic risk prediction). We do not try to beat this $R^2$, because our method uses the same predictor. We verify that we reproduce it and then show what fraction of that extreme-tail population keeps valid intervals under our method.
The strongest uncertainty competitor is Mondrian conformal stratified at a single fixed level, gene-level carrier status, which is the direct analogue of STRATCP's class stratification applied to genotypes. It uses the same predictor, calibration pool, and split sizes, with the gene level chosen to match our average abstention rate. Beating it means higher coverage in the worst decile of strata at equal or lower abstention, or lower abstention at equal worst-stratum coverage.
For ancestry shift, the competitor is clipped weighted conformal prediction on ancestry principal components over the whole pool, without strata (Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts). Its density-ratio model gets the same features and the same tuning budget as ours.
Two ablations isolate the key ingredients. The first removes the pooling test and always pools to the coarsest admissible level, which isolates what the heterogeneity check buys. The second uses the hierarchy without clipped reweighting, which isolates the aggregation effect in ancestry transfer.
Why not just…?
Why not just abstain on everyone who carries a qualifying rare variant? This controls error trivially and is a real reference point, so we include it. It discards exactly the people whose predictions RareEffect improves most, since the reported $R^2$ gain lives in the extreme tails. Our primary metric is the fraction of carriers who receive a valid, finite interval. Blanket abstention scores zero on that metric by definition. If our method retains only a negligible fraction of carriers, the honest conclusion is that current cohorts cannot support carrier-level intervals, and we would report that.
Why not just use gene-level Mondrian conformal prediction? This is the most serious threat, and it is the main baseline. Our method reduces to it whenever the hierarchy always selects the gene level. We win only if two things hold: finer gene × annotation bins are frequently populated and more homogeneous in large cohorts, and coarser pathway pooling rescues genes with too few carriers without failing the heterogeneity test. If worst-stratum coverage and abstention match gene-level Mondrian on UK Biobank lipids, the added hierarchy is not worth it, and the experiments will show that directly.
Why not just reweight by ancestry with weighted conformal prediction? Reweighting needs overlap. For variants private to a non-European test person, the individual-level density ratio is effectively unbounded. That is the regime where Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts reports instability for unclipped weighting and needs clipping plus coverage inflation. Even clipped, global reweighting ignores the fact that the carrier's residual distribution is governed by their gene. The ancestry experiment compares coverage for carriers of novel variants under global clipped weighting and under within-stratum weighting.
Why not just threshold the model's own predicted variance? Shrinkage predictors are overconfident exactly where data are sparse, because shrinkage pulls carrier estimates toward zero. Separately, How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? shows that heuristic uncertainty fails to flag shifted inputs. We include variance-thresholded abstention, matched to our abstention rate, and compare carrier-conditional coverage.
Proposed Experiments
The primary testbed is UK Biobank exome data with the RareEffect trait set, about 393,000 participants, using LDL, HDL, triglycerides, height, and BMI. Models are trained on one split, pooling tests run on a second, calibration on a third, and evaluation on a fourth. Metrics are:
- marginal coverage;
- coverage conditional on carrier status;
- worst-decile stratum coverage;
- the fraction of carriers retained (not abstained);
- mean interval width among retained carriers.
The ancestry experiment calibrates on European-ancestry participants and evaluates on held-out non-European participants. Because the file gives no subgroup sizes, we will report their counts and confidence intervals on coverage explicitly.
Two controlled experiments test the pooling logic where the truth is known. The first uses GAW17 real exome genotypes with simulated phenotypes, 697 individuals with a known causal architecture (Pathway-based joint effects analysis of rare genetic variants using Genetic Analysis Workshop 17 exon sequence data). The second plants effects into UK Biobank genotypes, some within-gene effects same-direction and some opposite-direction. In the same-direction case, pooling should be accepted and should raise retention without losing coverage. In the opposite-direction case, which mimics SLC12A3, naive pooling should break coverage for one variant subgroup and the test should block the merge. A secondary check on ADNI (750 participants, entorhinal thickness) asks whether the method abstains almost entirely in a small cohort, which is the expected and correct behaviour.
The decisive result is this. On UK Biobank lipids, at $\alpha=0.1$, the hierarchy must keep worst-decile stratum coverage at or above 0.85 while retaining more carriers than gene-level Mondrian at matched worst-stratum coverage. For non-European carriers of calibration-novel variants, within-stratum weighting must reach coverage closer to nominal than global clipped weighting.
Either result would be informative. If both hold, aggregation becomes the working unit of calibration for genetic risk. If the pooling test rejects most merges, that is direct evidence that functional annotations do not define exchangeable carrier groups, which matters in its own right for anyone building rare-variant risk scores.
References
- Conformalized Multiple Testing after Data-dependent Selection
- Rare variant effect estimation and polygenic risk prediction
- Act or Defer: Error-Controlled Decision Policies for EHR Foundation Models
- How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?
- Robust Conformal Volume Estimation in 3D Medical Images
- Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts
- Knowledge-driven binning approach for rare variant association analysis: application to neuroimaging biomarkers in Alzheimer's disease
- Mapping rare protein-coding variants on multi-organ imaging traits
- Pathway-based joint effects analysis of rare genetic variants using Genetic Analysis Workshop 17 exon sequence data