Choosing Quantile Polygenic Scores Without Target Phenotypes

A polygenic risk score adds up thousands of small genetic effects into one number per person. A newer variant, the quantile polygenic score, builds a separate score for each slice of the trait distribution: one tuned to people at the low end of, say, triglyceride levels, one for the median, one for the high end. The motivation is that some variants matter only in the tails. These scores are reported to predict skewed metabolic traits better than ordinary scores, but they come with many more settings to choose: which quantiles to keep, which significance threshold to use at each, and how to combine them. Today every one of those choices is tuned against measured phenotypes in a validation set. The people we most want to score, such as a new biobank or an under-studied ancestry group, often have genotypes but few or no measured outcomes.

This note proposes a label-free way to make those choices. The rule rests on a simple decomposition. How well a score predicts a trait in a target population factors into two parts. The first is reliability: how much of the score's spread in that population is reproducible signal rather than estimation noise. The second is validity: how strongly that reproducible signal tracks the trait. Reliability can be measured on target genotypes alone, by building the score twice from independent halves of the discovery data and checking how well the two versions agree on target individuals. Validity cannot be measured in the target. It can be estimated on held-out discovery samples, where phenotypes exist, and then carried over. Tail quantiles are exactly where reliability varies most from one setting to the next, because statistical power collapses at the extremes. So quantile scores are the setting where measuring reliability in the target, rather than assuming it, should pay off most.

Core Claim

For a candidate configuration $c$ (a choice of quantiles, thresholds and combination weights), let $s^A_c$ and $s^B_c$ be the scores built from two disjoint halves of the discovery cohort. Suppose each half-score equals a shared reproducible component $g_c$ plus independent noise, and the noise is unrelated to the phenotype $Y$. Then the squared correlation between a score and the trait in the target population is

$$R^2_c = r_c \cdot v_c,$$

where $r_c = \mathrm{corr}(s^A_c, s^B_c)$ is the split-half agreement computed on target genotypes, and $v_c = \mathrm{corr}(g_c, Y)^2$ is the validity of the reproducible part. Because the full-discovery score has half the noise variance of either half-score, its reliability is the Spearman–Brown correction of $r_c$:

$$\rho_c = \frac{2 r_c}{1 + r_c}.$$

We estimate $v_c$ from a held-out slice of the discovery cohort and $\rho_c$ from target genotypes. We then rank configurations by $\hat\rho_c \hat v_c$.

The claim is narrow and testable. This product should rank quantile-score configurations in an unlabeled target nearly as well as target labels would. It should beat selecting on discovery-holdout validity alone whenever the target's allele frequencies change how noisy different configurations are. It should fail, in a predictable way, when the main error is that discovery variants tag causal variants differently in the target, a problem that split-half agreement cannot see because both halves share the discovery population's correlation structure between nearby variants (linkage disequilibrium).

How This Relates to Prior Work

The quantile machinery comes from existing work. Enhancing polygenic risk prediction by modeling quantile-specific genetic effects runs genome-wide quantile regression at $\tau = 0.1,\dots,0.9$ and builds a clumping-and-thresholding score per quantile. It then fits a supervised model that combines these with an ordinary linear score, and reports higher $R^2$ than the linear score alone for triglycerides and two-hour glucose in the Korean KARE cohort. We reuse its pipeline unchanged. What we replace is the step that needs target labels. A quantile integral linear model to quantify genetic effects on phenotypic variability collapses the whole quantile process into a single variance score and validates it against measured BMI variability in longitudinal cohorts. That work motivates the proposal: it shows quantile-type genetic signal is widespread, with an estimated 85% of common SNPs affecting BMI variability. It also shows the cost of supervised validation, since that study needed repeated phenotype measurements to check its score. Regularized quantile regression under heterogeneous sparsity and Genetic associations vary across the spectrum of fasting serum insulin document variants active only at specific quantiles, the latter only at the 85th or 15th percentile of insulin. They also illustrate the constraint that drives this proposal: tail estimates rest on few effective observations and are noisy.

The validation idea comes from Context Sensitive Information: Model Validation by Information Theory. That paper scores an unsupervised model by how much structure survives between two independent draws of the data. It also warns that stability alone is degenerate, because a model that says nothing is perfectly stable. Our decomposition is a cheap, linear special case of that two-sample idea. Validity $v_c$ plays the role of informativeness, so a trivially stable empty score is not rewarded, and no partition functions or sampling are needed. Maximin Separation Probability Clustering and Mixed Samples as Probes for Unsupervised Model Selection in Domain Adaptation supply the evaluation template. Each validates a label-free criterion by its correlation with true accuracy and by its gap to an oracle selector, and we adopt that template. As argued below, their specific criteria do not carry over to polygenic scores.

Research Program

The method has three parts. First, we split the discovery cohort into halves A and B plus a validity holdout H. On each half we run quantile-regression GWAS at the nine quantiles, and build every candidate configuration's per-quantile scores and combined score. Combination weights are fit on H, never on target data. Second, on target genotypes we compute split-half agreement for each per-quantile score and for each combined score. For combined scores this means applying the H-fitted weights to the A-built and B-built component scores separately. Third, on H we estimate validity as $\widehat{R^2}_H / \hat r_{c,H}$, which is the observed holdout accuracy divided by the holdout's own split-half agreement. We then multiply by target reliability.

A by-product of this work is a directional tail score. The difference between the 0.9-quantile and 0.1-quantile scores keeps the asymmetric tail information that the integral variance score discards. Its target reliability tells us whether it can be reported in a given population at all.

A secondary label-free diagnostic is the quantile-crossing rate in the target: the fraction of individuals whose predicted low quantile exceeds their predicted high quantile. Computing it requires intercepts, which we take from the discovery marginal quantiles, so it is a rough diagnostic. We use it to flag configurations as unstable. We do not use it to rank them.

The theory work is modest. It derives when the decomposition holds: independent half-noise, and noise uncorrelated with $Y$. It also gives the bias of $\hat\rho \hat v$ when validity changes between populations, which happens when the genetic correlation is below one or tagging differs. That bias analysis predicts exactly where the method should break.

Baselines

The naive reference point is a fixed default configuration chosen without any selection. One version is the ordinary linear score at a conventional threshold. The other is the full nine-quantile score with the published combination recipe from Enhancing polygenic risk prediction by modeling quantile-specific genetic effects, with weights fit on discovery holdout H. Any selection rule that cannot beat a sensible default is not worth using.

The strongest label-free competitor is selection on discovery-holdout accuracy alone: pick the configuration with the best $R^2$ on H and ignore the target. This baseline is also the key ablation, because it is our method with target reliability removed. The comparison isolates the single new ingredient. A second ablation ranks by target reliability alone, which tests whether validity is needed to avoid the degenerate "stable but empty" choice that the two-sample paper warns about.

The strongest supervised competitor is the published joint model with weights and configuration tuned on $n$ labeled target individuals, sweeping $n$ from 100 to several thousand. All methods share identical GWAS halves, candidate grids and compute. The labeled baseline gets the same configuration grid, and its $n$ labels are drawn from the same target pool we later evaluate on, kept disjoint from the evaluation set.

The upper reference is an oracle that selects by true target accuracy, analogous to the "Best" bound in Mixed Samples as Probes for Unsupervised Model Selection in Domain Adaptation. There, the label-free selector reached 69.68% against an oracle of 70.34% on DomainNet. That gives a rough sense of what small regret looks like in a different field.

For the quantile score itself, we found no reported $R^2$ value on Lacuna for KARE triglycerides or glucose. The file states only that the quantile score "achieved substantially higher predictive accuracy." We will therefore reproduce that baseline rather than quote a number.

Beating the competitors means three things. Our method must have lower selection regret than discovery-holdout selection in shifted targets, with a paired bootstrap interval excluding zero. It must stay within the oracle gap in in-distribution targets. And it must match the labeled baseline up to a stated break-even $n$.

Why not just…?

Why not just tune on a small labeled target sample? This is the most serious threat. If a few hundred measured individuals give regret as low as ours, the label-free method only matters for cohorts with literally no phenotypes. We do not argue around this. The sweep over $n$ reports the break-even sample size directly, separately for the median and tail configurations. We expect labeled tuning to need many more samples for tail quantiles, because few target individuals fall in the tails. If it does not, we will say the method is useful only below that $n$.

Why not just select on discovery-holdout accuracy? In the same population this may be nearly as good as our method, since reliability then matches between H and the target. The case for target reliability is that tail-quantile and lenient-threshold configurations draw on variants whose allele frequencies differ across ancestries, so their noise share shifts unevenly. The ablation settles this. If the product does not beat validity alone in the non-European targets, the central ingredient adds nothing and the claim fails.

Why not just port MixVal or separation-probability scores? Both have concrete obstacles here. A polygenic score is linear in genotype, so the score of a mixed genotype is exactly the mixture of the two scores. MixVal's interpolation probes are therefore automatically satisfied and cannot distinguish configurations. Separation probability scores binary partitions. Tail strata defined by the score are a linear function of the genotypes, so they are perfectly learnable from genotypes regardless of whether they mean anything. Both criteria are uninformative by construction, which is why we test the two-sample stability idea instead.

Why not just fix quantile crossing by sorting the predicted quantiles? Sorting repairs the output, but it does not tell us which configuration to choose. We include sorted variants of every configuration in the candidate grid, so this alternative is covered rather than competing.

Proposed Experiments

The main experiment uses UK Biobank, with European-ancestry participants as discovery data. Targets are a held-out European subset (in-distribution) and the non-European ancestry subsets (shift). Traits are BMI, where the integral variance model found 49 variance-associated loci; triglycerides, a right-skewed trait on which the quantile score gained; and LDL cholesterol as a control trait with expected little quantile heterogeneity. If access allows, KARE is a second target, since it is the cohort where quantile scores were originally shown to help.

Target phenotypes are hidden during selection and revealed only for scoring. The primary metric is the Spearman correlation, across roughly a hundred configurations, between $\hat\rho_c \hat v_c$ and true target $R^2$. Secondary metrics are selection regret relative to the oracle and per-quantile check-loss reduction for the tail scores.

The decisive mechanism check is semi-synthetic. We use real genotypes with simulated phenotypes containing quantile-specific and variance effects. We vary two things independently: target allele-frequency divergence, which the method should handle, and causal-variant tagging mismatch, which it should not. The method is supported if regret relative to validity-only selection falls as frequency divergence grows. It must also degrade gracefully as tagging mismatch grows, and its estimated $R^2$ should overshoot true $R^2$ in exactly that regime, which documents the boundary.

The method is refuted under any of three outcomes. The correlation with true target accuracy is no better than validity alone in real shifted targets. The break-even labeled sample size is under about a hundred. Or the reliability of tail scores is high everywhere, which would mean noise was never the bottleneck.