Genotype as a Referee for Noisy Clinical Labels

Medical machine learning trains on diagnoses that clinicians assign, and clinicians disagree with one another. Radiologists, pathologists and coders also share blind spots. When every reader leans the same way, as with "safety-first" overcalling of lung nodules, adding readers only makes the error more confident. No method that looks only at the images can detect a bias that all the image readers share.

This note proposes a referee that cannot share that bias: the patient's germline genotype. It is fixed at conception, the readers never see it, and polygenic risk scores (weighted sums of many small-effect variants) are almost uncorrelated with clinical risk factors. If label errors do not depend on genotype, then any error weakens the statistical link between a polygenic score and the label by an exactly predictable factor. That makes genotype a measuring stick for how accurate a set of labels is, and it works without biopsies or redundant expert reads. The measuring stick is sharpest in early-onset patients, where inherited risk dominates disease, which brings the classical early-onset enrichment design from genetic epidemiology into label-noise research.

Core Claim

Let $D$ be true disease status and $Y$ the label a reader assigns. Let $\mathrm{Se}$ and $\mathrm{Sp}$ be the reader's sensitivity and specificity, and $G$ a polygenic score. If reader errors are independent of $G$ given $D$ (the "non-differential" assumption), then

$$\mathrm{Cov}(Y, G) = (\mathrm{Se} + \mathrm{Sp} - 1)\,\mathrm{Cov}(D, G).$$

This identity comes from writing $P(Y=1 \mid G)$ as a mixture over $D$. The bracketed term is the reader's Youden index $J$. Suppose $\mathrm{Cov}(D,G)$ is known from external genome-wide studies and true prevalence $\pi$ is known from registries for an age stratum. Then the label prevalence $P(Y=1) = \mathrm{Se}\,\pi + (1-\mathrm{Sp})(1-\pi)$ supplies a second equation. Together the two equations identify both $\mathrm{Se}$ and $\mathrm{Sp}$, the full 2×2 noise transition matrix, from one label per patient.

Doing this separately in each age band gives an age-dependent noise model. Existing estimators instead need several annotators or clean anchor samples. We expect the standard error of $J$ to scale roughly as $1/(\sqrt{n}\,|\mathrm{Cov}(D,G)|)$, because it is a ratio estimator with a fixed denominator. Strata where the genetic effect is larger, namely the early-onset strata, should therefore yield more precise noise estimates per case.

The claim is deliberately population-level. A polygenic score is weak for any one person: the integrated coronary score in Polygenic risk score improves the accuracy of a clinical risk score for coronary artery disease reaches a C-statistic of only 0.640 alone. We therefore never use genotype to relabel individual patients. We use it to estimate how noisy a labeling process is, and then correct training through that estimate.

Position Relative to Prior Work

Two strands motivate the gap. On the label side, Re-thinking and Re-labeling LIDC-IDRI for Robust Pulmonary Cancer Prediction showed that consensus radiologist malignancy scores carry a shared overcalling bias. Models trained on them reached 0.97 sensitivity and 0.23 specificity, and relabeling against 180 biopsy-proven nodules moved this to 0.67/0.66. That paper establishes that shared bias is real, but its fix depends on surgical pathology, which exists only for a small anchor set. Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment found that consensus beyond three pathologists adds only about +0.013 F1. This constrains the "just add readers" route.

On the genetics side, the early-onset studies motivate the age design. These include Identification of 13 novel susceptibility loci for early-onset myocardial infarction, hypertension, or chronic kidney disease, P708Identification of 26 novel loci that confer susceptibility to early-onset coronary artery disease in a Japanese population and P3718Identification of nine genes as novel susceptibility loci for early-onset ischemic stroke, intracerebral hemorrhage, or subarachnoid hemorrhage. Each restricts cases to premature onset to raise genetic signal relative to environmental noise. The coronary score paper supplies the near-zero correlation ($r = 0.01$) between polygenic and clinical risk, and reports stronger discrimination under age 55. Both facts are what make genotype a plausible independent referee.

Three methods supply machinery we reuse or compete with. Leveraging Inter-Rater Agreement for Classification in the Presence of Noisy Labels recovers the transition matrix from annotator agreement by eigendecomposition. It reaches 79.4% versus 74.0% for majority vote on CIFAR10-N, but it needs multiple labels per sample, and shared bias remains invisible to it because agreement among biased readers looks like accuracy. LiNC: Lightweight Noise Correction via Per-Sample Trust and Gaussian Mixture Modeling learns a per-sample trust weight from early training dynamics. It raises mean last-epoch accuracy on ten MedMNISTv2 datasets at 50% symmetric noise from 58.82% to 80.23%, but it assumes early training tracks the true signal, an assumption that systematic bias violates. Learning from Multiple Experts with Random Forests: Application to the Segmentation of the Midbrain in 3D Ultrasound weights experts by local image features, again entirely within image space.

The closest bridging work, iGWAS: Image-based genome-wide association of self-supervised deep phenotyping of retina fundus images, links images and genomes, but it removes human labels altogether rather than auditing them. The HUNT lung-SNP model: genetic variants plus clinical variables improve lung cancer risk assessment over clinical models uses a polygenic score as a predictor, raising AUC from 0.844 to 0.875, but it treats the registry diagnosis as clean.

Our contribution is different in kind. Genotype is not a model input here. It is an instrument that identifies label error rates, and it is the only such instrument in this literature that is independent of reader bias by construction.

Research Program

The first component is an estimator. In each age band $a$ we regress labels on the standardized polygenic score, adjusting for genetic ancestry principal components and genotyping batch. We divide by the externally estimated disease–score covariance to obtain $\hat J(a)$. We then solve with registry prevalence $\pi(a)$ for $\widehat{\mathrm{Se}}(a)$ and $\widehat{\mathrm{Sp}}(a)$, and obtain confidence intervals by bootstrap.

The external covariance needs care, because it must refer to the same disease definition. We take it from a validation cohort with the most stringent available endpoint. We then propagate its uncertainty and treat "external covariance mis-specified by ±20%" as a sensitivity analysis.

The second component turns the estimated, age-dependent transition matrices into training signal through forward loss correction. The classifier's predicted clean-label probabilities are multiplied by $\hat T(a)$ before the cross-entropy against observed labels is computed. Genotype is used only at this estimation step. The deployed model takes images or clinical features alone and needs no genotype at inference.

The third component is an audit tool. Given any candidate relabeling, whether from LiNC, agreement decomposition or biopsy propagation, we compute its implied $J$ and ask whether denoising moved the labels toward the disease. An increase in genotype association is evidence of real denoising. A decrease shows the method has fit something other than disease.

The non-differential assumption can fail in three ways, and the program tests each.

First, a denoiser could learn a heritable trait that is not the disease. We guard against this with a negative-control polygenic score for an unrelated heritable trait (height). If its association with the labels changes, the audit is compromised.

Second, reader bias could track ancestry, which also shifts score distributions. We guard against this with ancestry adjustment and by restricting to the White British subset where the scores were validated.

Third, readers could see family history, a proxy for genotype. We test this by comparing patients with and without recorded family history.

The ancestry restriction is also a limitation of the method. The coronary score gives negligible gains in African-ancestry individuals, so the referee only works where the polygenic score transfers.

Baselines

The naive reference point is training directly on the observed single-source labels with plain cross-entropy. Every method must beat it, and its genotype-implied $J$ anchors the audit.

The strongest existing single-label competitor is LiNC, run with its published three-cluster mixture and trust parameterization. We give it the same backbone, the same training data and the same hyperparameter search budget as our forward-corrected model. Its best reported result is 80.23% mean last-epoch accuracy at 50% symmetric noise on MedMNISTv2, with noise-detection AUC 0.9837 on OrganSMNIST at 20% noise. Those numbers come from symmetric synthetic noise, so they do not transfer to our setting. We will report LiNC's accuracy and noise-detection AUC on our age-dependent, asymmetric noise. Beating it means higher accuracy against held-out reference labels and lower transition-matrix error, especially in the older strata where we inject shared overcalling.

Where several label sources exist per patient, the strongest competitor is the agreement-matrix decomposition of Leveraging Inter-Rater Agreement for Classification in the Presence of Noisy Labels, which reports 79.4% against 74.0% for majority vote on CIFAR10-N. We run it on the same patients with the same number of sources, and we also report majority vote. Beating it means smaller transition-matrix error whenever sources share a bias. When sources err independently, we expect to tie, and a tie there would be informative rather than a failure.

The key ablation removes age stratification, estimating one transition matrix for all ages. A second ablation replaces the real polygenic score with a permuted score, which should make $\hat J$ uninformative and its confidence interval cover everything. A third replaces the disease score with the height score. If either of the last two ablations still "improves" labels, the gain is not coming from genetics.

Why not just…?

Why not just add the polygenic score as an input feature, as HUNT and the coronary score paper do? That improves risk prediction against whatever labels exist. It cannot tell you the labels are wrong, and it evaluates the model against the same biased endpoint. Using the score as a feature also makes the deployed model depend on genotyping. Our approach uses genotype once, offline, to estimate the noise and correct training. We will train a feature-augmented model as a side comparison and show that its evaluation against biased labels misranks models whose ranking against reference labels is known.

Why not just collect more readers and take a consensus? The LIDC-IDRI relabeling study shows that consensus among readers who share a bias produced 0.23 specificity. The mitotic-figure study shows gains flatten after three readers. Several reads per scan across a biobank are also unaffordable. Agreement-based estimators are the principled version of this idea, and experiment two pits them directly against the genotype estimator on real multi-source labels.

Why not just use biopsy anchors, as the LIDC-IDRI relabeling did? Where pathology exists, it beats any statistical referee for individual cases. But the anchor set was 180 nodules, and many conditions, coronary disease included, have no routine tissue gold standard. We see biopsy anchors as complementary: the genotype estimator works at population scale where anchors are absent.

Why not just run an image-only denoiser such as LiNC separately within each age band? This is the real threat. If reader noise is mostly idiosyncratic rather than shared, LiNC per stratum may match us without genotype. Experiment one settles this by varying the fraction of shared versus idiosyncratic noise. We predict LiNC matches us at low shared-bias fractions and falls behind as shared bias grows. If it matches us throughout, the training-time contribution collapses, and the proposal reduces to the audit tool. The audit tool alone is still worth having.

Proposed Experiments

Experiment one is semi-synthetic, with real genotypes, in UK Biobank coronary disease. It uses the coronary score paper's validation cohort of 272,307 participants with 7,036 incident cases, and treats the recorded incident diagnoses as reference truth. We inject asymmetric label flips that mimic age-dependent overcalling and undercalling, with false-positive rates rising with age. Some of the noise is shared, meaning it is driven by observed clinical features, and some is idiosyncratic. Models are trained on clinical features plus available imaging-derived measures.

The metrics are absolute error in recovered $\mathrm{Se}(a)$ and $\mathrm{Sp}(a)$, interval coverage, and C-statistic against the reference labels. For reference, the coronary score paper reports C-statistics of 0.718 for the clinical equations and 0.753 with the polygenic score added. The decisive check is whether estimation error per case is lower in the under-55 band than in older bands, as the early-onset argument predicts.

Experiment two uses real disagreeing observers without new data collection. In UK Biobank the same coronary event can be recorded by self-report, by primary care codes and by hospital inpatient codes. These are three observers that disagree and plausibly share biases, such as coding conventions and attention to symptomatic patients. We estimate each source's transition matrix two ways: by the genotype method on each source alone, and by agreement decomposition across all three. Where the two estimates diverge, we check which one better predicts the later hospital-confirmed endpoint during follow-up. That later endpoint serves as a delayed, partial gold standard.

Experiment three is the audit. We apply LiNC and agreement decomposition to produce relabeled training sets, compute each relabeling's genotype-implied $J$ with height-score and permuted-score controls, and test whether the audit ranks methods in the same order as accuracy against reference labels in experiment one. If the audit's ranking agrees with the reference ranking in simulation, and its controls stay flat, genotype has earned its place as a referee.

The imaging case that motivates this work, thoracic CT with shared radiologist bias, needs a cohort with both scans and genotypes. No reported result was found on Lacuna for any CT nodule benchmark paired with germline data. This remains the next step once the method is validated.