Separating Rare from Mislabeled: GAN Coverage Time as a Label-Free Rarity Signal for Medical Label Cleaning

Medical image labels are noisy because experts disagree, and medical datasets are imbalanced because disease is rare. The two problems interfere. Most label-cleaning methods flag a training example as suspicious when the classifier struggles to fit its label, measured by high loss, slow learning, or low "trust." A rare but correctly labeled case produces the same signal, and so does an atypical presentation of a common disease. So the cleaner discards or relabels exactly the minority cases that were already too scarce. We propose to get the missing ingredient, a measure of how rare an image is that never looks at the label, from the training dynamics of a generative adversarial network (GAN). A GAN is a pair of networks: a generator that learns to produce realistic images, and a discriminator that learns to tell real images from generated ones. While a GAN trains, the generator reproduces common image modes early and rare modes late or never. The discriminator can therefore tell, for each real training image, roughly when the generator first learned to imitate its region of image space.

The core claim is that label-based difficulty, minus the part explained by this label-free "coverage time," isolates annotator error from biological rarity. If the claim holds, a cleaner built on the residual should flag mislabeled images as well as current methods do while wrongly flagging far fewer clean minority-class and rare-subtype images. We test this on datasets with real multi-expert disagreement and controlled class-dependent noise.

Core Claim

For each training image $x_i$, let $d_i$ be a standard label-based difficulty score. We use one minus the per-sample trust parameter of LiNC, which weights how much the loss relies on the observed label versus the network's own prediction and drops when the two disagree. Let $\tau_i$ be the coverage time of $x_i$: the first generator checkpoint after which the discriminator's real-versus-fake score on $x_i$ stays within a fixed band of the score it gives generated samples, smoothed over nearby checkpoints. Images the generator never learns to imitate get $\tau_i = T$, the final checkpoint. The noise score is the difficulty left over after accounting for rarity:

$$ s_i = d_i - \hat{g}(\tau_i), $$

where $\hat{g}$ is a monotone (isotonic) regression of $d$ on $\tau$ fitted across the whole training set. In words, $\hat{g}(\tau)$ is the difficulty we would expect from an image that is exactly this rare. A mislabeled image of a typical-looking lesion is covered early but is hard to fit, so its residual is large. A correctly labeled rare subtype is covered late and is hard to fit, and its difficulty is explained by rarity. We add a second label-free check: disagreement between $x_i$'s label and the labels of its nearest neighbours in the frozen discriminator's intermediate features.

The coverage-time idea follows from the density-ratio reading of the discriminator. At a fixed generator, the optimal discriminator scores a point by comparing the true image distribution with the generator's distribution there. Real images in regions the generator has not yet reached are therefore easy to call real. The mode-coverage behaviour documented by Diverse Image Generation via Self-Conditioned GANs motivates reading the order in which modes are covered as a rarity ranking. That paper clusters discriminator features to force coverage of missing modes, raising recovered modes on a 25-mode grid from 17.3 to 25.0. We do the opposite: we leave the GAN's bias toward common modes in place and read it off as a measurement.

Position Relative to Prior Work

The motivating evidence comes from label-noise work on medical data. Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation shows that noise filters discard hard, correctly labeled minority cases. It counters this with a curriculum driven by two uncertainties: one learned from multi-rater disagreement, and one from Monte-Carlo dropout, which estimates uncertainty from the spread of many randomly perturbed forward passes. Both of its uncertainties are computed from label-supervised models, and the disagreement term needs several raters per image. Our rarity signal needs neither.

LiNC supplies our difficulty measure. The research file notes its central weakness: it relies on early training separating signal from noise, which fails under systematic bias. LiNC was also evaluated with symmetric noise, where every class is equally likely to be flipped to every other class. That is precisely the regime where rarity does not confound difficulty.

GRID is the closest methodological relative. It flags noise by comparing the losses of a generative head (a variational autoencoder modelling images and labels jointly) and a discriminative head on a shared backbone. Its generative model is conditioned on labels, so its signal is not independent of label frequency, and it was tested on satellite imagery without imbalance.

On the representation side, GAN-DL shows that a frozen StyleGAN2 discriminator is a strong feature extractor for microscopy, beating ImageNet features by up to 26%. It assumes clean labels downstream and uses only the final discriminator, not its trajectory.

Two papers constrain the design. Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment shows that consensus of three pathologists cuts model variance from an interquartile range of 0.038 to 0.016, but F1 caps near 0.74 to 0.76. Label cleaning should therefore be judged against a three-rater consensus model, not only against single-rater training. MedMNIST v2 provides fast controlled benchmarks, but at 28×28 pixels. Resolution-sensitive conclusions must be confirmed on native-resolution data.

Research Program

We build three components.

1. GAN training and coverage times. We train an unconditional StyleGAN2, following GAN-DL's choice, on the training images only, with labels never shown to it. We save discriminator checkpoints at a fixed schedule. Discriminators overfit small medical datasets, which would make every real image look "uncovered." We monitor this by also scoring a held-out set of real images and stopping when training and held-out real scores diverge. Coverage times are computed only from checkpoints before that point.

2. The cleaner. From the saved checkpoints we compute $\tau_i$, the isotonic curve $\hat{g}$, the residual $s_i$, and the neighbour-disagreement score from the final discriminator's features. We then plug the combined score into LiNC's three-cluster mixture in place of raw trust. Everything else in LiNC's correction schedule stays unchanged. This makes the only difference from LiNC the rarity adjustment.

3. Analysis, the main scientific output. We stratify noise-detection quality by class frequency and by within-class rarity. For within-class rarity we use expert-marked subtypes where they exist, such as Gleason pattern and ISIC diagnostic category. We then ask where coverage time and label difficulty diverge.

We also examine borderline images, where experts genuinely disagree. We do not expect to call these "noise." They should land at intermediate coverage times and intermediate residuals. The planned treatment for them is soft labels rather than relabelling, and we check that the method sends them there.

Proposed Experiments

Datasets. We use four settings.

Gleason 2019 has real disagreement among pathologists. We train on single-pathologist labels and hold out the multi-rater majority as reference.

The canine mast cell tumour mitotic-figure dataset has 12 pathologists and 32,917 annotations. We train on one pathologist's labels and measure agreement of the cleaned labels, and of the resulting detector, with the 12-expert consensus.

ISIC 2019 is used at its native imbalance, with injected noise at 40% to match Ju et al.

Five MedMNISTv2 2D subsets are used with two noise types. The first is symmetric noise, which reproduces LiNC's regime. The second is class-dependent asymmetric noise that flips minority labels into the majority class, mimicking clinicians under-calling rare disease.

Metrics. We report noise-detection AUC overall and within frequency bins. The key imbalance metric is the false-flag rate on clean minority and rare-subtype images. Downstream we report macro-F1, balanced accuracy, and minority-class recall.

On the mitotic data, the reference point is Wilm et al.'s three-consensus model. Success would mean a single-rater model trained on cleaned labels approaching that model's F1 and variance, which would amount to buying back two raters' worth of annotation.

Baselines

Naive training. Standard cross-entropy on the observed labels, with the same ResNet-18 classifier, augmentation, and epochs as every other method. We add class-balanced loss weighting so that "doing nothing" about noise is not also "doing nothing" about imbalance. Ju et al. report cross-entropy at 78.27% AUC on ISIC 2019 at 40% noise.

LiNC. The strongest single-annotator cleaner on MedMNISTv2. It reports 80.23% mean last-epoch accuracy across 10 MedMNISTv2 datasets at 50% symmetric noise, versus 58.82% for standard training, and a noise-detection AUC of 0.9837 on OrganSMNIST at 20% noise, versus 0.8652 for AUM and 0.9126 for VoG. We rerun it with the authors' schedule and the same tuning budget as our method, in two versions: its original global mixture, and a per-class mixture. We expect to match it under symmetric noise and must beat it on minority false-flag rate and macro-F1 under asymmetric noise.

Dual-uncertainty curriculum of Ju et al. The strongest method that explicitly targets the rarity-versus-noise confound. It reports 81.01% AUC on ISIC 2019 at 40% noise, versus 79.38% for CurriculumNet, and 83.22% F1 on Gleason 2019, versus 81.88% for majority vote. It gets multiple raters on Gleason, which our single-rater setting does not use. We therefore report both a head-to-head against its published numbers and a single-rater comparison where its disagreement term is unavailable. Beating it with single-rater labels on Gleason would be strong evidence. Matching it would already be useful, because our method does not need multiple raters.

GRID. The generative-versus-discriminative loss comparison, run on our medical data with the same backbone. The file reports only an 8% NDCG gain over ELR on satellite imagery, and there is no reported result found on Lacuna for GRID on medical data.

Majority vote. Consensus of three raters, where raters are available.

Ablations. Three ablations isolate the key ingredient. The first replaces coverage time with k-nearest-neighbour density computed in the same discriminator features. The second computes the same density-based rarity in ImageNet features. The third removes the rarity adjustment entirely, keeping only neighbour disagreement.

Fairness. All methods share data splits, classifier architecture, and a matched hyperparameter search. GAN training compute is reported separately and charged to our method.

Why not just…?

Why not just fit LiNC's mixture separately within each class? This is the most serious simple competitor. It removes the class-frequency confound at essentially no cost. It cannot handle rarity within a class, such as an atypical presentation of a common label, and the file notes that atypical presentations are a documented failure mode in imbalanced medical data. If per-class LiNC matches us on the within-class rare-subtype false-flag rate on Gleason and ISIC, the coverage-time idea adds nothing beyond class counts, and we will report that.

Why not estimate rarity with k-nearest-neighbour density in any good feature space, with no GAN trajectory? This is also a genuine threat, and the first ablation tests it directly. Our reason to expect a difference is that density estimated from nearest neighbours rates blurry, artifact-laden, and borderline images as rare. Those are often the images experts disagree on, so the adjustment would excuse real noise. A generator, by contrast, learns to reproduce common imaging artifacts early. If feature-space density performs as well, the finding narrows to "label-free rarity matters," and the GAN reduces to a feature extractor.

Why not just collect three annotations per image? Wilm et al. show that three-rater consensus removes most model variance, and the Gleason majority-vote baseline in Ju et al. is already strong. But extra raters triple annotation cost and still cap F1 near 0.75 on the mitotic task. The mitotic experiment measures directly how much of the three-rater benefit single-rater cleaning recovers.

Why not use Ju et al.'s Monte-Carlo dropout uncertainty? It needs repeated forward passes for every sample during training. It is also computed from a label-supervised model, so rare-but-correct cases still look uncertain. The single-rater Gleason comparison shows whether a label-free signal does better.

Decisive Checks

The idea succeeds only if all of the following hold.

First, under minority-to-majority asymmetric noise on MedMNISTv2 and on ISIC 2019, the rarity-adjusted cleaner has a clearly lower false-flag rate on clean minority images than both global and per-class LiNC, without losing overall detection AUC.

Second, on Gleason, the gain persists within classes for rare patterns, which per-class thresholds cannot provide.

Third, coverage time beats nearest-neighbour density in the first ablation. If this check fails, the proposal reduces to a cleaner built on rarity from any good feature space, and we will report that result.

Fourth, the time at which the generator starts imitating a group of images correlates with that group's frequency on subtypes marked by experts. This is checked before any cleaning results are examined.