Use Unlabeled Spectra to Estimate Class Priors, Not to Move Boundaries

When two tissue classes look almost identical to a hyperspectral camera, as stomach and jejunum do, unlabeled pixels cannot tell a semi-supervised learner where the boundary between them belongs. A hyperspectral camera records reflectance at roughly a hundred wavelengths per pixel. Semi-supervised learning trains on a few labeled examples plus many unlabeled ones, and it usually assumes the true boundary runs through a sparse "valley" in the data. Two classes with overlapping spectra have no such valley. When one class is also rare, the standard procedure fails in a specific way. The learner assigns its own guesses ("pseudo-labels") to unlabeled pixels and trains on them, and in the overlap region those guesses drift toward the common class. The final model can end up worse than one trained on the labeled data alone.

This note argues that, inside overlapping regions, the unlabeled data are still useful, but for a different purpose. There, the optimal decision depends mostly on how common each class is, and a tiny, skewed labeled set estimates that badly. Unlabeled pixels can estimate class frequencies without anyone having to guess their labels. We propose a method that uses unlabeled pixels in two separate ways. Pixels outside the overlap are pseudo-labeled as usual. Pixels inside it are used only to re-estimate class frequencies. A final step combines this model with the supervised one so that the result is protected, under stated assumptions, against falling below supervised performance on per-class recall.

Core Claim

Take a pixel $x$ in a region where classes $a$ and $b$ overlap. The Bayes posterior is $p(y \mid x) \propto \pi_y\, p(x \mid y)$, where $\pi_y$ is the class frequency and $p(x \mid y)$ is the class-conditional spectrum density. When $p(x \mid a) \approx p(x \mid b)$, the decision between $a$ and $b$ is set almost entirely by the ratio $\pi_a/\pi_b$.

We make two claims about class-imbalanced semi-supervised learning with overlapping spectra. First, most of the harm comes from pseudo-labeling pixels in the overlap region. Second, most of the available benefit there comes from using unlabeled data to correct the class-frequency estimate.

The method, which we call overlap-gated safe semi-supervised learning, has three parts:

  1. Find the overlap. Estimate which class pairs overlap from cross-validated confusion of a supervised model on the labeled spectra. Then score each unlabeled pixel by how much of its predicted probability falls on overlapping pairs.
  2. Use the two kinds of pixel differently. Pseudo-label only the low-overlap pixels. Use the high-overlap pixels only in an EM re-estimation of class frequencies, which then adjusts the classifier's outputs. EM here means expectation–maximization: alternately computing soft class memberships and updating the frequencies.
  3. Add a safety step. Combine the supervised model, a supervised model with corrected frequencies, and the gated semi-supervised model using a worst-case weighting.

The safety step follows the maximin combination of A General Formulation for Safely Exploiting Weakly Supervised Data. That method maximizes worst-case gain over a baseline across unknown mixing weights. We apply it with a quadratic (Brier) loss on class-balanced targets. The choice of loss is deliberate. The Pessimistic Limits and Possibilities of Margin-based Losses in Semi-supervised Learning proves that, for linear classifiers, no semi-supervised procedure can guarantee non-degradation under strictly decreasing losses such as cross-entropy, while quadratic loss allows safe improvement. Our design keeps cross-entropy for training the deep base learners, where it works well. It uses quadratic loss only in the low-dimensional combination step, where the safety argument needs it. This is our partial answer to the tension between safe-learning theory and practical medical semi-supervised learning.

Relation to Prior Work

Three kinds of prior work motivate the proposal.

  • Overlap and imbalance interact. Overlap versus Imbalance showed, for supervised SVMs on synthetic 2D data, that overlap and imbalance interact non-additively. Overlap was the dominant factor, dragging F1 from near-perfect to about 0.5, with a breaking point near overlap 0.6 and imbalance 0.78. It never studied unlabeled data. We extend its controlled design to the semi-supervised setting.
  • Consistency regularization erases rare classes. Class-Imbalanced Semi-Supervised Learning identified the "cluster assumption trap": consistency regularization treats compact minority clusters as empty space. It responds by suppressing consistency loss for minority predictions. That paper treats rarity as the problem. We treat overlap plus rarity as the problem, and we change what unlabeled data are used for rather than how strongly they are used.
  • The overlap is real in surgical data. Spectral organ fingerprints and HeiPorSPECTRAL document the stomach–jejunum overlap in fully supervised work. Their finding that organ type explains 83.4% of spectral variance while inter-pig variation explains only 2.3% matters for our design. It means a labeled/unlabeled split by animal introduces little distribution shift, so overlap can be studied nearly in isolation.

Two lines of work constrain the design.

  • Deep safe semi-supervised learning targets the wrong failure. Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data (DS³L) reweights unlabeled instances with a bi-level optimization. Not All Parameters Should Be Treated Equally masks parameters instead. Both target unseen distractor classes, optimize accuracy, and assume known classes separate cleanly. Mutual ambiguity between two known classes is outside their scope. They also put no explicit weight on rare classes, so the overall validation loss is dominated by common classes, a weakness this note conjectures rather than demonstrates.
  • Safe methods for imbalance metrics are shallow and binary. Towards Safe Semi-Supervised Learning for Multivariate Performance Measures showed that accuracy-oriented safe methods such as S4VM can degrade on F1 and AUC. Its fix works only for shallow binary classifiers. S4VM itself searches for low-density separators, which by construction do not exist between overlapping classes.

Two papers supply methods we reuse or compare against.

  • Combination step. SAFEW supplies the maximin combination.
  • Closest empirical medical work. Towards Realistic Semi-supervised Medical Image Classification handles open-set samples and long tails in RGB medical images with prototype-based pseudo-labels and class-adaptive thresholds. It offers no floor relative to supervised training and does not address per-pixel spectral overlap.

Research Program

The work has three stages.

Controlled synthetic study. We extend the synthetic design of Overlap versus Imbalance to high dimension and to a labeled/unlabeled split. Overlap $\mu$ and imbalance ratio $\gamma$ are varied on a grid. At each point we measure the gain of semi-supervised over supervised learning, split into pixels inside and outside the overlap region. The hypothesis is that the loss from naive pseudo-labeling is concentrated in the overlap region and grows with both $\mu$ and $\gamma$. If degradation instead spreads uniformly, the premise of gating is wrong and we stop there.

Surgical hyperspectral study. We move to HeiPorSPECTRAL, using spectra of 100 bands from 500–1000 nm and 20 organs.

  • Splits. Data are split by pig into labeled, unlabeled, validation and test animals.
  • Imbalance. Labeled organs are subsampled to imbalance ratios $\gamma \in \{10, 50, 100\}$, following the long-tail protocol of the CIFAR-10 imbalance benchmarks.
  • Frequency mismatch. The unlabeled pool is kept either at matched or at shifted class frequencies. The shifted condition is the one in which frequency re-estimation should matter most.
  • Base model. The base learner is the 1D convolutional network from Spectral organ fingerprints.

Safety analysis. We state the conditions under which the combined model's balanced Brier risk does not exceed that of the supervised baseline. These are SAFEW's conditions: the truth must be covered by the convex hull of the ensemble's predictions. We then test empirically how often per-class recall falls below the supervised model across repeated pig splits.

Baselines

Supervised reference (the "do nothing" point). This is the 1D convolutional network trained only on the imbalanced labeled pigs, with class-balanced loss. Every safety claim is measured against it. For context only, Spectral organ fingerprints reports 95.4% ± 3.6% average accuracy and 93.0% average sensitivity with full labels. That is an upper reference, not the low-label baseline. We found no reported result on Lacuna for any semi-supervised method on HeiPorSPECTRAL, so every semi-supervised number will be produced here.

Standard semi-supervised methods. We run FixMatch-style pseudo-labeling and Mean Teacher, adapted to spectra using noise and band-dropout augmentation. These show how much naive use of unlabeled data hurts.

Strongest imbalance-aware competitors. These are Adsh, which uses class-adaptive confidence thresholds, and SCL. On CIFAR-10, Adsh reports 71.49% accuracy at $\gamma=150$ against 68.47% for FixMatch. SCL with Mean Teacher reports 22.62% error at $\gamma=100$ against 36.71% for supervised training. These are the best evidence in the file that imbalance-aware semi-supervised learning helps. They are, however, measured on natural images without spectral overlap.

Strongest safe competitor. This is DS³L with its outer-loop validation loss changed to class-balanced loss. We consider it the most serious threat, because it may learn to discard overlap pixels on its own. The file quotes no exact DS³L scalar, so there is no reported number to beat.

Fairness rules. All methods share the same backbone, the same augmentations, and the same labeled, unlabeled and validation pigs. All receive equal tuning trials chosen on the validation pigs. Training is matched in gradient steps, and the bi-level methods are additionally reported at matched wall-clock time.

Ablations that isolate the key ingredient. The first removes frequency re-estimation and simply discards overlap pixels. The second pseudo-labels overlap pixels as well, removing the gate. The third excludes a random subset of pixels equal in size to the overlap subset, testing whether it matters which pixels are excluded. The fourth removes the maximin combination.

What counts as winning. The proposal beats a baseline if it improves worst-class recall and balanced accuracy averaged over pig splits, with paired significance. It must also never fall below the supervised model's per-class recall on more classes than the baseline does.

Why not just…?

Why not just reweight unlabeled instances with DS³L using a class-balanced validation loss? This is a real threat. Reweighting could learn to zero out overlap pixels without any explicit overlap estimate. Two things count against it.

  • Its outer loop estimates harm from a validation set containing only a handful of rare-organ pixels. Those weights should be noisy, and the noise should be worst exactly where it matters.
  • Even when it works, reweighting can only discard overlap pixels. It cannot use them to correct class frequencies.

The ablation that discards overlap pixels without frequency correction is the direct comparison. If balanced DS³L matches the full method on the stomach–jejunum pair, the gating contribution is not new, and we will report it that way.

Why not just use class-adaptive thresholds, given that overlap pixels are low-confidence anyway? Under imbalance, the common class is confidently wrong in overlap regions, so confidence does not flag them. Adsh's mechanism also lowers thresholds for rare classes, which pulls ambiguous pixels in rather than keeping them out. The synthetic grid tests this directly. We will measure what fraction of Adsh's accepted pseudo-labels fall in the overlap region, and how accurate they are, as $\gamma$ grows.

Why not just train supervised and correct class frequencies post hoc with EM on the unlabeled pool, with no pseudo-labeling at all? This alternative may capture most of the gain, and our own core claim predicts that it should help. The proposal therefore includes it as a baseline and as an ensemble member, not as a straw man. The full method adds value only if pseudo-labeling outside the overlap region contributes further gains on well-separated organs. The results table is split by overlap region to settle this. If the simple pipeline matches the full method everywhere, the conclusion is that unlabeled spectra should feed frequency estimation and nothing more. That would still be a useful negative result for surgical hyperspectral imaging.

Why not just oversample rare organs before running semi-supervised learning? Overlap versus Imbalance found that rebalancing does not fix breakdown caused by overlap. Oversampling also leaves the class frequencies used for pseudo-labeling wrong in the unlabeled pool. We include oversampling plus FixMatch as an extra baseline.

Decisive Checks

The method's value rests on four checks:

  1. Where the loss is. On the synthetic grid, the semi-supervised loss must concentrate in the overlap region, rising sharply near the supervised breaking point.
  2. The hardest pair. On HeiPorSPECTRAL with shifted unlabeled frequencies, the full method must raise rare-organ recall on stomach versus jejunum above both the supervised model and supervised training with EM frequency correction.
  3. Harm rate. Across repeated pig splits, the full method must reduce, relative to every baseline, the fraction of (split, class) cases in which recall falls below supervised.
  4. The safety step earns its place. Removing the maximin combination must measurably increase that harm rate.

If the second check fails while the others pass, the honest conclusion is the simpler one: in overlapping spectra, unlabeled data are worth exactly a frequency estimate.