Abstract
Semi-supervised learning algorithms that use pseudo-labeling have become increasingly popular for improving model performance by utilizing both labeled and unlabeled data. In this paper, we offer a fresh perspective on the selection of pseudo-labels, inspired by theoretical insights. We suggest that pseudo-labels with a high degree of local variance are more prone to inaccuracies. Based on this premise, we introduce the Local Variance Match (LVM) method, which aims to optimize the selection of pseudo-labels in semi-supervised learning (SSL) tasks. Our methodology is validated through a series of experiments on widely-used image classification datasets, such as CIFAR-10, CIFAR-100, and SVHN, spanning various labeled data quantity scenarios. The empirical findings show that the LVM method substantially outpaces current SSL techniques, achieving state-of-the-art results in many of these scenarios.
Introduction
In semi-supervised learning (SSL), models leverage a small set of labeled data alongside a large pool of unlabeled data. A dominant strategy is pseudo-labeling, where the model generates predictions for unlabeled samples and treats high-confidence predictions as ground truth for subsequent training iterations.
However, existing state-of-the-art methods like FixMatch and FreeMatch rely primarily on posterior probability (confidence) as the selection criterion. In "low-label" regimes, models frequently become "confidently wrong"—assigning high probability to incorrect classes because they haven't adequately learned the underlying decision boundaries. This paper introduces Local Variance Match (LVM) to bridge this quality gap by distinguishing between true certainty and mere overconfidence.
The Local Variance Match (LVM) Method
The core innovation of LVM is a two-dimensional filtering strategy that considers both confidence and stability. Instead of just looking at how confident a model is, LVM examines how stable that confidence remains under minor input perturbations.
Local Variance (LV)
To assess the stability of a prediction, the authors introduce Local Variance. For an unlabeled image, the model applies several minor "noise perturbations." LV measures the variance of the model's predictions across these slightly different versions of the same input. High local variance indicates that the model's decision is sensitive to small changes, suggesting the pseudo-label is likely unreliable.
Selection Criterion
LVM only accepts a pseudo-label if it satisfies two conditions:
- High Confidence: The predicted posterior probability must exceed a threshold.
- Low Local Variance: The prediction must remain stable across perturbations, staying below a variance threshold.
To maintain computational efficiency, the authors found that using just 2 or 3 perturbations is sufficient to approximate local variance without significantly increasing training time.
Key Insights: Stability as a Proxy for Accuracy
Theoretical analysis using Gaussian Mixture Models (GMM) reveals a "confidently wrong" paradox: near decision boundaries, a model might output high confidence while remaining highly sensitive to small input changes. Local variance serves as a superior indicator of whether a model has truly captured the data distribution.
Empirical evidence supports this: labels discarded by the LVM filter (those with high confidence but high variance) consistently show significantly lower accuracy than those retained. This confirms that local variance is an effective "noise detector" for unreliable pseudo-labels.
Experimental Results
LVM was evaluated across computer vision benchmarks (CIFAR-10/100, SVHN, ImageNet) and speech recognition tasks (AISHELL).
- CIFAR-10: With only 10 labels (1 per class), LVM achieved an error rate of 5.41%, coming within 1% of fully supervised performance (4.62%). This significantly outperformed the previous best of 8.07%.
- CIFAR-100: With 400 labels, LVM reached a 35.87% error rate, a nearly 2% improvement over previous state-of-the-art methods.
- ImageNet: Large-scale tests with 100k labels confirmed its superiority, with LVM achieving a 39.31% top-1 error rate, outperforming FreeMatch (40.57%).
- Speech Recognition: The utility of LVM extends beyond vision. In Automatic Speech Recognition (ASR) tasks, LVM reduced the Word Error Rate (WER) from 5.17% (supervised only) to 4.71%.
Conclusion
Local Variance Match (LVM) provides a robust framework for pseudo-label selection by incorporating prediction stability. By filtering out "confidently wrong" labels, LVM achieves state-of-the-art performance across multiple domains, nearly matching fully supervised results with a fraction of the data. Future work will focus on further characterizing the theoretical properties of the algorithm and refining hyperparameter tuning for the local variance threshold.
Summary updated