Introduction
Semi-supervised learning (SSL) has become a cornerstone of modern machine learning, allowing models to leverage vast amounts of unlabeled data. However, a significant challenge remains: real-world data often follows a "long-tailed" distribution, where a few "head" classes dominate while many "tail" classes are underrepresented. Standard SSL techniques like FixMatch often fail in these scenarios due to the "overconfidence trap"—where models generate high-confidence but incorrect pseudo-labels for tail classes—and the "threshold barrier," where fixed high-confidence thresholds exclude tail-class samples entirely.
To address these issues, this paper introduces Uncertainty-Aware Dynamic Threshold Selection (UDTS). UDTS moves beyond simple confidence-based filtering by incorporating model uncertainty and adaptive, class-specific thresholds.
The UDTS Framework
The core of UDTS is a dual-model architecture that integrates uncertainty estimation directly into the pseudo-labeling pipeline.
Uncertainty Estimation via Monte Carlo Dropout
Unlike traditional methods that rely on a single prediction, UDTS employs Monte Carlo (MC) Dropout. By performing multiple forward passes ($T \approx 10$) with dropout enabled, the model generates a distribution of predictions for each unlabeled sample. The variance across these passes serves as a robust measure of uncertainty. Low variance indicates a stable, reliable prediction, while high variance suggests the model is biased or guessing, even if the individual confidence score is high.
Dynamic Threshold Selection
A key innovation of UDTS is the move from a global fixed threshold to class-specific, time-evolving thresholds. Using an Exponential Moving Average (EMA), the framework tracks the learning state of each category.
In the early stages of training, thresholds are kept lower to encourage the model to explore tail-class samples. As the model matures, these thresholds tighten, ensuring that only the most reliable pseudo-labels are incorporated into the training set. This "curriculum" approach prevents the model from being overwhelmed by noise early on while maintaining high precision in later stages.
Experimental Evaluation
The researchers evaluated UDTS on several long-tailed benchmarks, including CIFAR10-LT, CIFAR100-LT, STL-10-LT, and the medical imaging dataset TissueMNIST.
Key Results:
- CIFAR10-LT: UDTS achieved a significant accuracy boost of over 5.26% compared to the FixMatch baseline.
- STL-10-LT: The method showed a remarkable 9.96% improvement in accuracy, demonstrating its strength in scenarios with large unlabeled pools.
- TissueMNIST: An improvement of 1.28% in medical imaging tasks highlights the generalizability of UDTS beyond natural images.
- Robustness: UDTS consistently outperformed other specialized imbalanced SSL methods like CReST, DARP, and DASO.
Conclusion and Future Work
UDTS effectively mitigates the biases inherent in long-tailed semi-supervised learning by decoupling confidence from reliability. By perceiving its own uncertainty and adapting its selection criteria dynamically, the model achieves superior performance on rare classes without degrading accuracy on common ones.
While the current implementation of MC Dropout introduces computational overhead, future research will focus on more efficient uncertainty estimation techniques, such as single-pass uncertainty heads or knowledge distillation, to scale UDTS to even larger datasets and more complex tasks like object detection.
Summary updated