Abstract
Traditional semi-supervised learning (SSL) frameworks typically employ a "keep-or-discard" logic based on fixed confidence thresholds for pseudo-labeling. This approach often suffers from overconfidence, data starvation, and class imbalance. We introduce the Uncertainty-aware Ensemble Structure (UES), a lightweight, architecture-agnostic framework that replaces binary thresholds with continuous "long-tailed weights." By aggregating uncertainty from multiple prediction heads, UES effectively utilizes the entire unlabeled dataset, achieving state-of-the-art performance in both image classification and 2D pose estimation.
Introduction
Semi-supervised learning leverages small amounts of labeled data alongside large unlabeled datasets. The standard Self-Training Strategy (STS) generates pseudo-labels for unlabeled samples, incorporating those with high confidence into the training set. However, this method faces three critical challenges: the difficulty of selecting an optimal threshold, the inherent overconfidence of deep neural networks, and the "Matthew Effect," where easy-to-learn classes dominate the training process while difficult classes are neglected.
The authors argue that low-confidence samples, often discarded by traditional methods, contain valuable supervisory signals. To capture this utility, they propose moving from hard selection to a soft, uncertainty-aware weighting mechanism.
Methodology: Uncertainty-aware Ensemble Structure (UES)
The UES framework is designed to be integrated into existing SSL pipelines like FixMatch or DualPose. It employs a multi-head architecture to estimate prediction consistency.
Mean-based Uncertainty Estimator (MUE)
Instead of a single prediction, UES utilizes multiple "heads" via a Channel-based Ensemble (CBE) module. The MUE calculates sample uncertainty by measuring the deviation of individual head predictions from the ensemble mean. This provides a more robust measure of reliability than raw softmax confidence.
Two-Tiered Weighting
UES implements a dual weighting strategy:
- Sample Uncertainty ($w^S$): Quantifies the disagreement between heads for a specific input.
- Prediction Head Uncertainty ($w^H$): Evaluates the historical reliability of each specific head across the training batch.
The Long-tailed Weighting Mechanism
The core innovation of UES is the mapping of uncertainty to a continuous weight distribution. Unlike binary thresholding, which assigns a weight of 0 or 1, UES uses a mathematical function to produce a long-tailed distribution.
This ensures that every unlabeled sample contributes to the model's learning, particularly during early training phases where capturing broad data features is essential. As uncertainty increases, the weight decreases but remains non-zero, acting as a safety net against noisy pseudo-labels.
Experimental Results
The UES method was evaluated on 2D pose estimation (Sniffing, FLIC, LSP) and image classification (CIFAR-10/100) benchmarks.
- Pose Estimation: UES demonstrated significant gains, improving the Percentage of Correct Keypoints (PCK) by 3.47% on the Sniffing dataset and 7.29% on the FLIC dataset compared to previous state-of-the-art methods.
- Image Classification: Integrated with FixMatch, UES achieved consistent accuracy improvements on CIFAR-10 and CIFAR-100, even with extremely limited labeled data (e.g., 40 labels).
Ablation studies confirmed that the combination of sample-level and head-level uncertainty is vital for optimal performance, providing a more refined weighting than either component alone.
Conclusion
UES represents a shift in SSL from rigid data selection to a more nuanced, uncertainty-aware inclusion strategy. By utilizing long-tailed weights, the model avoids the pitfalls of hard thresholds and leverages the full spectrum of unlabeled data. Future work will explore dynamic weighting mechanisms that adapt the distribution tail as the model's expertise grows during the training lifecycle.
Summary updated