TL;DR
This paper introduces SimiS, a remarkably simple yet highly effective baseline for imbalanced semi-supervised learning (SSL). By strategically augmenting labeled data with pseudo-labels derived from unlabeled data, SimiS significantly reduces class imbalance, leading to state-of-the-art performance across various benchmarks and demonstrating impressive robustness to diverse data distributions.
What Problem Does This Paper Address?
Semi-supervised learning (SSL) aims to leverage abundant unlabeled data to improve model performance, typically assuming a balanced distribution of classes. However, real-world datasets often exhibit a long-tailed or imbalanced distribution, where a few classes have many samples (head classes) and many classes have very few (tail classes). This imbalance poses a significant challenge for standard SSL methods, leading to confirmation bias where models over-rely on majority classes, producing biased pseudo-labels and ultimately hurting generalization performance. Existing methods for imbalanced SSL struggle to effectively mitigate this imbalance, especially under severe imbalance ratios.
Methods
The core of SimiS lies in its simplicity: it augments the labeled dataset with pseudo-labels generated from unlabeled data. Specifically, after each training epoch, SimiS generates pseudo-labels for the unlabeled data and then supplements the infrequent (tail) classes in the labeled set with the most confident pseudo-labels. The number of pseudo-labels added for each tail class is determined by the difference in class distribution between the most frequent class and that specific tail class. This approach focuses on boosting the representation of tail classes without excessively adding to the head classes, thereby effectively reducing the class imbalance in the labeled set. The method is designed to be easily integrated with existing SSL algorithms.
Datasets
The authors evaluated SimiS on several imbalanced datasets:
- CIFAR10-LT, CIFAR100-LT, STL10-LT: These are standard datasets with artificially created long-tailed distributions.
- FOOD101-LT: Another dataset with an artificially created long-tailed distribution.
- ImageNet127: A naturally long-tailed dataset derived from ImageNet.
Key Insights and Ideas
- Simplicity is Key: The paper strongly advocates for the effectiveness of a simple approach. By focusing on reducing the imbalance in the labeled data, SimiS achieves remarkable results without complex mechanisms.
- Reducing Labeled Data Imbalance is Sufficient: The experiments suggest that reducing the imbalance ratio of the labeled set is a crucial and often sufficient step for achieving strong performance in imbalanced SSL, even if the unlabeled data remains imbalanced.
- Robustness to Distribution Shifts: SimiS demonstrates impressive robustness to various combinations of training and testing data distributions, a significant advantage over many existing methods that degrade under distribution shifts.
- Flexibility and Compatibility: The straightforward nature of SimiS allows it to be easily combined with other SSL algorithms and re-balancing techniques, often leading to further performance improvements.
Key Results
- State-of-the-Art Performance: SimiS consistently outperforms existing imbalanced SSL methods by a significant margin across various datasets and imbalance ratios. For instance, it achieves improvements of 12.8%, 13.6%, and 14.6% over previous SOTA on CIFAR100-LT, FOOD101-LT, and ImageNet127, respectively.
- Faster Convergence and Better Pseudo-Label Accuracy: SimiS leads to faster convergence and improved accuracy of pseudo-labels, particularly for minority classes, which helps alleviate confirmation bias.
- Robustness Demonstrated: Figures and tables show SimiS maintaining high accuracy across a wide range of imbalance ratios (γ) for both labeled (γl) and unlabeled (γu) data, unlike other methods that degrade significantly.
- Synergy with Other Methods: When combined with techniques like Logits Adjustment (LA), SimiS further pushes the performance boundaries.
Limitations and Future Work
- Lack of Formal Theoretical Analysis: The paper acknowledges that while empirical results are strong, a formal theoretical guarantee for SimiS's effectiveness is still needed.
- Future Work: The authors hope their work will inspire further research, particularly on the theoretical underpinnings of SimiS, and encourage pushing the boundaries of imbalanced SSL for real-world deployment.
Summary updated