Towards Realistic Long-Tailed Semi-Supervised Learning: Consistency is All You Need

Tong Wei, Kai GanCVPR 2023 · OpenReview

Abstract

While long-tailed semi-supervised learning (LTSSL) has received tremendous attention, existing algorithms typically assume that the class distributions of labeled and unlabeled data are almost identical. These algorithms suffer when distributions are mismatched because they utilize biased pseudo-labels. We propose the Adaptive Consistency Regularizer (ACR) to effectively utilize unlabeled data of unknown class distributions. ACR dynamically refines pseudo-labels by estimating the true class distribution of unlabeled data. ACR achieves state-of-the-art performance, including a 10% absolute increase in test accuracy in mismatched scenarios, while consistently outperforming sophisticated algorithms even when distributions are identical.

Introduction: The Challenge of Mismatched Distributions

In real-world scenarios, datasets are often "long-tailed," where a few majority classes dominate while many minority classes have few samples. Semi-supervised learning (SSL) leverages unlabeled data to mitigate this, but standard Long-Tailed SSL (LTSSL) methods assume the unlabeled data follows the same distribution as the labeled set.

In practice, this assumption is frequently violated. For instance, labeled data might be carefully curated (long-tailed), while unlabeled data is collected broadly (uniform or reversed). When these distributions mismatch, models produce biased pseudo-labels that degrade performance on minority classes. ACR aims to provide a robust solution for these realistic, mismatched LTSSL settings.

Methods: Adaptive Consistency Regularizer (ACR)

ACR builds upon the FixMatch framework using a dual-branch architecture to resolve the tension between learning high-quality features and a balanced classifier.

  1. Standard Branch: Focuses on learning robust feature representations using standard cross-entropy.
  2. Balanced Branch: Employs "balanced softmax" to explicitly account for class priors, focusing on learning a class-balanced classifier.
The ACR architecture featuring dual branches (Standard and Balanced) and an Adaptive Adjustment module for pseudo-label refinement.

The core of the method is the Adaptive Adjustment module. It estimates the unlabeled distribution on-the-fly by comparing model predictions against three anchor distributions (Consistent, Uniform, and Reversed) using bidirectional KL divergence. Based on this estimate, the model dynamically adjusts a scaling parameter to refine pseudo-labels. This ensures that the balanced branch receives labels that compensate for the estimated distribution of the unlabeled data.

Results and Discussion

The researchers evaluated ACR on benchmarks including CIFAR10/100-LT, STL10-LT, and ImageNet-127. The experiments covered three unlabeled distribution scenarios: Consistent, Uniform, and Reversed.

Top-1 accuracy comparison between ACR and DASO across varying imbalance ratios for the unlabeled set, highlighting ACR's robustness to distribution mismatch.

As shown in the figure, ACR significantly outperforms the previous state-of-the-art method, DASO, particularly as the imbalance ratio of the unlabeled set deviates from the labeled set. In the most challenging "reversed" scenarios, ACR achieves an absolute accuracy gain of approximately 10%. Furthermore, ACR scales effectively to large-scale datasets like ImageNet-127, showing a 9.5% to 11% improvement over existing methods like TRAS.

Conclusion

ACR demonstrates that a simple dual-branch consistency approach can effectively handle the complexities of mismatched class distributions in LTSSL. By adaptively refining pseudo-labels based on on-the-fly distribution estimation, ACR provides a robust and scalable framework for realistic semi-supervised learning.

Summary updated

Abstract

While long-tailed semi-supervised learning (LTSSL) has received tremendous attention in many real-world classification problems, existing LTSSL algorithms typically assume that the class distributions of labeled and unlabeled data are almost identical. Those LTSSL algorithms built upon the assumption can severely suffer when the class distributions of labeled and unlabeled data are mismatched since they utilize biased pseudo-labels from the model. To alleviate this issue, we propose a new simple method that can effectively utilize unlabeled data of unknown class distributions by introducing the adaptive consistency regularizer (ACR). ACR realizes the dynamic refinery of pseudolabels for various distributions in a unified formula by estimating the true class distribution of unlabeled data. Despite its simplicity, we show that ACR achieves state-of-the-art performance on a variety of standard LTSSL benchmarks, e.g., an averaged 10% absolute increase of test accuracy against existing algorithms when the class distributions of labeled and unlabeled data are mismatched. Even when the class distributions are identical, ACR consistently outper-forms many sophisticated LTSSL algorithms. We carry out extensive ablation studies to tease apart the factors that are most important to ACR's success. Source code is available at https://github.com/Gank0078/ACR.
BibTeX
@inproceedings{wei2023towards,
  author = {Tong Wei and Kai Gan},
  title = {{Towards Realistic Long-Tailed Semi-Supervised Learning: Consistency is All You Need}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  url = {https://openreview.net/forum?id=HK2kHFXact},
}
Figure