Abstract
The Rare-class Sample Generator (RSG) is a plug-and-play module designed to address the "long-tail" problem in machine learning. By synthesizing new feature-space samples for rare classes through the transfer of class-irrelevant variations (such as pose or lighting) from frequent classes, RSG enhances model generalization. This module is used only during training, ensuring no additional computational overhead during inference. RSG achieves state-of-the-art results on major imbalanced benchmarks.
Introduction: The Long-Tail Challenge
In real-world scenarios, data distributions are often highly imbalanced. While standard datasets like ImageNet are curated for balance, natural data typically follows a long-tailed distribution: a few "head" classes possess abundant examples, while many "tail" classes have very few. Standard deep learning models trained on such data tend to become biased toward frequent classes, leading to poor performance on rare ones.
Previous attempts to solve this, such as oversampling or noise-based hallucination, often lead to overfitting or unrealistic samples. RSG addresses these issues by operating in the feature space and leveraging real variations from frequent classes to augment rare ones.
The RSG Architecture
RSG is a fully parameterized module integrated into a CNN backbone. It consists of three primary components that work together to generate realistic synthetic features.
- Center Estimation Module: This module learns the "centers" of each class in the feature space. It supports multi-modal distributions to account for the diverse appearances within a single class.
- Contrastive Module: To ensure that the transferred information is truly class-irrelevant, this module prevents the "leakage" of identity-specific features from frequent classes into rare ones.
- Vector Transformation Module: This component extracts the "displacement" (the difference between a sample and its class center) from a frequent class and applies it to a rare class sample.
To maximize the utility of these generated samples, the authors employ a Maximized Vector (MV) Loss. This loss pushes the decision boundary by forcing generated features to have a larger magnitude while remaining co-linear with the original rare-class displacement.
Experimental Evaluation
The effectiveness of RSG was evaluated across several standard imbalanced benchmarks:
- Imbalanced CIFAR-10 & CIFAR-100: Testing on artificial long-tailed and step imbalances.
- iNaturalist 2018: A large-scale, naturally imbalanced dataset comprising over 8,000 species.
- Places-LT & ImageNet-LT: Long-tailed versions of major scene and object recognition benchmarks.
Key Insights
- Realistic Variation: By using actual variations (pose, lighting, background) from abundant classes instead of random noise, RSG produces more stable and realistic synthetic features.
- Zero Inference Overhead: RSG is a training-time-only module. Once training is complete, the module is discarded, leaving the final model as efficient as a standard CNN.
- End-to-End Integration: As a differentiable module, RSG can be trained alongside the backbone and acts as a "booster" for other imbalanced learning techniques like Focal Loss or LDAM.
- Mitigating Aliasing: The contrastive loss ensures that transferred variations do not carry over class-specific traits, a common failure point in earlier feature-transfer methods.
Results and Conclusion
RSG consistently outperformed existing methods across all datasets. Notably, on iNaturalist 2018, it achieved a state-of-the-art Top-1 error rate of 29.74%. On ImageNet-LT, it reached 51.8% accuracy, with significant gains in "medium-shot" and "few-shot" categories.
While RSG's performance depends on the diversity of the "head" classes, its simplicity and effectiveness make it a powerful tool for learning from imbalanced data. Future work may explore more complex transformation functions to further enhance the diversity of generated samples.
Summary updated