Abstract
Deep learning models often struggle with long-tailed data distributions where a few "head" classes dominate and many "tail" classes have sparse representation. We present the Rare-class Sample Generator (RSG), a plug-and-play module that addresses this imbalance by transferring intra-class variations—such as lighting, pose, and background—from frequent classes to rare ones. By "hallucinating" realistic new samples in the feature space during training, RSG enables models to learn more robust decision boundaries for infrequent categories. RSG is computationally efficient, discarded after training, and achieves state-of-the-art results on major imbalanced benchmarks including iNaturalist 2018 and ImageNet-LT.
Introduction
In real-world applications, data distributions are rarely uniform. While curated datasets like ImageNet provide balanced class counts, natural data typically follows a "long-tail" distribution. Standard deep learning models trained on such data become biased toward frequent classes, leading to poor generalization on rare categories.
Common strategies to mitigate this include oversampling rare classes (which risks overfitting), reweighting loss functions (which can destabilize training), or using generative models like GANs (which are complex and often fail to capture realistic intra-class variations). RSG offers a simpler, more effective alternative by leveraging the rich variation information already present in the frequent classes of the dataset.
The RSG Module
The RSG module is designed to be integrated into any standard Convolutional Neural Network (CNN) architecture, such as ResNet or DenseNet. Its primary function is to extract "feature displacements"—vectors representing class-irrelevant variations—from frequent classes and apply them to rare classes to generate synthetic training samples.
Core Components
The module consists of three primary sub-modules:
- Center Estimation Module: This component estimates the centroid of each class within the feature space, providing a reference point for calculating variations.
- Vector Transformation Module: It computes the displacement between a specific sample's features and its class center. This displacement captures variations like camera angle or illumination that are independent of the class identity.
- Contrastive Module: To ensure that the transferred variations do not include class-defining features (e.g., transferring a bird's wing to a car), the Contrastive Module acts as a filter, ensuring the extracted displacements remain class-agnostic.
Training Workflow
During the training phase, RSG identifies a "displacement" from a frequent-class sample and adds it to a real rare-class sample. This creates a "hallucinated" feature map that represents the rare class under a new variation. To further improve the classifier, the authors employ a Maximized Vector (MV) loss. This loss function encourages generated samples to lie near decision boundaries, forcing the network to learn more precise separators between classes. Notably, RSG is only active during training; it is removed for inference, ensuring no added latency during deployment.
Experimental Results
The effectiveness of RSG was validated across several large-scale imbalanced datasets:
- ImageNet-LT & Places-LT: On ImageNet-LT, RSG reached 51.8% accuracy, with significant improvements in "few-shot" classes. On Places-LT, it achieved 39.3% top-1 accuracy, surpassing previous benchmarks like OLTR.
- iNaturalist 2018: In this naturally imbalanced dataset of over 8,000 species, RSG reduced the error rate to 29.74%, outperforming the BBN baseline.
- CIFAR-10/100: On synthetic long-tailed versions of CIFAR, RSG consistently improved performance over strong baselines like LDAM-DRW.
Conclusion
RSG provides a robust and flexible solution for the long-tail problem in machine learning. By intelligently transferring variations across classes, it enriches the feature space of rare categories without the need for complex generative architectures or additional data. Its "plug-and-play" nature and zero-cost inference make it a highly practical tool for real-world imbalanced learning tasks.
Summary updated