Abstract
Long-tailed visual recognition remains a significant challenge in computer vision due to the extreme imbalance between common "head" classes and rare "tail" classes. Standard deep learning models often exhibit high learning uncertainty and bias toward majority categories. This paper presents NCL++, a Nested Collaborative Learning framework designed to stabilize the learning process through inter-expert and intra-expert consistency. By combining balanced online distillation with nested feature learning, NCL++ significantly improves accuracy for rare categories while maintaining computational efficiency during inference. Experimental results across five major benchmarks, including iNaturalist 2018 and ImageNet-LT, demonstrate that NCL++ achieves state-of-the-art performance.
Introduction
In real-world scenarios, data distribution is rarely uniform. While curated datasets like ImageNet provide balanced class distributions, natural data typically follows a "long-tail" distribution. In these settings, a few head classes dominate the dataset, while hundreds of tail classes contain very few samples. Standard models trained on such data become biased, frequently misclassifying rare samples as common ones.
The authors of NCL++ identify learning uncertainty as a primary driver of this failure. They observed that identical models trained on imbalanced data often produce inconsistent predictions for the same image, a problem exacerbated by different data augmentations. NCL++ aims to mitigate this instability by forcing multiple "experts" and various views of the same image to reach a consensus through a structured collaborative framework.
Methodology
NCL++ is built upon three primary pillars that address uncertainty and bias through nested learning objectives.
1. Collaborative Learning (Inter & Intra)
To reduce prediction variance, NCL++ employs two levels of collaborative learning:
- Inter-expert (InterCL): Multiple models, or "experts," are trained concurrently. They function as both teachers and students, utilizing Online Distillation to encourage mutual consistency in their predictions.
- Intra-expert (IntraCL): A single model evaluates multiple augmented versions (e.g., different crops or flips) of the same image. The framework enforces consistency across these views, ensuring the model learns robust, transformation-invariant features.
2. Balanced Online Distillation
Standard knowledge distillation can inadvertently reinforce head-class bias if the teacher's predictions are skewed. NCL++ introduces a balanced probability calculation that adjusts for class frequency. This ensures that the knowledge transferred between experts does not ignore rare tail classes, facilitating a more equitable learning process across the entire distribution.
3. Hard Category Mining (HCM) and Nested Feature Learning (NFL)
The "Nested" component of the framework refers to a hierarchical approach to feature learning. The model first evaluates a Global View of all classes. It then identifies "Hard Categories"—negative classes that are most likely to be confused with the target sample. Through Nested Feature Learning (NFL), the model performs a secondary "Partial View" learning step focused specifically on distinguishing the correct class from these specific rivals. This granular focus helps the model develop a more meticulous ability to differentiate between visually similar categories.
Experimental Results
The authors evaluated NCL++ on five standard benchmarks: CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, Places-LT, and iNaturalist 2018. The latter is a naturally imbalanced dataset containing over 8,000 species and is considered the gold standard for long-tail recognition.
Performance Highlights
- iNaturalist 2018: NCL++ achieved 74.0% accuracy with a single expert and 75.2% with a two-expert ensemble, representing a significant improvement over previous state-of-the-art (SOTA) methods.
- ImageNet-LT: Using a standard ResNet-50 backbone, the framework reached 58.0% accuracy.
- Efficiency: NCL++ is more efficient than its predecessor; it achieves superior performance with only two experts compared to the three experts required by the original NCL.
- Tail Accuracy: The most notable gains were observed in the "Few" (tail) splits, where NCL++ outperformed baseline methods like Balanced Softmax Cross-Entropy (BSCE) by several percentage points.
Discussion and Conclusion
NCL++ demonstrates that treating model disagreement as a signal rather than noise can lead to more robust representations. By minimizing KL divergence between experts (InterCL) and between views (IntraCL), the framework smooths the optimization landscape for long-tailed data.
While NCL++ involves a higher training cost due to the requirement of multiple experts and augmentations, this overhead is restricted to the training phase. At inference time, a single expert can be deployed, allowing users to benefit from the collaborative training process without additional computational latency. Future research may focus on reducing the training-time overhead while maintaining the stability and accuracy gains provided by the collaborative approach.
Summary updated