Long-Tailed Out-of-Distribution Detection: Prioritizing Attention to Tail

Yina He, Lei Peng, Yongcun Zhang, Juanjuan Weng, Zhiming Luo, Shaozi Li

Abstract

Real-world data distributions are often long-tailed, posing a significant challenge for Out-of-Distribution (OOD) detection. Standard models frequently confuse rare "tail" classes with OOD samples, and existing remedies often sacrifice "head" class accuracy. This paper introduces PATT (Prioritizing Attention to Tail), a framework that utilizes implicit semantic augmentation and post-hoc feature calibration to enhance OOD detection while simultaneously boosting classification performance across all classes.

Addressing the Long-Tail Challenge in OOD Detection

Most research in Out-of-Distribution (OOD) detection assumes a balanced "In-Distribution" (ID) dataset. However, real-world scenarios typically involve long-tailed distributions where a few "head" classes dominate and many "tail" classes have sparse representation.

Standard models struggle in this setting because tail classes lack sufficient feature representation, leading the model to misidentify them as OOD. Previous attempts to mitigate this often involved re-balancing techniques that reduced the influence of head classes. While this can improve OOD detection, it typically degrades the model's accuracy on head classes—a classic "robbing Peter to pay Paul" scenario. The challenge is to improve OOD detection without sacrificing the accuracy of the classes the model is intended to recognize.

The PATT Framework

The PATT framework addresses these issues through two main components designed for training and inference:

1. TISAC: Implicit Semantic Augmentation

Temperature Scaling-Based Implicit Semantic Augmentation Contrastive Learning (TISAC) operates during the training phase. Instead of reducing head-class data, PATT augments the tail. It models ID data using a mixture of von Mises-Fisher (vMF) distributions. By mathematically deriving a closed-form loss for an infinite set of contrastive pairs, the model effectively "sees" an infinite variety of tail-class features. Additionally, a temperature-scaled logit adjustment (TLA) pushes ID and OOD samples further apart in confidence space.

The TISAC training framework uses implicit semantic augmentation on a hypersphere to better represent tail classes.

2. Post-Hoc Feature Calibration (FC)

During inference, PATT applies an attention-based calibration to the penultimate feature layer. By identifying feature channels most critical for tail classes and those prone to OOD confusion, the model prioritizes tail-class semantics. This calibration reduces the confidence scores of OOD data while maintaining high performance for ID classes.

Experimental Results

PATT demonstrates superior performance across standard long-tailed benchmarks like CIFAR-LT and ImageNet-LT.

PATT significantly improves accuracy for both Head and Tail classes compared to previous methods like EAT and PASCL.

Key findings include:

  • State-of-the-Art Performance: On ImageNet-LT, PATT achieved a 4.29% increase in AUROC for OOD detection and an 8.35% increase in ID classification accuracy over the previous state-of-the-art (EAT).
  • Balanced Improvements: Unlike methods that sacrifice head-class performance, PATT improved head-class accuracy by 15.43% and tail-class accuracy by 13.29% compared to a standard Outlier Exposure (OE) baseline.
  • Computational Efficiency: PATT is an end-to-end approach. On CIFAR10-LT, it trained in roughly half the time required by EAT (2078s vs. 4248s) while delivering better results.

Conclusion

PATT successfully resolves the performance trade-off in long-tailed OOD detection. By focusing on "augmentation over reduction" and utilizing directional feature modeling, it provides a robust and efficient solution for real-world deployment. Future work may involve scaling these techniques to large-scale foundation models and multi-modal tasks where long-tailed distributions remain a significant hurdle.

Summary updated

BibTeX
@inproceedings{he2025long,
  author = {Yina He and Lei Peng and Yongcun Zhang and Juanjuan Weng and Zhiming Luo and Shaozi Li},
  title = {{Long-Tailed Out-of-Distribution Detection: Prioritizing Attention to Tail}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i3.32357},
}
Figure