Introduction
Deep learning models are notoriously overconfident. When presented with Out-of-Distribution (OOD) data—samples from classes they have never seen—they often assign high-probability predictions to known classes rather than signaling uncertainty. This overconfidence poses significant safety risks in critical applications like autonomous driving and medical diagnostics.
While many high-performing OOD detection methods require access to internal model features or training data statistics ("white-box" methods), real-world scenarios often limit access to a model's final softmax outputs ("black-box" or post-hoc methods). Historically, post-hoc methods have lagged in accuracy. Researchers from Chalmers University of Technology have introduced Generalized ENtropy (GEN), a simple yet powerful post-hoc score that closes this performance gap without requiring model retraining or access to training data.
Methodology
The core of the GEN approach lies in magnifying the subtle deviations in predictive distributions when a model encounters unfamiliar data. It consists of two primary components:
1. Generalized Entropy Function
Standard Shannon entropy treats all probabilities equally and can be relatively "flat" near the extremes. GEN utilizes a specific family of concave functions:
$$G_\gamma(\mathbf{p}) = -\sum_{j=1}^{M} p_j^\gamma (1-p_j)^\gamma$$
By setting the parameter $\gamma$ to a low value (typically 0.1), the score becomes hyper-sensitive to small changes in the predictive distribution. This sensitivity allows the system to flag OOD samples that might otherwise appear as high-confidence ID samples under standard metrics.
2. Truncation (Trimming)
In large-scale datasets like ImageNet, the "tail" of the probability distribution (the hundreds of low-probability classes) often contains random noise that can degrade the reliability of an OOD score. GEN addresses this by only considering the top $M$ classes (e.g., the top 100). This truncation focuses the signal on the most relevant predictions, providing a cleaner indicator of uncertainty.
Experimental Setup
The authors evaluated GEN using ImageNet-1k as the In-Distribution (ID) dataset. Performance was tested against four diverse OOD benchmarks: OpenImage-O, Textures, iNaturalist, and the challenging ImageNet-O (natural adversarial examples). Testing spanned six architectures, including ResNet, RepVGG, and modern Vision Transformers (ViT, Swin, DeiT).
Key Results
GEN consistently outperformed existing post-hoc methods, improving the average AUROC (Area Under the Receiver Operating Characteristic curve) by at least 3.5% across all tested architectures.
- State-of-the-Art Performance: On the difficult ImageNet-O dataset, GEN achieved significantly higher AUROC than previous standards like MaxLogit or Energy scores.
- Synergy with Feature Clipping: When combined with methods like ReAct (which clips internal features), GEN's performance improved further, rivaling complex white-box methods despite its simpler post-hoc nature.
- Architectural Robustness: The method proved effective for both traditional CNNs and newer Vision Transformers, suggesting it captures a fundamental property of how neural networks represent uncertainty in their final layers.
Conclusion
GEN demonstrates that the performance of softmax-based OOD detection has been significantly underestimated. By applying a more sensitive entropy function and trimming distributional noise, GEN provides a robust, easy-to-implement solution for safer AI deployment. Future work may explore the theoretical underpinnings of why the top 10% of logits consistently provide the cleanest OOD signal across such a wide variety of models.
Summary updated