Introduction
Deep learning models often exhibit overconfidence when encountering Out-of-Distribution (OOD) data, posing significant safety risks in real-world applications. While simple post-hoc methods like MaxLogit are computationally efficient, they typically lag behind complex training-based approaches. This paper identifies a fundamental limitation in MaxLogit: it "couples" feature direction and magnitude, which often carry conflicting signals for OOD detection.
The Decoupling Framework
The core insight of this work is the mathematical decomposition of a logit ($z$) into its constituent parts: the feature norm ($\|h\|$), the classifier weight norm ($\|w\|$), and the cosine similarity ($\cos(\theta)$): $$z = \|h\| \cdot \|w\| \cdot \cos(\theta)$$
Standard MaxLogit treats these components with equal weight. However, empirical analysis reveals that MaxCosine (direction) is generally a strong indicator of OOD samples, whereas MaxNorm (magnitude) can be an "encumbrance" due to high variance in standard training. The authors propose Decoupling MaxLogit (DML), a post-hoc score that balances these signals: $DML = \lambda \text{MaxCosine} + \text{MaxNorm}$.
DML+: Optimized Training for OOD Detection
To further enhance performance, the authors introduce DML+, which utilizes a dual-model training strategy to optimize the individual components:
- Model F (Focal): Trained with Focal Loss to refine angular boundaries by focusing on hard samples, thereby improving the reliability of MaxCosine.
- Model C (Center): Trained with Center Loss to create a compact feature space, making the feature norm (MaxNorm) a more discriminative signal for OOD detection.
Key Insights: Feature Convergence
The effectiveness of DML+ is rooted in how specific loss functions shape the feature space. The authors demonstrate a strong negative correlation between feature convergence metrics and OOD detection performance.
As shown in the figure above, reducing Within-class Feature Convergence (WFC) through Center Loss directly improves the MaxNorm AUROC. Similarly, optimizing the angular space improves the MaxCosine signal.
Experimental Results
DML+ achieves state-of-the-art performance across multiple benchmarks without requiring exposure to auxiliary OOD data during training.
On the large-scale ImageNet-1k benchmark, DML+ achieves an AUROC of 93.16%, significantly outperforming existing methods like ViM (82.51%) and MaxLogit (86.60%). On CIFAR-100, it reaches 91.57% AUROC, a substantial jump from the previous SOTA, LogitNorm (85.73%). Furthermore, DML+ drastically reduces the False Positive Rate (FPR95) on ImageNet to 29.79%, compared to MaxLogit's 61.16%.
Conclusion
By decoupling the logit into direction and magnitude and optimizing them via specialized loss functions, DML+ provides a powerful yet principled approach to OOD detection. While the dual-model requirement increases computational overhead, the significant gains in reliability and accuracy set a new standard for the field.
Summary updated