Decoupling MaxLogit for Out-of-Distribution Detection

Zihan Zhang, Xiang XiangCVPR 2023 · DOI

Introduction

Deep learning models often exhibit overconfidence when encountering Out-of-Distribution (OOD) data, posing significant safety risks in real-world applications. While simple post-hoc methods like MaxLogit are computationally efficient, they typically lag behind complex training-based approaches. This paper identifies a fundamental limitation in MaxLogit: it "couples" feature direction and magnitude, which often carry conflicting signals for OOD detection.

The Decoupling Framework

The core insight of this work is the mathematical decomposition of a logit ($z$) into its constituent parts: the feature norm ($\|h\|$), the classifier weight norm ($\|w\|$), and the cosine similarity ($\cos(\theta)$): $$z = \|h\| \cdot \|w\| \cdot \cos(\theta)$$

Standard MaxLogit treats these components with equal weight. However, empirical analysis reveals that MaxCosine (direction) is generally a strong indicator of OOD samples, whereas MaxNorm (magnitude) can be an "encumbrance" due to high variance in standard training. The authors propose Decoupling MaxLogit (DML), a post-hoc score that balances these signals: $DML = \lambda \text{MaxCosine} + \text{MaxNorm}$.

DML+: Optimized Training for OOD Detection

To further enhance performance, the authors introduce DML+, which utilizes a dual-model training strategy to optimize the individual components:

  1. Model F (Focal): Trained with Focal Loss to refine angular boundaries by focusing on hard samples, thereby improving the reliability of MaxCosine.
  2. Model C (Center): Trained with Center Loss to create a compact feature space, making the feature norm (MaxNorm) a more discriminative signal for OOD detection.

Key Insights: Feature Convergence

The effectiveness of DML+ is rooted in how specific loss functions shape the feature space. The authors demonstrate a strong negative correlation between feature convergence metrics and OOD detection performance.

Correlation between feature convergence metrics (WFC, CFC) and OOD detection performance (MaxNorm, MaxCosine AUROC).

As shown in the figure above, reducing Within-class Feature Convergence (WFC) through Center Loss directly improves the MaxNorm AUROC. Similarly, optimizing the angular space improves the MaxCosine signal.

Experimental Results

DML+ achieves state-of-the-art performance across multiple benchmarks without requiring exposure to auxiliary OOD data during training.

Performance comparison of DML+ against state-of-the-art methods on CIFAR-100 and ImageNet benchmarks.

On the large-scale ImageNet-1k benchmark, DML+ achieves an AUROC of 93.16%, significantly outperforming existing methods like ViM (82.51%) and MaxLogit (86.60%). On CIFAR-100, it reaches 91.57% AUROC, a substantial jump from the previous SOTA, LogitNorm (85.73%). Furthermore, DML+ drastically reduces the False Positive Rate (FPR95) on ImageNet to 29.79%, compared to MaxLogit's 61.16%.

Conclusion

By decoupling the logit into direction and magnitude and optimizing them via specialized loss functions, DML+ provides a powerful yet principled approach to OOD detection. While the dual-model requirement increases computational overhead, the significant gains in reliability and accuracy set a new standard for the field.

Summary updated

BibTeX
@inproceedings{zhang2023decoupling,
  author = {Zihan Zhang and Xiang Xiang},
  title = {{Decoupling MaxLogit for Out-of-Distribution Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00330},
}
Figure