Understanding the Feature Norm for Out-of-Distribution Detection

Jaewoo Park, Jacky Chen Long Chai, Jaeho Yoon, Andrew Beng Jin TeohICCV 2023 · DOI · arXiv

Abstract

Deep neural networks often exhibit overconfidence when encountering out-of-distribution (OOD) data, posing significant safety risks. While empirical evidence suggests that the vector norm of internal feature representations is a potent signal for OOD detection, the theoretical basis for this phenomenon has remained elusive. This paper provides a theoretical link between feature norms and the confidence of internal "hidden classifiers." We introduce the Negative-Aware Norm (NAN), a hyperparameter-free metric that incorporates both neuron activation and deactivation signals. NAN achieves state-of-the-art OOD detection performance across multiple benchmarks, including ImageNet-1k.

Introduction

A critical challenge in deploying deep learning models in safety-critical applications is their tendency to assign high-confidence scores to OOD samples. Previous research identified a curious trend: in-distribution (ID) samples typically elicit larger feature norms than OOD samples. However, these observations were largely empirical.

Evolution of feature norms for in-distribution (ID) and out-of-distribution (OOD) data during training.

As shown in Figure 1, the gap between ID and OOD feature norms emerges early in training and correlates with the model's ability to distinguish between the two distributions. This paper demystifies this gap and leverages it to create a more robust OOD detector.

The "Hidden Classifier" and the Deactivation Signal

The authors prove that the $l_1$-norm of a feature vector is mathematically equivalent to the maximum logit of a "hidden classifier" embedded within the network's weights. This implies that the feature norm is not merely a statistical artifact but a direct proxy for the network's internal certainty.

A key insight of this work is the role of neuron deactivation. While standard norms measure activation, ID samples also cause more neurons to stay at zero (deactivate) compared to OOD samples. OOD data tends to produce a "mushy" response where neurons are neither strongly active nor completely silent.

The reciprocal of the l0-norm (sparsity) and performance metrics across network layers.

Figure 5 illustrates that the reciprocal of the $l_0$-norm (a measure of sparsity or deactivation) provides a clear separation between ID and OOD data, particularly in deeper layers of the network.

Methods: Negative-Aware Norm (NAN)

To capture both activation and deactivation signals, the authors propose the Negative-Aware Norm (NAN):

$$\lVert \mathbf{a} \rVert_{\text{NAN}} = \lVert \mathbf{a} \rVert_1 \cdot \lVert \mathbf{a} \rVert_0^{-1}$$

The first term ($\lVert \mathbf{a} \rVert_1$) captures the strength of activation, while the second term ($\lVert \mathbf{a} \rVert_0^{-1}$) captures the sparsity of the representation. NAN is entirely hyperparameter-free and "plug-and-play," requiring no specialized training or stored feature banks.

Experimental Results

The proposed method was evaluated on standard benchmarks including CIFAR-10, CIFAR-100, and ImageNet-1k, using OOD datasets such as LSUN, iSUN, and SVHN.

  • State-of-the-Art Performance: On ImageNet-1k, NAN achieved an average AUROC of 92.32%, outperforming standard baselines like Maximum Softmax Probability (86.88%) and Energy-based scores (90.42%).
  • Error Reduction: NAN reduced the False Positive Rate (FPR95) by 11-26% compared to standard feature norms without requiring any hyperparameter tuning.
  • Versatility: NAN works effectively for both supervised and self-supervised models (e.g., MoCo-v2) and can be combined with other techniques like ReAct for further gains, reaching a SOTA AUROC of 94.61% on ImageNet.

Conclusion and Future Work

This work provides a theoretical foundation for feature-norm-based OOD detection and introduces a simple yet powerful metric, NAN. By recognizing that "silence" in neuron activation is as informative as "noise," NAN sets a new standard for hyperparameter-free OOD detection. Future research may explore using this "hidden classifier" perspective to regularize models during training for inherently better OOD robustness.

Summary updated

BibTeX
@inproceedings{park2023understanding,
  author = {Jaewoo Park and Jacky Chen Long Chai and Jaeho Yoon and Andrew Beng Jin Teoh},
  title = {{Understanding the Feature Norm for Out-of-Distribution Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00150},
}
Figure