Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks

Ahmad Rashid, Serena Hacker, Guojun Zhang, Agustinus Kristiadi, Pascal Poupart

Abstract

Discriminatively trained, deterministic neural networks are the de-facto choice for classification problems. However, even though they achieve state-of-the-art results on in-domain test sets, they tend to be overconfident on out-of-distribution (OOD) data. For instance, ReLU networks—a popular class of neural network architectures—have been shown to almost always yield high confidence predictions when the test data are far away from the training set, even when they are trained with OOD data. We overcome this problem by adding a term to the output of the neural network that corresponds to the logit of an extra class, that we design to dominate the logits of the original classes as we move away from the training data. This technique provably prevents arbitrarily high confidence on far-away test data while maintaining a simple discriminative point-estimate training. Evaluation on various benchmarks demonstrates strong performance against competitive baselines on both far-away and realistic OOD data.

Introduction

Standard deep neural networks, particularly those utilizing ReLU activation functions, suffer from a critical vulnerability: they are often overconfident in their predictions for data far outside their training distribution. Because ReLU networks are piecewise affine, their output logits grow linearly as inputs move away from the training data. In high-dimensional spaces, this allows a model to classify random noise or unrelated images with near-total certainty. This "blind spot" poses significant risks for safety-critical applications like autonomous driving or medical diagnostics, where a model must reliably signal uncertainty.

Methodology: The PreLoad Approach

To address this, the researchers introduced PreLoad (Producing Larger Logits away from Data). The method adds an auxiliary $(k+1)$-th class to the network's output layer to represent OOD samples. The key innovation is the mathematical definition of this extra class's logit ($z_{k+1}$). While standard classes use a linear transformation of the network's internal embedding $G(x)$, PreLoad employs a component-wise square of the embedding:

$$z_{k+1}(x) = w_{k+1}^\top G_\psi(x)^2 + b_{k+1}$$

Logit and probability behavior of PreLoad compared to standard models.

As shown in Figure 2, the quadratic term ensures that $z_{k+1}$ grows faster than the linear terms of the original classes. As an input $x$ moves infinitely far away, $z_{k+1}$ dominates the softmax denominator, forcing the predictive probability of the original classes to zero and the probability of the OOD class toward 1.0. This behavior is formally proven in the paper (Theorems 4 and 6), providing a mathematical guarantee against far-away overconfidence.

Experimental Evaluation

PreLoad was tested across a variety of benchmarks, including standard image sets (MNIST, CIFAR-10/100, SVHN), realistic OOD challenges (E-MNIST, K-MNIST, LSUN-CR), and synthetic "Far-Away" datasets designed to test extreme magnitudes.

Qualitatively, the impact of PreLoad is visible in the confidence surfaces of the model. While standard models and even those trained with Outlier Exposure (OE) maintain high confidence in regions far from the training data, PreLoad successfully constrains high-confidence predictions to the immediate vicinity of the data manifold.

Confidence surfaces for Standard, Outlier Exposure, and PreLoad models.

Key Results and Insights

  • Perfect Far-Away Detection: On synthetic "Far-Away" datasets, PreLoad achieved an FPR-95 of 0.0, perfectly distinguishing extreme OOD data from training data where standard models often failed completely.
  • Competitive Realistic OOD Performance: On standard benchmarks like CIFAR-100, PreLoad reduced the FPR-95 to 25.9%, a significant improvement over the standard model's 77.2%.
  • Improved Calibration: Beyond OOD detection, PreLoad improved the model's "calibration"—the alignment between confidence and accuracy—under distribution shifts like image rotation or corruption.
  • Efficiency: Unlike complex Bayesian or generative approaches, PreLoad maintains a simple point-estimate training procedure, avoiding the computational overhead of ensembles or GANs.

Conclusion

PreLoad offers a simple yet mathematically rigorous solution to the problem of overconfidence in ReLU networks. By leveraging quadratic logit growth, it ensures models remain uncertain when encountering data far from their training experience. Future work may involve extending this technique to handle adversarial examples and adapting it for Large Language Models (LLMs), where reliable uncertainty quantification remains a major challenge.

Summary updated

Abstract

Discriminatively trained, deterministic neural networks are the de facto choice for classification problems. However, even though they achieve state-of-the-art results on in-domain test sets, they tend to be overconfident on out-of-distribution (OOD) data. For instance, ReLU networks—a popular class of neural network architectures—have been shown to almost always yield high confidence predictions when the test data are far away from the training set, even when they are trained with OOD data. We overcome this problem by adding a term to the output of the neural network that corresponds to the logit of an extra class, that we design to dominate the logits of the original classes as we move away from the training data. This technique provably prevents arbitrarily high confidence on far-away test data while maintaining a simple discriminative point-estimate training. Evaluation on various benchmarks demonstrates strong performance against competitive baselines on both far-away and realistic OOD data.
BibTeX
@inproceedings{rashid2024preventing,
  author = {Ahmad Rashid and Serena Hacker and Guojun Zhang and Agustinus Kristiadi and Pascal Poupart},
  title = {{Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2024},
  url = {https://openreview.net/forum?id=zw8LRQYVDY},
}
Figure