Abstract
Discriminatively trained, deterministic neural networks are the de facto choice for classification problems. However, even though they achieve state-of-the-art results on in-domain test sets, they tend to be overconfident on out-of-distribution (OOD) data. For instance, ReLU networks—a popular class of neural network architectures—have been shown to almost always yield high confidence predictions when the test data are far away from the training set, even when they are trained with OOD data. We overcome this problem by adding a term to the output of the neural network that corresponds to the logit of an extra class, designed to dominate the logits of the original classes as we move away from the training data. This technique, termed PreLoad, provably prevents arbitrarily high confidence on far-away test data while maintaining simple discriminative point-estimate training. Evaluation on various benchmarks demonstrates strong performance against competitive baselines on both far-away and realistic OOD data.
Introduction: The Overconfidence Problem
Deep Neural Networks (DNNs), particularly those utilizing ReLU activations, exhibit a problematic behavior: they often produce high-confidence predictions (approaching 100%) for inputs that are entirely unrelated to the training distribution. This overconfidence is a significant barrier to deploying AI in safety-critical domains such as autonomous driving or medical diagnostics.
The technical root of this issue lies in the "piecewise affine" nature of ReLU networks. As an input moves sufficiently far from the training data, the network's internal mappings eventually stabilize into simple linear functions. Consequently, the "logits" (raw scores before the final softmax layer) continue to grow linearly. The softmax function then squashes these scores, eventually yielding a probability of 1.0 for one of the known classes. While existing techniques like Outlier Exposure (OE) improve detection for "nearby" outliers, they often lack mathematical guarantees for data that is "far away."
Methodology: The PreLoad Approach
The authors propose PreLoad (Producing re-Larger ogits away from data) to address this fundamental architectural flaw. The method introduces an elegant architectural modification:
- The Extra Class: An additional $(k+1)$-th class is added to the network to represent "Out-of-Distribution" (OOD) or "None."
- Quadratic Growth Mechanism: Unlike standard class logits, which are linear functions of the network's penultimate layer $G(x)$, the PreLoad logit for the OOD class is defined using the square of the penultimate layer: $$z_{k+1}(x) = w_{k+1}^\top G(x)^2 + b_{k+1}$$
- Dominance Guarantee: Because the logits for the original $k$ classes grow at most linearly, the quadratic growth of the OOD logit ensures it will mathematically dominate as the input magnitude increases. This forces the softmax probability of any in-distribution class toward zero.
The model is trained using standard cross-entropy loss. In-distribution data is used for the first $k$ classes, while an auxiliary OOD dataset (e.g., random noise or generic images) is used to train the $(k+1)$-th class to recognize "not-data."
Experimental Evaluation
The researchers evaluated PreLoad across a diverse set of benchmarks:
- In-distribution: MNIST, Fashion-MNIST, SVHN, CIFAR-10, and CIFAR-100.
- OOD Benchmarks: E-MNIST, K-MNIST, LSUN-CR, and various noise types.
- Far-Away Data: Specifically designed "FarAway" datasets where images are scaled by massive factors (e.g., 10,000) to test the limits of model confidence.
- Dataset Shift: Rotated MNIST and Corrupted CIFAR-10 (CIFAR-10-C) were used to assess calibration under distribution shift.
Key Results and Insights
Provable Humility and Detection
The most significant contribution is the mathematical proof that for any ReLU network, PreLoad ensures the probability of any original class must vanish as the input moves infinitely far from the training data. In practice, this led to a 0.0% False Positive Rate (FPR-95) on "FarAway" datasets. In contrast, standard models and even some OOD-trained baselines reached nearly 100% FPR, being completely fooled by scaled noise.
Competitive Performance on Realistic OOD
On standard OOD benchmarks (e.g., CIFAR-10 vs. CIFAR-100), PreLoad performed as well as or better than state-of-the-art baselines. For instance, on SVHN, it achieved an FPR-95 of 1.1%, outperforming the standard Outlier Exposure baseline.
Improved Calibration
Under dataset shifts like image corruption, the fine-tuned version of PreLoad (PreLoad-FT) demonstrated significantly lower Expected Calibration Error (ECE). This indicates that the model's confidence scores are more reliable indicators of its actual predictive accuracy.
Conclusion
PreLoad offers a simple yet mathematically grounded solution to the overconfidence problem in discriminative neural networks. By maintaining the simplicity of point-estimate training while providing guarantees for far-away data, it bridges the gap between standard discriminative models and more complex Bayesian or generative approaches. Future work may explore its application to other domains, such as Large Language Models, or its efficacy against adversarial attacks.
Summary updated