Abstract
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. However, they are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. The problem of overconfidence becomes especially apparent in cases where the test-time data distribution differs from that which was seen during training. We propose a solution to this problem by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels. Our method results in a better calibrated network and is agnostic to the underlying model structure, so it can be applied to any neural network which produces a probability density as an output. We demonstrate the effectiveness of our method and validate its performance on both classification and regression problems, applying it to recent probabilistic neural network models.
Introduction
Modern deep neural networks are powerful but notoriously overconfident, particularly when encountering data outside their training distribution. In a perfectly calibrated model, a 90% confidence prediction should correspond to a 90% actual success rate. However, standard networks often make wildly confident, incorrect predictions in "knowledge gaps" between training clusters. This lack of reliable epistemic uncertainty is a major safety concern for applications like autonomous driving or medical diagnosis.
Methodology: Prior Augmented Data (PAD)
This paper introduces Prior Augmented Data (PAD), a model-agnostic framework that uses an adversarial generative model to "hunt" for gaps in the training data where a model is unjustifiably certain. The system consists of two main components:
- The Generator ($g_\phi$): A set-encoding architecture that identifies gaps in the feature space. It is trained adversarially to seek out regions where the predictive model currently makes high-confidence (low-entropy) predictions.
- The Predictor ($f_\theta$): The main model is trained on both original data and generated pseudo-OOD data. For the latter, the model is penalized if its prediction deviates from a defined label prior (e.g., a uniform distribution for classification or a standard Gaussian for regression).
By forcing the model to admit ignorance in these generated regions, PAD effectively "teaches" a standard neural network to mimic the desirable behavior of Gaussian Processes (GPs)—which naturally revert to a prior in empty spaces—without the prohibitive computational cost of GPs.
Experimental Evaluation
The authors evaluated PAD across three primary domains:
- Regression: Using eight UCI datasets, the authors employed spectral clustering to split data into groups, training on some and testing on others to simulate distributional shifts.
- Classification: Robustness was tested on corrupted versions of MNIST and CIFAR-10 (MNIST-C, CIFAR-10-C).
- Temporal Shift: A New York Real Estate dataset spanning 12 years was used to evaluate how models handle market changes over time after being trained on a narrow 2008-2009 window.
Key Results
PAD-augmented models consistently outperformed their baselines in both calibration and likelihood metrics. In regression tasks, PAD improved Negative Log Likelihood (NLL) in nearly 90% of cases where baselines struggled. For classification, PAD significantly reduced Expected Calibration Error (ECE); for instance, MC Dropout's ECE on MNIST dropped from 0.31 to 0.07.
Temporal Robustness
In the real-world temporal shift task, PAD models maintained significantly more stable performance as the test data drifted further from the training period. While baseline models saw their calibration and NLL degrade sharply as the years progressed, PAD-augmented versions remained robust.
Discussion and Limitations
While PAD offers a powerful defense against overconfidence, it introduces additional training complexity due to the alternating adversarial training pattern. Future work may explore more sophisticated manifold geometry handling or direct weight-space regularization to achieve similar effects more efficiently. Nonetheless, PAD represents a significant step toward making deep learning models safer and more reliable in unpredictable, real-world environments.
Summary updated