Beyond In-Domain Scenarios: Robust Density-Aware Calibration

Christian Tomani, Futa Waseda, Yuesong Shen, Daniel Cremers

Abstract

Deep neural networks are frequently miscalibrated, exhibiting high confidence even when making incorrect predictions. This issue is exacerbated under domain shift and out-of-distribution (OOD) scenarios, where standard calibration techniques often fail. This paper introduces Density-Aware Calibration (DAC), a post-hoc method that leverages the density of training data in a model's hidden layers to adjust confidence scores. By using k-nearest neighbors (KNN) as a density proxy, DAC identifies samples in low-density regions and scales their confidence accordingly. Experimental results across CIFAR and ImageNet benchmarks demonstrate that DAC significantly improves the reliability of uncertainty estimates without compromising predictive accuracy.

1. Introduction

Modern deep learning models are notoriously overconfident. While a model might achieve high accuracy, its predicted probabilities often fail to reflect the true likelihood of correctness—a property known as calibration. While existing methods like Temperature Scaling effectively calibrate models on in-domain data, they struggle when the test distribution deviates from the training distribution (e.g., due to sensor noise, weather changes, or novel objects).

In safety-critical applications such as autonomous driving or medical diagnosis, a model must "know what it doesn't know." This paper addresses the challenge of calibration under domain shift and OOD scenarios by proposing Density-Aware Calibration (DAC), a lightweight, post-hoc approach that utilizes internal feature representations to detect and mitigate overconfidence.

2. The Problem of Overconfidence

Standard calibration methods typically rely on the final output layer (logits). However, these methods assume that the test data follows the same distribution as the training data. When a model encounters a sample from a "low-density" region—where training data was sparse—it often produces high-confidence predictions despite a lack of evidence. Existing solutions to this problem often require expensive retraining or the generation of synthetic shifted data, which may not accurately represent real-world shifts.

3. Methodology: Density-Aware Calibration (DAC)

DAC is designed as a plug-and-play module that can be applied to any pre-trained classifier. Its core premise is that the density of training samples in a model's feature space is a strong indicator of predictive reliability.

3.1 Density Proxy via KNN

For any given input $x$, DAC calculates the distance to its $k$-th nearest neighbor in the training set. Unlike methods that only look at the final layer, DAC extracts these distances across multiple hidden layers of the network. This allows the method to capture uncertainty-related information that might be discarded by the time the signal reaches the final output.

3.2 Learnable Scaling and Logit Adjustment

DAC learns a scaling function $S(x, w)$, which is a linear combination of the KNN distances across layers plus a bias term. The final calibrated output is achieved by dividing the model's original logits by this scaling factor:

$$\hat{y} = \text{softmax}(\frac{\text{logits}}{S(x, w)})$$

Because this is a simple scalar division (similar to Temperature Scaling), the relative ranking of classes remains unchanged. Consequently, DAC is accuracy-preserving—it improves the reliability of the confidence score without changing the model's predicted class.

3.3 Hybrid Implementation

DAC is typically used in conjunction with existing calibration methods like Temperature Scaling or Splines. This hybrid approach allows the model to handle both standard in-domain calibration and the unique challenges posed by distribution shifts simultaneously.

4. Experimental Evaluation

The authors evaluated DAC using a variety of architectures (ResNet, DenseNet, Vision Transformers) across several challenging benchmarks:

  • Standard Datasets: CIFAR-10, CIFAR-100, and ImageNet-1k.
  • Domain Shift: CIFAR-C and ImageNet-C, featuring 18 corruption types (e.g., blur, fog, noise) at varying severities.
  • Real-world OOD: ObjectNet-OOD, which tests models on objects in unusual orientations and backgrounds.

5. Key Insights and Results

5.1 Robustness to Corruption

DAC consistently reduced the Expected Calibration Error (ECE) across all levels of data corruption. In high-severity scenarios (Severity 5), DAC improved calibration by up to an order of magnitude compared to baseline methods. When integrated with existing state-of-the-art techniques, DAC provided relative improvements ranging from 5% to 40%.

5.2 The Importance of Hidden Layers

A critical finding of the research is that intermediate layers contain significant uncertainty signals. By aggregating density information from multiple depths, DAC outperforms methods that rely solely on the final feature representation.

5.3 Computational Efficiency

Despite the use of KNN, DAC is computationally efficient. Training the scaling function on a DenseNet169 for ImageNet takes approximately 14 minutes on a single GPU. Inference time remains low, making it suitable for real-time applications.

6. Limitations and Future Work

While effective, DAC's performance can be sensitive to the choice of layers. While the authors found that a subset of layers is generally sufficient, the optimal selection can vary by architecture. Additionally, for extremely large models (e.g., those with hundreds of layers), the scaling function could potentially overfit the validation set if not properly regularized. Future research will focus on automating layer selection and exploring the theoretical reasons why certain layers provide better density signals than others.

7. Conclusion

Density-Aware Calibration offers a robust, efficient, and accuracy-preserving solution for the overconfidence problem in deep learning. By grounding confidence in the density of the training data across hidden layers, DAC ensures that models remain reliable even when faced with the unpredictable nature of real-world data shifts.

Summary updated

BibTeX
@inproceedings{tomani2023beyond,
  author = {Christian Tomani and Futa Waseda and Yuesong Shen and Daniel Cremers},
  title = {{Beyond In-Domain Scenarios: Robust Density-Aware Calibration}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=taBCtI0m5Y},
}
Figure