ATS (Adaptive Temperature Scaling) is a novel post-hoc technique that significantly boosts Out-of-Distribution (OOD) detection by calculating a unique "temperature" for every single input sample. By extracting statistical cues from intermediate neural network layers and using them to scale final output logits, ATS reduces false positive rates by an average of nearly 30% on standard benchmarks without requiring any model retraining.
The Overconfidence Problem in OOD Detection
Deep learning models are notoriously "overconfident" even when they encounter data they have never seen before. In safety-critical applications like autonomous driving or medical diagnosis, a model needs to know when to say "I don't know."
Current "post-hoc" detection methods—which work on pre-trained models without needing extra training—typically only look at the final output (the logits or softmax probabilities). The authors argue that this is a missed opportunity: by the time data reaches the final layer, much of the statistical information that distinguishes "familiar" from "strange" data has already been filtered out. Furthermore, most existing methods use a fixed, global temperature parameter to smooth these outputs, which fails to account for the unique characteristics of individual images.
Methodology: Adaptive Temperature Scaling (ATS)
The authors introduce Adaptive Temperature Scaling (ATS). Instead of using a single fixed value to scale the model's outputs, ATS calculates a sample-specific temperature $\hat{T}(x)$ through a multi-step process involving intermediate activation extraction and statistical calibration.
The process follows these steps:
- Intermediate Activation Extraction: As an image passes through the network, ATS records the mean activations from several intermediate layers.
- Statistical Calibration (eCDF): During a one-time "calibration" phase using the training data, the method pre-computes the empirical Cumulative Distribution Function (eCDF) for these activations.
- P-Value Calculation: At test time, the model compares a new sample's activations against the stored eCDF to generate a $p$-value for each layer. This $p$-value represents how "normal" the activation looks compared to the training set.
- Fisher’s Method Fusion: These layer-wise $p$-values are aggregated using Fisher’s method into a single temperature value.
- Logit Scaling: This temperature is used to scale the final logits. Because OOD samples typically produce "abnormal" intermediate activations, they result in higher temperatures, which flattens their probability distribution and makes them easier to flag as OOD.
Crucially, ATS is plug-and-play: it can be combined with existing state-of-the-art OOD detectors like Energy-based scores, ReAct, or ASH.
Evaluation and Key Results
The researchers evaluated ATS across a wide variety of computer vision benchmarks, including CIFAR-10, CIFAR-100, and ImageNet-1k as In-Distribution (ID) sets. OOD datasets included SVHN, Textures, iSUN, LSUN, Places365, MNIST, Fashion-MNIST, and NINCO.
ATS consistently improved every baseline method it was tested against (MSP, ODIN, MLS, Energy, ReAct, DICE, and ASH-B).
- CIFAR-100: When applied to a ResNet18, ATS reduced the average FPR95 (False Positive Rate at 95% recall) of baseline methods by 29.62%.
- ImageNet: On the much larger ImageNet benchmark, ATS provided a solid improvement, reducing the average FPR95 by 7.91%.
- Zero Performance Degradation: Because temperature scaling doesn't change the relative ranking of the logits, the model’s original classification accuracy on standard data remains completely untouched.
Insights and Discussion
- Information Recovery: Intermediate layers act as a statistical "early warning system." Information about a sample being OOD is often present in shallow or middle layers before it is "washed away" by the final classification layers.
- Sample-Specific vs. Global: Moving from a global hyperparameter to a per-sample adjustment allows the model to be much more surgical. An ID sample gets a low temperature (preserving its confidence), while a confusing OOD sample gets a high temperature (diluting its confidence).
- Limitations: For certain datasets like Places365 (when used as OOD for ImageNet), intermediate layers provided less useful information due to high semantic similarity. Additionally, there is a modest computational overhead (approx. 6-12% increase in inference time).
ATS demonstrates that looking "under the hood" of neural networks during inference provides valuable signals for identifying unfamiliar data, offering a simple yet powerful enhancement for existing OOD detection frameworks.
Summary updated