Abstract
Deep neural networks are known to be overconfident when applied to out-of-distribution (OOD) inputs which clearly do not belong to any class. This is a problem in safety-critical applications since a reliable assessment of the uncertainty of a classifier is a key property, allowing the system to trigger human intervention or to transfer into a safe state. In this paper, we aim for certifiable worst case guarantees for OOD detection by enforcing not only low confidence at the OOD point but also in an $l_\infty$-ball around it. For this purpose, we use interval bound propagation (IBP) to upper bound the maximal confidence in the $l_\infty$-ball and minimize this upper bound during training time. We show that non-trivial bounds on the confidence for OOD data generalizing beyond the OOD dataset seen at training time are possible. Moreover, in contrast to certified adversarial robustness which typically comes with significant loss in prediction performance, certified guarantees for worst case OOD detection are possible without much loss in accuracy.
The Problem: Overconfidence and Fragility
Deep learning models are often "black boxes" that fail to signal when they encounter data outside their training distribution. When presented with an image that doesn't belong to any of its trained classes (Out-of-Distribution or OOD data), a standard model often assigns it a high-confidence label. While empirical defenses like Outlier Exposure (OE) have been developed to lower this confidence, they remain fragile. An attacker can find a tiny, invisible perturbation—an adversarial attack—that flips the model from "low confidence" back to "100% certain" for an OOD input. In safety-critical fields like medical diagnosis or autonomous driving, we require provable guarantees that a model will remain uncertain within a neighborhood around OOD points.
Methodology: The GOOD Framework
The authors propose GOOD (Guaranteed Out-of-distribution Detection), a training framework designed to provide these certificates. The core of GOOD relies on Interval Bound Propagation (IBP). IBP is used to mathematically bound the maximum possible confidence a model can have for any point within a small $l_\infty$-ball radius of an OOD sample.
By minimizing a differentiable Confidence Upper Bound (CUB) Loss during training, the model is forced to be "certifiably" uncertain. To prevent this constraint from degrading performance on the primary task, the authors introduce Quantile-GOOD. This variant only enforces guaranteed low confidence on a specific percentage ($q$) of the "easiest" OOD samples, allowing the model to maintain high accuracy on in-distribution data even when OOD samples are ambiguous.
Key Results and Insights
A significant finding is that, unlike adversarial robustness for in-distribution data, certified OOD detection does not necessarily require an "accuracy tax." GOOD models often maintain or even slightly improve baseline accuracy while providing strong guarantees.
- SVHN Robustness: The GOOD100 model maintained 96.3% accuracy (surpassing the baseline) while achieving near-perfect Guaranteed AUC (GAUC) against adversarial OOD attacks.
- CIFAR-10 Trade-off: On the more challenging CIFAR-10 dataset, Quantile-GOOD (specifically GOOD80) maintained 90.1% accuracy while providing significantly stronger worst-case OOD detection than previous methods.
- Generalization and Human-like Uncertainty: Certificates trained on one set of images (e.g., Tiny Images) generalized well to completely different data types. On the EMNIST dataset, GOOD100 achieved a state-of-the-art 99.0% AUC.
As illustrated above, the GOOD model exhibits "human-like" uncertainty, only showing high confidence when an OOD letter truly resembles an in-distribution digit (such as a lowercase 'l' looking like a '1').
Limitations and Future Work
Despite its strengths, GOOD training can be unstable, requiring careful hyperparameter "warm-up" schedules for the perturbation radius and loss weighting. Additionally, choosing the optimal quantile $q$ for Quantile-GOOD remains a heuristic process. Finally, scaling these provable guarantees to high-resolution datasets like ImageNet remains a significant computational challenge for future research.
Summary updated