TL;DR
This paper introduces GOOD (Guaranteed Out-of-distribution Detection), a training method that provides provable, worst-case guarantees that a neural network will maintain low confidence when encountering data outside its training domain. Unlike standard "certified robustness" techniques—which often severely degrade a model's accuracy on its primary task—GOOD achieves these safety guarantees with almost no loss in standard performance.
What Problem Does This Paper Address?
Deep neural networks are notoriously "overconfident." When shown an image that looks nothing like their training data (Out-of-Distribution or OOD data), they often classify it into a known category with near 100% certainty. In safety-critical fields like autonomous driving or medical diagnosis, this is a deal-breaker; we need systems that "know what they don't know."
While previous methods like Outlier Exposure (OE) improve OOD detection empirically, they are vulnerable to adversarial OOD attacks. An attacker can slightly perturb an OOD image (like adding a subtle noise pattern) to make the model suddenly highly confident that it's seeing a familiar class, like a "dog." This paper moves beyond empirical "cat-and-mouse" defenses to provide mathematical certificates: a guarantee that for any perturbation within a certain range, the model's confidence will remain below a safe threshold.
Methods
The authors propose a training objective called GOOD (Guaranteed Out-of-distribution Detection). The core technical components are:
- Interval Bound Propagation (IBP): This is a technique used to track "worst-case" values through the layers of a neural network. By treating the input as an interval (an $l_\infty$-ball) rather than a single point, IBP allows the authors to calculate a provable upper bound on the model's confidence for any point within that neighborhood.
- Confidence Upper Bound (CUB) Loss: Instead of just minimizing confidence on specific OOD points, the authors minimize the upper bound of the confidence across the entire neighborhood around those points. They use a specific logarithmic loss formulation to keep training numerically stable.
- Quantile-GOOD ($GOOD_Q$): A major challenge is that massive OOD training sets (like "80 Million Tiny Images") often contain images that actually look like the in-distribution data. Forcing low confidence on these "overlapping" images would hurt the model's primary accuracy. The authors solve this by only enforcing the guarantee on a specific quantile (e.g., the "easiest" 80%) of the OOD data.
Datasets
To validate their approach, the authors used a variety of standard benchmarks:
- In-Distribution (Training/Eval): MNIST (digits), SVHN (street view house numbers), and CIFAR-10 (objects).
- Out-Distribution (Training): 80 Million Tiny Images (used as a proxy for "all natural images").
- Out-Distribution (Evaluation/Testing): FashionMNIST, EMNIST Letters, grayscale CIFAR-10, Uniform Noise, CIFAR-100, and LSUN Classroom. These were used to see if the guarantees generalize to OOD data the model never saw during training.
Key Insights and Ideas
- The "No-Free-Lunch" Exception: In the world of adversarial robustness, there is usually a massive trade-off: if you want a model to be robust to attacks, its clean accuracy usually drops significantly. The authors find that for OOD detection, this trade-off is much smaller. You can have a "safe" model that is still just as accurate as a "standard" one.
- Generalization of Guarantees: Training to be robust on one broad OOD dataset (Tiny Images) actually provides provable safety on completely different OOD datasets (like EMNIST letters).
- Handling Ambiguity: The EMNIST experiment is particularly insightful. When a digit classifier sees the letter "I," it should be confident it's a "1." When it sees an "H," it should have low confidence.
As shown above, GOOD is uniquely capable of distinguishing between "OOD that looks like a class" and "OOD that is truly unknown," correctly assigning high confidence to ambiguous letters and low confidence to distinct ones.
Key Results
- SVHN: $GOOD_{100}$ achieved 96.3% accuracy (better than the plain model) while providing a Guaranteed AUC (GAUC) of 97.3% against $l_\infty$ perturbations. This means that even under attack, the model's OOD detection remains nearly perfect.
- CIFAR-10: Using the quantile approach ($GOOD_{80}$), the authors maintained 90.1% accuracy while achieving a GAUC of over 40-50% on most OOD tasks—a massive improvement over standard models which have 0% GAUC (meaning they can always be fooled).
- Provable Superiority: In almost all tests, the guaranteed worst-case performance of GOOD was better than the empirical (attacked) performance of previous state-of-the-art methods like OE or ACET.
Limitations and Future Work
- Dataset Overlap: The method relies on the OOD training set being somewhat distinct from the in-distribution set. If the OOD data is too similar, the Quantile-GOOD approach is necessary, but choosing the right quantile $Q$ remains a hyperparameter challenge.
- IBP Tightness: While IBP is fast and allows for scaling to larger networks, the bounds it provides can be "loose" (overly conservative). Exploring tighter bounding methods that remain computationally efficient is a suggested path forward.
- Scaling to High Resolution: The experiments were conducted on relatively low-resolution images (32x32). Scaling these certificates to high-resolution ImageNet-scale data remains an open challenge in the field.
Summary updated