TL;DR
Out-of-Distribution (OOD) detectors are crucial for safely deploying machine learning models, yet they are highly vulnerable to adversarial attacks. This paper introduces ROSS, a post-hoc, gradient-free framework that uses median smoothing and score instability (Median Absolute Deviation) to provide symmetric robustness against both score-maximizing and score-minimizing attacks, outperforming prior methods by up to 40 AUROC points.
What Problem Does This Paper Address?
Deep neural networks (DNNs) are notorious for making highly confident but completely incorrect predictions when faced with data outside their training distribution (Out-of-Distribution, or OOD data). While researchers have developed OOD detectors to flag these unsafe inputs, recent work shows that these detectors themselves can be easily fooled by adversarial perturbations.
Existing defenses that use input perturbations (like ODIN or PRO) suffer from an asymmetric vulnerability. Because they rely on directional, gradient-based optimization to either artificially boost or reduce model confidence, they successfully defend against one type of attack direction but remain completely blind to the opposite one (e.g., defending against score-minimizing attacks while failing against score-maximizing attacks). The authors address this gap by proposing a defense that is robust against attacks from any direction without requiring expensive model retraining.
Methods
The authors introduce ROSS (Robust OOD Detector via Synergistic Smoothing), a model-agnostic, post-hoc framework that can be applied to any standard OOD scoring function (such as MSP, Energy, or GEN).
Instead of using gradients to perturb the input in a specific direction, ROSS generates $N$ noisy variants of an input by adding random Gaussian noise. It passes these variants through the base OOD detector to create a "score stack" and extracts two robust statistical metrics:
- Median Score ($S_{med}$): Represents the robust central tendency of the smoothed OOD score.
- Score Stability ($\omega_{med}$): Measured via the Median Absolute Deviation (MAD), this quantifies the local instability of the score landscape.
Crucially, ROSS employs a confidence-thresholded scoring function. It only applies a stability bonus if the median score exceeds a threshold ($S_{95}$, defined as the 5th percentile of the validation set). This gating mechanism ensures that "far-OOD" samples—which might be highly stable but consistently low-confidence—are not mistakenly rewarded and misclassified as in-distribution (ID).
Datasets
The framework was evaluated using the standardized protocol from the OpenOOD benchmark:
- In-Distribution (ID) Datasets: CIFAR-10, CIFAR-100, and ImageNet.
- Out-of-Distribution (OOD) Datasets: Divided into near-OOD (CIFAR-100, TinyImageNet) and far-OOD (MNIST, SVHN, Texture, Places365).
Key Insights and Ideas
- Instability as an OOD Signal: ID samples typically sit in smooth, well-defined regions of the model's score landscape, meaning small random perturbations won't change their scores much. OOD samples inhabit uncharted territory where the model behaves erratically, leading to high score variance (instability) under the exact same perturbations.
- The Danger of Gradients in Defenses: Relying on gradient-based, directional optimization to calculate OOD scores creates a fundamental blind spot. By switching to a gradient-free, non-directional approach using random perturbations, ROSS achieves symmetric robustness against both score-minimizing and score-maximizing attacks.
- The Necessity of Gating: Simply rewarding score stability across the board fails because some completely unrelated "far-OOD" inputs produce highly stable, flat, but low-confidence score landscapes due to network indifference. Gating via the $S_{95}$ threshold ensures stability only benefits plausible ID candidates.
Key Results
- Symmetric Adversarial Robustness: Under severe adversarial attacks (PGD-Max, $\epsilon = 8/255$) on CIFAR-10, the baseline Maximum Softmax Probability (MSP) detector's performance collapses to an AUROC of 3.76%. Applying ROSS (ROSS-MSP) recovers this performance substantially to 52.65%.
- Outperforming Prior Perturbation Methods: On the CIFAR-100 benchmark under PGD-Max attacks ($\epsilon = 4/255$), the prior state-of-the-art perturbation method, PRO-fDBD, actually degrades the baseline performance down to 28.38% AUROC due to its directional nature. ROSS-fDBD avoids this trap, achieving 58.48% AUROC—representing a massive 30+ point absolute improvement.
- Complexity Amplifies Gains: The performance improvements delivered by ROSS were significantly more pronounced on CIFAR-100 than on CIFAR-10, indicating that ROSS is particularly effective at safeguarding the complex, high-dimensional decision boundaries found in many-class problems.
Limitations and Future Work
- Not a Complete Substitute for Adversarial Training: While ROSS provides an easily deployable, lightweight post-hoc first line of defense, extremely sophisticated or intense attacks may still require computationally heavy adversarial training methods (like ALOE or ATOM).
- Scope of Attacks: The evaluation primarily focused on gradient-based adversarial attacks. The authors note that future work should explore extending ROSS to handle broader distribution shifts, such as semantic perturbations, real-world environmental corruptions, and shifts induced by generative models.
Summary updated