A classifier deployed in the open world needs a second component: an out-of-distribution (OOD) detector, which assigns each input a score saying whether it looks like the training classes and rejects the input if it does not. Detectors are now accurate on standard benchmarks. They also collapse when an attacker who knows the scoring function nudges pixels by an imperceptible amount. Two papers have shown that the collapse can be repaired without the usual cost to clean accuracy. Both perturb only the outlier training data, so the classifier itself is left unconstrained. This runs against the familiar rule that adversarial robustness costs clean accuracy, and the literature has not explained why the rule seems to have an exception here.
We argue that the exception is an artifact of which attack was defended. An attacker facing a classifier plus detector has three goals, not one. The first is to disguise a foreign input so that it is accepted. The second is to make a legitimate input get rejected, a form of denial of service. The third is to make a legitimate input get misclassified while it still passes the detector. Existing "free" defenses train only against the first goal. We predict that the second goal can also be defended almost for free, but the third cannot. Any detector that is stable around clean inputs will pass their adversarial neighbors along with them. The cost of the third goal therefore has to be paid somewhere: in clean accuracy, through classifier robustness, or in coverage, by rejecting clean inputs whose predictions are fragile. The project builds the three-goal adaptive evaluation, proves the simple tension behind the prediction, and measures both cost curves.
Core Claim
Let $S(x)$ be the detector's in-distribution score, where an input is accepted when $S(x) > \tau$ for a threshold $\tau$. Let $h(x)$ be the classifier's prediction. Suppose we defend the denial-of-service goal by making $S$ change by at most $L\epsilon$ over any perturbation of size $\epsilon$ around clean inputs. Then every clean input with $S(x) > \tau + L\epsilon$ is still accepted after an $\epsilon$-perturbation. This is a one-line consequence of the stability bound:
$$\Pr[\text{accepted and misclassified under attack}] \;\ge\; \Pr\big[S(x) > \tau + L\epsilon \ \text{and}\ h \text{ is not } \epsilon\text{-robust at } x\big].$$
The detector cannot lower the right-hand side without either rejecting clean inputs, which lowers the first event, or making the classifier robust, which lowers the second. The first route costs coverage. The second costs clean accuracy through the classic trade-off.
The claim has three testable parts. First, robustness against disguised outliers and robustness against false rejection are both achievable with negligible loss of clean accuracy, as long as the stability constraint is applied to the score and not to the class logits. Second, robustness against accepted misclassification sits on a genuine frontier. Third, paying for it in coverage is cheaper than paying in accuracy for at least part of that frontier.
Prior Work and Its Role Here
The motivating evidence comes from Adversarial vulnerability of powerful near out-of-distribution detection. That paper attacks OOD scores directly with gradients at $\epsilon = 1/255$ on a ViT-L/16. Mahalanobis AUROC falls from 97.98% to 41.33% on CIFAR-100 vs. CIFAR-10, and the best defense it finds, Relative Mahalanobis with ensembling, holds 78.64%. It covers the first two goals implicitly by pushing scores in both directions. It trains nothing and does not measure clean accuracy or the joint misclassify-and-accept goal.
Two papers establish the apparent free lunch, and we build on them as methods.
- Certifiably Adversarially Robust Detection of Out-of-Distribution Data (GOOD) bounds worst-case confidence on perturbed outliers using interval bound propagation, a technique that pushes boxes of possible inputs through the network. It reports 96.3% clean accuracy with 97.3% guaranteed AUC on SVHN, and 90.1% clean accuracy with 40–50% guaranteed AUC on CIFAR-10.
- Out-of-Distribution Detection with Adversarial Outlier Exposure (AOE) perturbs generated outliers toward the training data and trains the energy score to reject them. It reports 93.7% AUROC and 24.7% FPR95 on CIFAR-100 while keeping 94.4% CIFAR-10 accuracy.
Both perturb only outliers. Our claim explains their result: they defend the first goal and never pay for the third.
Two papers constrain our design. OODRobustBench documents that classifier robustness is costly and brittle: it degrades 18–31% under dataset shift and hits a projected ~43% ceiling on shifted ImageNet. This tells us that the accuracy route for the third goal will be expensive. DeepCloak shows that static feature masking falls to attackers who search within the unmasked features. This warns us that any rejection rule we add must be attacked adaptively and not evaluated statically.
Post-hoc subspace detectors, such as Activation Subspaces for Out-of-Distribution Detection and Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection, preserve accuracy exactly because they never retrain. They serve as zero-cost reference points whose adaptive robustness has never been tested.
Research Program
Evaluation. We first build an adaptive evaluation that attacks each defense with three separate objectives:
- lowering the score of outliers (disguise);
- lowering the score of clean inputs until they are rejected (denial of service);
- a joint objective that maximizes a wrong-class loss plus a hinge penalty whenever the score drops below $\tau$ (misclassify but stay accepted).
Every attack differentiates through the full pipeline, including any rejection rule we add. It uses multi-step projected gradient descent with restarts and step-size schedules tuned per defense. Thresholds are fixed at 95% clean true-positive rate before the attack, so an attacker cannot win merely by shifting the operating point.
Frontier study. Second, we train a family of models that isolate where cost enters. Starting from energy-score training with outlier exposure, we add a score-stability term on clean inputs, in the style of TRADES but applied only to the scalar score $S$, weighted by $\lambda_S$. Separately, we add standard adversarial training on the class logits, weighted by $\lambda_h$. Sweeping both weights traces clean accuracy against robustness on each of the three goals.
Coverage-based defense. Third, we test whether coverage is a cheaper way to pay. We add an abstention signal that rejects inputs whose classifier margin collapses under a short internal perturbation search, and we train it jointly with the detector. Then we attack the whole system, including the abstention search, with the joint objective. We report accuracy on accepted inputs against the fraction of clean inputs rejected. This is the setting where the tax is paid in coverage.
Baselines
Naive reference. A standard cross-entropy network scored by energy, and the same network with ordinary outlier exposure. These show what the three attacks do with no defense.
Strongest detection-side competitor. AOE, reimplemented with the authors' single-step outlier perturbation and also with a multi-step variant. On the disguise goal, the reported figure to match is 93.7% clean AUROC on CIFAR-100 with 94.4% CIFAR-10 accuracy (Botschen et al.). GOOD runs alongside it as the certified competitor, with reported 90.1% clean accuracy on CIFAR-10 (Hein et al.).
Strongest post-hoc competitor. Relative Mahalanobis with an ensemble. The target is its reported 78.64% attacked AUROC (Fort). ActSub and P-OCS are added as untested subspace detectors.
Accuracy-route competitor. A PGD-adversarially-trained backbone combined with outlier exposure. This is the obvious way to defend the joint goal.
Fairness conditions. All trained methods share the same architecture (a WideResNet on CIFAR, plus a ViT fine-tuned from the same checkpoint for the ViT experiments), the same outlier source, the same number of gradient steps per batch counted in forward-backward passes, and the same hyperparameter search budget.
Reported numbers. No reported result was found on Lacuna for any method under the joint misclassify-and-accept attack, and none for the denial-of-service goal under training-time defenses.
What counts as beating the baselines. On the first two goals, a defense must match AOE's attacked AUROC while losing under half a point of clean accuracy. On the joint goal, a method beats PGD-AT+OE if it achieves the same accepted-adversarial error at higher clean accuracy, or at a clean rejection rate the practitioner would accept, with the full curve reported.
Key ablation. The score-stability term is applied to the scalar score only, versus the same term applied to the full logit vector. Our claim predicts that the logit version, and only the logit version, reintroduces an accuracy tax.
Why not just…?
Why not just run stronger attacks on AOE and GOOD and report the drop? That is necessary and forms our first step. It would only tell us whether the disguise goal holds under multi-step attacks. It would not explain why the defense is free, and it would not reveal that the joint goal is left completely open. The joint attack and the stability lemma do that work. If AOE turns out to already resist the joint attack, the lemma predicts this can only be because it rejects many clean fragile inputs, and our coverage measurement would show exactly that.
Why not just adversarially train the classifier and add outlier exposure? This is the most serious alternative, and it may well be the best point on the frontier for the joint goal. OODRobustBench gives reason to expect a steep accuracy cost and poor transfer under shift, but we treat that as a hypothesis, not a conclusion. We include it as a full frontier curve. If it dominates the coverage-based defense everywhere, we will report that the accuracy route is the right currency. The first two claims do not depend on that outcome.
Why not just use a post-hoc detector on a frozen model, which costs no accuracy by construction? Accuracy is preserved, but Fort shows these scores fall to 41.33% AUROC at $\epsilon = 1/255$, and the best post-hoc option, an ensemble, stops at 78.64%. Our lemma also implies that no frozen detector can defend the joint goal unless the frozen classifier is already robust. Running ActSub, P-OCS and ensembled Relative Mahalanobis through all three attacks tests both points directly.
Proposed Experiments and Decisive Checks
Data and metrics. CIFAR-10 and CIFAR-100 are the in-distribution sets, evaluated on the near-OOD and far-OOD splits of OpenOOD v1.5: CIFAR-100/CIFAR-10 in both directions, TinyImageNet, SVHN, Textures and Places365. Perturbation budgets are 1/255, 2/255 and 8/255 under the $\ell_\infty$ norm. For each defense we report:
- clean accuracy;
- clean and attacked AUROC and FPR95 for the disguise and denial-of-service goals;
- for the joint goal, the accepted-adversarial error rate at 95% clean acceptance and the full accuracy-versus-rejection curve.
Scale check. A reduced ImageNet-1K run with SSB-hard and NINCO tests whether the frontier's shape holds at scale. It also tests whether higher-resolution ViT inputs widen the gap, as Fort suggests.
Decisive outcomes. The proposal is confirmed if three things hold:
- Score-only stability training defends denial of service within half a point of clean accuracy, while the logit-level version does not.
- Every detector that is robust against denial of service leaves joint-attack error near the classifier's own robust error on the inputs it accepts, as the lemma predicts.
- The coverage route reaches some joint-attack error levels at a lower combined cost than PGD-AT+OE.
The claim is refuted if a defense that perturbs only outliers achieves low joint-attack error without raising the clean rejection rate. That would mean the free lunch extends further than the lemma allows, and it would point to a flaw in how we attacked it, which we would then audit.
References
- Adversarial vulnerability of powerful near out-of-distribution detection
- Certifiably Adversarially Robust Detection of Out-of-Distribution Data
- Out-of-Distribution Detection with Adversarial Outlier Exposure
- OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift
- DeepCloak: Masking Deep Neural Network Models for Robustness Against Adversarial Samples
- Activation Subspaces for Out-of-Distribution Detection
- Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection
- OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection