A classifier can be attacked without changing its answer. An adversary adds a small perturbation that pushes an input parallel to the decision boundary rather than across it, inflating the model's confidence in a wrong prediction or suppressing the warning signals it would otherwise emit. This matters because most deployed safety mechanisms — abstention when unsure, human handoff, confidence gates — rely on those signals. Galil et al. showed such attacks triple-to-sextuple the risk of selective classifiers on ImageNet Disrupting Deep Uncertainty Estimation Without Harming Accuracy, and Ledda et al. collapsed deep-ensemble variance to $10^{-6}$ on CIFAR-10/100 with tiny perturbations Adversarial Attacks Against Uncertainty Quantification. One recent result offers hope: standard adversarial training (AT), where a model trains on worst-case perturbed inputs, was proven to raise baseline predictive entropy and thereby shrink the range of confidence an attacker can induce On the robustness of adversarial training against uncertainty attacks.
The core claim of this proposal is that this hope is likely misplaced in a specific, testable way. Adversarial training is famous for overfitting to the exact attack it was trained on — an $L_\infty$-trained model suffers 99.4% error against unseen $L_2$ attacks on SVHN Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks. We hypothesize that AT's protection against confidence attacks is itself an instance of this adversarial generalization gap: the elevated-entropy "shield" holds only inside the training norm's radius, under white-box access, and collapses under (i) unseen perturbation geometries and (ii) confidence attacks transferred zero-query from ordinary, non-robust surrogate models. We call this failure the uncertainty generalization gap, we propose to measure it for the first time, explain it through the geometry of how models pack features into limited dimensions, and close it with a training procedure that makes the entropy shield geometry-independent.
Core Claim
No prior work has evaluated the uncertainty robustness of adversarially trained models outside the white-box, seen-geometry setting. The paper that established AT's intrinsic shield On the robustness of adversarial training against uncertainty attacks tested only direct attacks under the training threat model; its authors explicitly leave cross-model transfer and unseen norms open. CCAT closes the generalization gap for label-flipping attacks by training confidence to decay smoothly as perturbations grow Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks, but never faced it against attacks whose objective is the confidence readout itself. Galil et al. showed confidence attacks transfer across CNN architectures Disrupting Deep Uncertainty Estimation Without Harming Accuracy, and the VLM frontier study showed an answer-preserving confidence attack flips 84.8% of previously rejected wrong answers through a confidence gate Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier, but neither evaluated hardened targets. Our proposal sits precisely in the unoccupied corner: the generalization behavior of uncertainty defenses, attacked white-box under unseen geometries and black-box via transfer, with a mechanistic account and a fix.
Research Program
The program has three parts: measure the gap, explain it, and close it.
Measuring the gap. We take RobustBench adversarially trained models on CIFAR-10, CIFAR-100, and ImageNet, plus their standard-trained counterparts, and attack them with ACE-style margin attacks Disrupting Deep Uncertainty Estimation Without Harming Accuracy and STAB-style entropy-suppression attacks Adversarial Attacks Against Uncertainty Quantification under four geometries: the training-time $L_\infty$ ball, and unseen $L_2$, $L_1$, and frame-style perturbations (following the unseen-attack suite of Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks). The primary metrics are the Uncertainty Span (US) — the range of entropy values an attacker can force, as defined in On the robustness of adversarial training against uncertainty attacks — plus inflation of the Area Under the Risk-Coverage curve (AURC, the error rate of a selective classifier averaged over abstention thresholds) and signed expected calibration error. The gap is quantified as the ratio of attacked AURC between the seen geometry and each unseen geometry.
Transfer, including a label-free attack. Galil's attack needs ground-truth labels to decide whether to inflate or deflate confidence — a limitation they flag. We construct a label-free variant: the surrogate ensemble's majority prediction serves as the pseudo-label, and the attack inflates confidence in that prediction regardless of its correctness, which is sufficient to break selective classification when the surrogate is wrong. We generate these attacks on non-robust surrogates (ResNet-50 ensembles, mirroring Galil's setup, and self-supervised backbones of the kind whose shared pretraining drives transferability in speech Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models) and fire them zero-query at AT targets. A secondary finding we will verify: output-agreement defenses like MVP-EARS, which detects audio attacks at 99.88% by comparing transcriptions across ASRs A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples, are structurally blind to answer-preserving confidence attacks — a cheap but decisive demonstration.
Mechanistic explanation. Stevinson et al. showed that when models compress more features than latent dimensions ($k > m$), the resulting non-orthogonal superposition creates interference channels that attackers exploit, with shared data correlations pushing cross-model transfer from 5% to 98% Adversarial Vulnerability from Interference Between Features in Superposition. We measure superposition metrics (effective rank, interference between class feature vectors) in AT versus standard models and test whether AT's entropy shield correlates with reduced interference — or whether the shield is a shallow local effect that interference geometry predicts will not transfer. This also lets us adjudicate a direct contradiction in the file: Stevinson et al. find weight decay increases vulnerability by forcing low-rank superposition, while Predicting Adversarial Examples with High Confidence claims weight decay and low-precision representations improve robustness. Our superposition-versus-manipulability measurements on matched models should settle which effect dominates.
Closing the gap. We propose Uncertainty-Span Adversarial Training (US-AT), which unifies CCAT's smooth-confidence idea with uncertainty-attack objectives across geometries. During training, the inner adversary alternates between a label-flipping loss and an entropy-suppression objective (pushing the model toward near-one-hot confidence on its current prediction), across the geometry set:
$$\min_\theta \; \mathbb{E}_{(x,y)}\left[ \max_{g \in \mathcal{G},\, \|\delta\|_g \le \epsilon_g} \; \alpha\, \mathcal{L}_{\text{label}}(f_\theta(x+\delta), y) + (1-\alpha)\, \mathcal{L}_{\text{conf}}(f_\theta(x+\delta)) \right]$$
where $\mathcal{G} = \{L_\infty, L_2, L_1, \text{frame}\}$, $\alpha$ alternates between batches, and $\mathcal{L}_{\text{conf}}$ rewards the model for resisting the adversary's entropy suppression — the model is trained so that no geometry in $\mathcal{G}$ can simultaneously force high confidence and wrongness. The key ingredient, isolated by ablation below, is the pairing of multi-geometry perturbations with uncertainty-targeted objectives; multi-norm label-only AT and uncertainty objectives alone are both ablated away.
Proposed Experiments
Experiments run on CIFAR-10, CIFAR-100, SVHN, and an ImageNet subset, all present in the source literature, using the attack implementations of ACE and STAB and the unseen-geometry suite from CCAT. Evaluation is a full grid: two attack families times four geometries, white-box and transferred, against standard models, RobustBench AT models, CCAT (retrained from the released procedure), deep ensembles, BEYOND, and US-AT. Success is judged on AURC inflation, selective risk at a fixed coverage, US, and transfer success (fraction of surrogate-crafted confidence perturbations that inflate target AURC by more than 2x). A small audio extension on LibriSpeech tests whether the label-free confidence attack transfers across the twelve SSL ASR models studied by Olivier et al. Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models, establishing whether the uncertainty generalization gap is modality-general.
Baselines
The "do nothing" reference point is an undefended standard-trained model under ACE. Galil et al. report that ACE increases AURC by 3x to 6x across MobileNetV2, EfficientNetB0, ResNet50, DenseNet161, and VGG16 on ImageNet, and raises SelectiveNet's selective risk on CIFAR-10 from 0.67% to over 8% Disrupting Deep Uncertainty Estimation Without Harming Accuracy. Beating this reference means keeping attacked AURC within 1.5x of clean AURC at matched coverage. The strongest existing competitor on our exact question is standard adversarial training: it is the only method with demonstrated resistance to uncertainty attacks On the robustness of adversarial training against uncertainty attacks, though that paper reports only qualitative US comparisons ("consistently exhibited smaller Uncertainty Spans") — no reported result found on Lacuna for a numeric US value on RobustBench models, so we reproduce the measurement ourselves. We run AT with the same architecture, data, and training budget as US-AT, tuning each method with an equal search over hyperparameters, and US-AT beats it only if it maintains the AT-level uncertainty resistance outside the training geometry where AT's resistance collapses. CCAT is the strongest competitor on the generalization axis: Stutz et al. report 29.0% error against unseen $L_2$ attacks on SVHN where standard AT suffers 99.4%, and 0.2% against adversarial frames on MNIST where standard AT suffers 73.7% Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks — but those are label attacks; CCAT has never faced ACE or STAB, and we test whether its confidence gate survives direct entropy suppression at equal compute. Finally, the ablation isolating our key ingredient is US-AT trained with label-only losses (no $\mathcal{L}_{\text{conf}}$) and US-AT trained on a single geometry: if the full version does not beat both under the unseen-geometry transfer suite, the multi-geometry uncertainty objective is not earning its cost.
Why not just…?
Why not just calibrate post hoc with temperature scaling? Temperature scaling applies one fixed scalar to all logits, calibrated on a held-out set. It helps on clean data How to measure deep uncertainty estimation performance and which models are naturally better at providing it, but an input-adaptive attack moves each sample's logit margins individually, which a constant rescaling cannot counteract — exactly the failure Galil and Ledda document for softmax-targeted optimization Disrupting Deep Uncertainty Estimation Without Harming Accuracy, and the calibration set itself can be poisoned Data Poisoning Attacks against Conformal Prediction. We will report temperature-scaled versions of every model in the grid to make the failure quantitative rather than rhetorical.
Why not just use standard adversarial training or deep ensembles? This is the most serious alternative, because standard AT is both the strongest baseline and the simpler defense. The evidence cuts both ways: Galil et al. found label-focused AT gives no significant protection against ACE Disrupting Deep Uncertainty Estimation Without Harming Accuracy, yet the RobustBench study found AT models do resist white-box uncertainty attacks On the robustness of adversarial training against uncertainty attacks. Our experiment 1 is designed to settle this honestly: if AT's shield survives unseen geometries and transfer, our premise is wrong and the paper becomes a strong negative result about the shield's generality, which is still publishable and still valuable. Ensembles are not a real threat: Ledda et al. drove ensemble variance to $10^{-6}$ under STAB on CIFAR-10/100 Adversarial Attacks Against Uncertainty Quantification, and ensemble proxies can even strengthen transfer Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models.
Why not just deploy a detector like BEYOND or multi-version voting? BEYOND detects label-flipping attacks at over 99% AUC on CIFAR-10 by checking whether 50 augmentations of an input stay consistent in a self-supervised embedding space BEYOND: Detecting Adversarial Examples by the Neighborhood Relations Built on Self-Supervised Learning, and MVP-EARS detects audio attacks at 99.88% by cross-model transcription disagreement A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples. Both are untested against boundary-parallel attacks: an ACE-style perturbation can leave neighborhood consistency and output strings intact while subverting only the confidence readout, and the MVP-EARS authors themselves concede attacks that preserve outputs defeat transcription voting. We will run BEYOND against our ACE suite as a baseline — if it does catch confidence manipulation, that is a genuine threat to the defense half of the proposal, and the study half (the gap and its geometric explanation) stands regardless.
Decisive Checks
Three outcomes would make or break the proposal. First, if attacked AURC on AT models degrades sharply between seen and unseen geometries (say, the inflation ratio more than doubles), the uncertainty generalization gap exists and is measurable. Second, if the label-free transferred attack inflates target AURC by 2x or more on AT models while MVP-style output voting stays at chance, the transfer threat is real and invisible to output-agreement defenses. Third, and most demanding: US-AT must keep selective risk under the full unseen-geometry, transferred attack suite within 1.5x of its clean value, at a clean-error cost no worse than CCAT's (which achieved 10.1% clean error versus 16.6% for standard AT on CIFAR-10 Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks). If the first two hold and the third fails, the contribution is the first systematic characterization and mechanistic explanation of the uncertainty generalization gap — a gap the field currently does not know it has.
References
- Disrupting Deep Uncertainty Estimation Without Harming Accuracy
- Adversarial Attacks Against Uncertainty Quantification
- On the robustness of adversarial training against uncertainty attacks
- Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks
- Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier
- Watch What You Pretrain For: Targeted, Transferable Adversarial Examples on Self-Supervised Speech Recognition models
- A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples
- Adversarial Vulnerability from Interference Between Features in Superposition
- Predicting Adversarial Examples with High Confidence
- How to measure deep uncertainty estimation performance and which models are naturally better at providing it
- Data Poisoning Attacks against Conformal Prediction
- Be Your Own Neighborhood: Detecting Adversarial Examples by the Neighborhood Relations Built on Self-Supervised Learning