Adversarial robustness does not entail human-aligned categorization

Itay Reznik, Raz Lapid, Moshe Sipper, Tal GolanarXiv 2026 · arXiv

Adversarial training is often touted as a way to make deep neural networks (DNNs) more "human-like." By forcing models to be invariant to small, pixel-level perturbations, we hope they learn the robust, high-level features that humans use. However, a new paper titled "Adversarial robustness does not entail human-aligned categorization" suggests that this hope might be misplaced.

TL;DR

Researchers used a generative model to create "controversial stimuli"—images that trigger high-confidence, conflicting labels from different robust models. They found that while humans consistently agree on what these images represent, the models themselves perform at near-chance levels in matching human judgments, proving that adversarial robustness and human alignment are distinct properties.

What Problem Does This Paper Address?

While adversarially robust models are less "fragile" than standard DNNs, it remains unclear if their internal decision boundaries actually align with human perception. Previous research suggested that robust models might share some sensitivities with biological vision, but these studies often looked at internal representations rather than final behavioral outcomes.

The authors argue that standard benchmarks (like ImageNet accuracy) are insufficient to test alignment because they only evaluate models on "natural" images. To truly test a model's "logic," you need to find the edges of its decision boundaries. This paper investigates whether the categorization boundaries of state-of-the-art robust models match those of human observers.

Methods

The researchers developed a novel framework to synthesize controversial stimuli:

  1. Generative Prior: Instead of optimizing pixels directly (which often results in unrecognizable noise), they optimized the latent space of a pre-trained BigGAN. This ensures the resulting images have "object-like" structures that humans can actually categorize.
  2. Black-Box Optimization: They used CMA-ES (Covariance Matrix Adaptation Evolution Strategy), a gradient-free optimizer. This allowed them to attack any model (including Transformers and CNNs) without needing access to internal gradients, bypassing issues like gradient obfuscation.
  3. The Objective: For a pair of models (e.g., ResNet-101 and ViT-B), they searched for a latent vector that caused Model A to say "Lion" with high confidence and Model B to say "Dog" with high confidence.
The controversial-stimulus generation pipeline. Latent vectors are fed into a GAN, and the resulting image is evaluated by two classifiers. CMA-ES iteratively updates the vectors to maximize disagreement.

The study evaluated four primary models—ResNet-101, ViT-B, ConvNeXt-L, and Swin-L—and later tested 17 additional models from the RobustBench leaderboard. Finally, they collected over 10,000 category judgments from 60 human participants to see which model (if any) the humans agreed with.

Datasets

  • Training: All models were trained on ImageNet-1K.
  • Evaluation: The authors introduced a new benchmark of 1,008 controversial stimuli spanning eight categories (e.g., golden retriever, race car, military plane).
  • Human Data: A dataset of 10,080 human judgments, providing a "noise ceiling" for how much humans agree with each other on these synthetic images.

Key Insights and Ideas

  • The "Targeting" Penalty: When a stimulus is optimized to exploit a specific model's boundaries, that model's alignment with humans collapses to chance. This suggests that every robust model has "idiosyncratic" ways of seeing the world that are completely alien to humans.
  • Robustness $\neq$ Alignment: Perhaps the most striking finding is that a model's score on adversarial benchmarks (like AutoAttack) does not correlate with its alignment with human categorization. You can build a "more robust" model that is actually less human-aligned.
  • CNNs vs. Transformers: When using confidence-sensitive scoring (SRCS), the two convolutional models (ResNet and ConvNeXt) showed a slight, albeit weak, positive correspondence with humans, while the Transformer models did not. This hints that architectural inductive biases might play a role in alignment.

Key Results

The results highlight a massive "alignment gap":

  • Model-Human Agreement: When evaluated on stimuli optimized to trick them, models agreed with humans only 48.6%–51.3% of the time (where 50% is chance).
  • Human Consensus: On those same images, humans agreed with each other ~86% of the time.
  • Non-Targeted Models: Models that weren't involved in the optimization did better (~67.3% agreement) but still lagged significantly behind the human-human agreement of ~81.3% on those same images.
Summary of the alignment gap. While humans agree with each other (~86%), models targeted by the optimization perform at chance level (~50%) when compared to human labels.

As shown in Figure 3, even when humans are "Certain" about a category, the targeted models remain at chance.

Model-human alignment for the four primary models. Mean binary agreement (A) is near the 0.5 chance line (dashed), far below the human noise ceiling (gray bars).

Limitations and Future Work

  • Threat Model Scope: The study focused on $\ell_\infty$ robustness. It's possible that other forms of robustness (e.g., $\ell_2$ or common corruptions) might yield different alignment properties.
  • Generative Classifiers: The authors suggest that purely discriminative training (mapping images to labels) might be the culprit. Future work could explore whether generative or hybrid classifiers—which model the distribution of the images themselves—show better alignment at the ImageNet scale.
  • Architecture vs. Training: Since the study used pre-trained checkpoints, it's difficult to perfectly isolate whether the misalignment stems from the model architecture or the specific adversarial training recipe used.

In conclusion, while adversarial training is a step toward more reliable AI, it is not a silver bullet for human-like vision. Robustness and human alignment are distinct goals, and we need specific benchmarks—like controversial stimuli—to measure our progress toward the latter.

Summary updated

Abstract

Adversarial training can improve some forms of correspondence between artificial and biological vision, but whether robust classifiers acquire human-aligned classification behavior remains unclear. We generated 1,008 controversial stimuli by optimizing latent inputs to a pretrained GAN to elicit high-confidence disagreements between pairs of robust ImageNet classifiers. We compared four adversarially robust ImageNet classifiers - ResNet-101, ViT-B, ConvNeXt-L, and Swin-L - with two-alternative category judgments from 60 participants. We first evaluated each classifier on stimuli optimized to elicit disagreement with another model. Across classifiers, accuracy in predicting individual participants' single-trial category choices was near chance (48.6-51.3%), whereas majority judgments from other participants predicted responses with 85.0-86.8% accuracy. An analysis incorporating model prediction strength and participants' reported confidence found weak positive model-human correspondence for the two convolutional models. On stimuli optimized for disagreement between other model pairs, all four robust classifiers predicted human responses above chance but less accurately than predictions based on other participants' majority judgments; the same pattern held for 17 additional RobustBench models not used in stimulus generation. Across these 17 models, mean model-participant agreement was 67.3%, versus 81.3% mean participant-participant agreement on the same images. These results identify model-specific failures of human alignment and a model-human mismatch that persists across robust classifiers and substantially exceeds disagreement among human observers. Our findings show that adversarial robustness alone does not guarantee human-aligned categorization and suggest that robustness and human alignment should be evaluated as distinct properties of visual recognition systems.
BibTeX
@misc{reznik2026adversarial,
  author = {Itay Reznik and Raz Lapid and Moshe Sipper and Tal Golan},
  title = {{Adversarial robustness does not entail human-aligned categorization}},
  year = {2026},
  eprint = {2611.00715},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00715},
}
Figure