Adversarial training is often touted as a way to make deep neural networks (DNNs) more "human-like." By forcing models to be invariant to small, pixel-level perturbations, we hope they learn the robust, high-level features that humans use. However, a new paper titled "Adversarial robustness does not entail human-aligned categorization" suggests that this hope might be misplaced.
TL;DR
Researchers used a generative model to create "controversial stimuli"—images that trigger high-confidence, conflicting labels from different robust models. They found that while humans consistently agree on what these images represent, the models themselves perform at near-chance levels in matching human judgments, proving that adversarial robustness and human alignment are distinct properties.
What Problem Does This Paper Address?
While adversarially robust models are less "fragile" than standard DNNs, it remains unclear if their internal decision boundaries actually align with human perception. Previous research suggested that robust models might share some sensitivities with biological vision, but these studies often looked at internal representations rather than final behavioral outcomes.
The authors argue that standard benchmarks (like ImageNet accuracy) are insufficient to test alignment because they only evaluate models on "natural" images. To truly test a model's "logic," you need to find the edges of its decision boundaries. This paper investigates whether the categorization boundaries of state-of-the-art robust models match those of human observers.
Methods
The researchers developed a novel framework to synthesize controversial stimuli:
- Generative Prior: Instead of optimizing pixels directly (which often results in unrecognizable noise), they optimized the latent space of a pre-trained BigGAN. This ensures the resulting images have "object-like" structures that humans can actually categorize.
- Black-Box Optimization: They used CMA-ES (Covariance Matrix Adaptation Evolution Strategy), a gradient-free optimizer. This allowed them to attack any model (including Transformers and CNNs) without needing access to internal gradients, bypassing issues like gradient obfuscation.
- The Objective: For a pair of models (e.g., ResNet-101 and ViT-B), they searched for a latent vector that caused Model A to say "Lion" with high confidence and Model B to say "Dog" with high confidence.
The study evaluated four primary models—ResNet-101, ViT-B, ConvNeXt-L, and Swin-L—and later tested 17 additional models from the RobustBench leaderboard. Finally, they collected over 10,000 category judgments from 60 human participants to see which model (if any) the humans agreed with.
Datasets
- Training: All models were trained on ImageNet-1K.
- Evaluation: The authors introduced a new benchmark of 1,008 controversial stimuli spanning eight categories (e.g., golden retriever, race car, military plane).
- Human Data: A dataset of 10,080 human judgments, providing a "noise ceiling" for how much humans agree with each other on these synthetic images.
Key Insights and Ideas
- The "Targeting" Penalty: When a stimulus is optimized to exploit a specific model's boundaries, that model's alignment with humans collapses to chance. This suggests that every robust model has "idiosyncratic" ways of seeing the world that are completely alien to humans.
- Robustness $\neq$ Alignment: Perhaps the most striking finding is that a model's score on adversarial benchmarks (like AutoAttack) does not correlate with its alignment with human categorization. You can build a "more robust" model that is actually less human-aligned.
- CNNs vs. Transformers: When using confidence-sensitive scoring (SRCS), the two convolutional models (ResNet and ConvNeXt) showed a slight, albeit weak, positive correspondence with humans, while the Transformer models did not. This hints that architectural inductive biases might play a role in alignment.
Key Results
The results highlight a massive "alignment gap":
- Model-Human Agreement: When evaluated on stimuli optimized to trick them, models agreed with humans only 48.6%–51.3% of the time (where 50% is chance).
- Human Consensus: On those same images, humans agreed with each other ~86% of the time.
- Non-Targeted Models: Models that weren't involved in the optimization did better (~67.3% agreement) but still lagged significantly behind the human-human agreement of ~81.3% on those same images.
As shown in Figure 3, even when humans are "Certain" about a category, the targeted models remain at chance.
Limitations and Future Work
- Threat Model Scope: The study focused on $\ell_\infty$ robustness. It's possible that other forms of robustness (e.g., $\ell_2$ or common corruptions) might yield different alignment properties.
- Generative Classifiers: The authors suggest that purely discriminative training (mapping images to labels) might be the culprit. Future work could explore whether generative or hybrid classifiers—which model the distribution of the images themselves—show better alignment at the ImageNet scale.
- Architecture vs. Training: Since the study used pre-trained checkpoints, it's difficult to perfectly isolate whether the misalignment stems from the model architecture or the specific adversarial training recipe used.
In conclusion, while adversarial training is a step toward more reliable AI, it is not a silver bullet for human-like vision. Robustness and human alignment are distinct goals, and we need specific benchmarks—like controversial stimuli—to measure our progress toward the latter.
Summary updated