Natural Jammers: When Non-Robust Features Become Antagonistic

Jiaming Zhang, Fuyao Zhang, Pengjun Xie, Wei Yang Bryan LimarXiv 2026 · arXiv

TL;DR

Deep neural networks are often misled by "Natural Jammers"—non-robust, high-frequency features in clean images that actively contradict semantic labels. By identifying a "Jammed Set" where robust models actually outperform standard ones, the authors demonstrate that these failures arise from a fragile "Spectral-Spatial Locking" mechanism that redistributes rather than disappears under small geometric transformations.

What Problem Does This Paper Address?

It is well-established that standard Deep Neural Networks (DNNs) rely on "non-robust features"—subtle, high-frequency patterns that are imperceptible to humans but predictive for models. Traditionally, researchers have viewed these as spurious shortcuts: brittle crutches that help models achieve high accuracy on clean data but fail under adversarial attack.

This paper challenges the assumption that non-robust features are always helpful. The authors ask: Can these features be antagonistic? Do they ever act as "Natural Jammers" that actively confuse a standard model on perfectly normal, clean images? If so, why do standard models fall for them while robustly trained models (which are usually less accurate on clean data) manage to ignore them and get the answer right?

Methods

The authors introduce a diagnostic framework called the Quad-Set Partition. By comparing a standard model ($f_{std}$) and an adversarially trained robust model ($f_{rob}$) on the same clean images, they categorize data into four sets:

  • Common Set ($\mathcal{C}$): Both models are correct.
  • Spurious Set ($\mathcal{S}$): Only $f_{std}$ is correct (the classic shortcut regime).
  • Jammed Set ($\mathcal{J}$): Only $f_{rob}$ is correct. Here, non-robust features mislead the standard model.
  • Uninformative Set ($\mathcal{G}$): Both models are incorrect.

To investigate why jamming happens, they employed several techniques:

  1. Spectral Filtering: Using Low-Pass Filters (LPF) to progressively remove high-frequency components.
  2. Spatial Transformations: Applying center-crops and zooms to see if the location of features matters.
  3. Frequency Grafting: A "causality test" where high-frequency components from a "jammed" image are transplanted onto the low-frequency "skeleton" of a correctly classified image to see if the "toxicity" transfers.
Figure 1: The Duality of Non-Robust Features. (Left) Spurious Shortcut (S): prediction relies on fragile non-robust cues and collapses when these cues are suppressed. (Right) Natural Jammer (J ): non-robust cues interfere with semantics-aligned evidence, suppressing them partially restores correct predictions. This illustrates that non-robust cues can be antagonistic rather than merely supportive. Accuracies are measured with the primary lens (standard vs. robust ResNet-50, ℓ2, ϵ = 1).

Datasets

The primary evaluation was conducted on the ImageNet validation set ($N = 50,000$). The authors tested their hypotheses across 17 different robust "lenses," including various architectures (ResNet-18, ResNet-50, WideResNet, ViT-B/16, ViT-B/32) and different adversarial training budgets ($\ell_2$ and $\ell_\infty$ norms).

Key Insights and Ideas

  • Feature Antagonism is Real: A significant portion of standard model errors (roughly 4-5% of the entire ImageNet validation set) are not just "hard" images, but images where the model is actively misled by non-robust cues.
  • Spectral-Spatial Locking: Jamming isn't just about high frequencies; it's about their alignment. The "toxicity" of a jammer is not intrinsic to the signal itself. When the authors grafted a jammer's high-frequency signal onto a new image, the "toxicity" failed to transfer 99.83% of the time. The interference only works when the frequency content is "locked" to a specific spatial configuration.
  • The Cleansing Paradox: Removing high-frequency noise destroys accuracy for the Spurious Set (proving they are shortcuts) but restores accuracy for the Jammed Set (proving they were interference).
  • The Substitution Paradox: This is the most surprising finding. While individual jammers are fragile (a tiny 2% crop can "fix" a jammed image), the total number of jammed images remains stable. As one image is fixed by a transformation, another previously "common" image becomes jammed. Non-robustness is a systemic property of the model-data geometry that redistributes rather than disappears.
Spectral and spatial sensitivity curves showing the "Cleansing Paradox" and "Sensitivity Inversion."

Key Results

  • Prevalence: The Jammed Set ($\mathcal{J}$) consistently occupies ~4.3% to 5.0% of ImageNet across various ResNet and WideResNet architectures.
  • Recovery: By applying a low-pass filter (radius $r=50$), the standard model's accuracy on the Jammed Set jumped from 0% to 31.8%.
  • Model Dependence: Jammed identities are highly model-specific. The Intersection-over-Union (IoU) of jammed sets between different architectures (e.g., ResNet vs. ViT) is often less than 0.15, suggesting that different models are "jammed" by different patterns.
  • Confidence: On the Jammed Set, standard models are often more confident in their wrong predictions than robust models are in their correct ones, indicating that the jamming signal is brittle but dominant.

Limitations and Future Work

  • Transformation Scope: The "Substitution Paradox" was primarily tested under mild, semantics-preserving geometric and spectral transformations. It is unclear if this redistribution holds under more aggressive augmentations.
  • Oracle Dependency: The study relies on robust models as a "diagnostic lens." While the authors show results are consistent across many robust models, the findings are inherently tied to the current state-of-the-art in adversarial robustness.
  • Future Directions: The authors suggest that understanding this "failure field" could lead to new types of data augmentation or architectural designs that break the spectral-spatial lock more effectively than current methods.

For more details, visit the project page.

Summary updated

Abstract

Deep neural networks are known to rely on non-robust features that align poorly with human perception. Prior work has largely framed these features as opportunistic shortcuts that help standard models on clean data. We show instead that non-robust cues can be antagonistic: they can act as Natural Jammers that actively mislead standard models on clean, unmodified images. Using disagreement between a standard model and robust models as an analysis lens, we identify a Jammed Set on which robust models outperform standard models. We provide causal evidence for Spectral-spatial Locking: jamming arises only when high-frequency components are precisely aligned with specific spatial structures. Disrupting this lock through spectral filtering or small geometric shifts can restore correct predictions. Crucially, we uncover a Substitution Paradox: while individual jammers are fragile to geometric shifts, the aggregate prevalence of jamming remains approximately stable and is redistributed rather than eliminated under such transformations. Our results reframe natural non-robustness as a failure field over the empirical model-data geometry: one that redistributes rather than disappears under small transformations.
BibTeX
@misc{zhang2026natural,
  author = {Jiaming Zhang and Fuyao Zhang and Pengjun Xie and Wei Yang Bryan Lim},
  title = {{Natural Jammers: When Non-Robust Features Become Antagonistic}},
  year = {2026},
  eprint = {2610.34785},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34785},
}
Figure