Training on Foveated Images Improves Robustness to Adversarial Attacks

Muhammad A. Shah, Bhiksha Raj

Abstract

Deep neural networks (DNNs) have been shown to be vulnerable to adversarial attacks—subtle, perceptually indistinguishable perturbations of inputs that change the response of the model. In the context of vision, we hypothesize that an important contributor to the robustness of human visual perception is constant exposure to low-fidelity visual stimuli in our peripheral vision. To investigate this hypothesis, we develop R-Blur, an image transform that simulates the loss in fidelity of peripheral vision by blurring the image and reducing its color saturation based on the distance from a given fixation point. We show that compared to DNNs trained on the original images, DNNs trained on images transformed by R-Blur are substantially more robust to adversarial attacks, as well as other, non-adversarial, corruptions, achieving up to 25% higher accuracy on perturbed data.

Introduction

While Deep Neural Networks (DNNs) currently define the state-of-the-art in computer vision, they suffer from a fundamental "brittleness." Adversarial attacks—tiny, strategically placed pixel perturbations—can trick a model into misclassifying an image with high confidence, even when the changes are invisible to humans. This vulnerability highlights a significant gap between artificial and biological vision.

Researchers at Carnegie Mellon University observe that while a DNN typically processes every pixel of a high-resolution input with equal fidelity, humans only see a tiny fraction (less than 1%) of their visual field in high definition. Our peripheral vision is a low-fidelity stream of blurred, desaturated information. This research investigates whether this biological "limitation" is actually a feature that helps the human brain ignore high-frequency noise and focus on robust, high-level structures.

The R-Blur Methodology

To test this hypothesis, the authors introduced R-Blur (Retina Blur), an image transformation pipeline designed to mimic the human eye's foveation. The process involves five key steps:

  1. Fixation Point Selection: A "fixation point" is chosen. During training, this is done randomly; during inference, a gaze-prediction model (DeepGaze-III) is used to simulate where a human might look.
  2. Stochastic Noise: Gaussian noise is added to simulate the firing rates of biological photoreceptors.
  3. Adaptive Blurring: The image is blurred using a Gaussian kernel with a standard deviation that increases exponentially with the distance from the fixation point.
  4. Color Desaturation: To mimic the distribution of rods and cones in the human retina, the model blends the color image with a grayscale version based on eccentricity.
  5. Quantization: To maintain computational efficiency, blur levels are quantized into discrete "bins," avoiding the overhead of calculating unique blurs for every pixel.

Experimental Setup

The study evaluated R-Blur across three primary datasets to ensure scalability: CIFAR-10, Ecoset (1.4M images organized by basic object classes), and ImageNet (ILSVRC 2012). The models were tested against Auto-PGD, a rigorous white-box adversarial attack, and compared against standard ResNets and other biologically inspired defenses like VOneBlock and R-Warp.

Key Insights and Results

1. Superior Adversarial Robustness

On ImageNet and Ecoset, standard models typically drop to near 0% accuracy under moderate adversarial attacks. In contrast, R-Blur models maintained 18-22% accuracy under the same conditions. R-Blur outperformed the R-Warp baseline by up to 33 percentage points and VOneBlock by 15 percentage points.

2. Broad-Spectrum Generalization

Unlike standard Adversarial Training (AT), which often overfits to specific attack types, R-Blur provides general robustness. It performs exceptionally well against both malicious attacks and natural "Common Corruptions" (e.g., snow, fog, or brightness changes). In tests against 17 types of non-adversarial corruptions, R-Blur was the most accurate model, even outperforming models specifically optimized via AT.

3. Texture vs. Structure

By blurring the periphery, the network is forced to rely on global structures rather than high-frequency textures. Because adversarial attacks often exploit these fine-grained textures, R-Blur effectively filters out the signal used by the attacker, forcing the model to learn more stable, shape-based representations.

4. The Power of "Looking Around"

The researchers found that evaluating an image using a "scanpath"—a sequence of five fixation points—significantly improved performance over a single static look. This mimics human saccades, where the brain integrates information from multiple glimpses to understand a scene. Furthermore, the authors provided "Certifiable Robustness" proofs, confirming that these gains are mathematically sound and not the result of gradient obfuscation.

Limitations and Future Directions

Despite the gains in robustness, there is a "clean data tax." R-Blur models show a slight drop in performance on unperturbed images compared to standard ResNets (e.g., ~60% vs ~70% on ImageNet). Additionally, the model's success is highly dependent on the fixation point; if the gaze-prediction model misses the primary object, the blur may obscure critical features.

Future work aims to develop "active vision" algorithms, where the model intelligently chooses subsequent fixation points based on initial visual data. Preliminary tests using an "oracle" to select optimal fixation points suggest that clean accuracy could be restored to within 2% of standard models, potentially bridging the gap between robustness and high-fidelity performance.

Summary updated

Abstract

Deep neural networks (DNNs) have been shown to be vulnerable to adversarial attacks -- subtle, perceptually indistinguishable perturbations of inputs that change the response of the model. In the context of vision, we hypothesize that an important contributor to the robustness of human visual perception is constant exposure to low-fidelity visual stimuli in our peripheral vision. To investigate this hypothesis, we develop \RBlur, an image transform that simulates the loss in fidelity of peripheral vision by blurring the image and reducing its color saturation based on the distance from a given fixation point. We show that compared to DNNs trained on the original images, DNNs trained on images transformed by \RBlur are substantially more robust to adversarial attacks, as well as other, non-adversarial, corruptions, achieving up to 25\% higher accuracy on perturbed data.
BibTeX
@inproceedings{shah2023training,
  author = {Muhammad A. Shah and Bhiksha Raj},
  title = {{Training on Foveated Images Improves Robustness to Adversarial Attacks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=2tfG9QaFA7},
}
Figure