Empirical Advocacy of Bio-inspired Models for Robust Image Recognition

Harshitha Machiraju, Oh-Hyeon Choung, Michael H. Herzog, Pascal FrossardarXiv 2022 · OpenReview

Abstract: Deep convolutional neural networks (DCNNs) have revolutionized computer vision and are often advocated as good models of the human visual system. However, there are currently many shortcomings of DCNNs, which preclude them as a model of human vision. There are continuous attempts to use features of the human visual system to improve the robustness of neural networks to data perturbations. We provide a detailed analysis of such bio-inspired models and their properties. To this end, we benchmark the robustness of several bio-inspired models against their most comparable baseline DCNN models. We find that bio-inspired models tend to be adversarially robust without requiring any special data augmentation. Additionally, we find that bio-inspired models beat adversarially trained models in the presence of more real-world common corruptions. Interestingly, we also find that bio-inspired models tend to use both low and mid-frequency information, in contrast to other DCNN models. We find that this mix of frequency information makes them robust to both adversarial perturbations and common corruptions.

Introduction

Standard Deep Convolutional Neural Networks (DCNNs) are notoriously brittle. While they achieve superhuman performance on clean datasets, they are easily deceived by adversarial attacks—imperceptible pixel perturbations—and struggle with common environmental corruptions like fog, snow, or sensor noise. This research explores whether mimicking the human visual system can bridge the gap between artificial and biological vision, providing inherent robustness without the heavy computational tax of traditional adversarial training.

Methodology

The study benchmarks four bio-inspired architectures against standard DCNN baselines:

  • Retina: Implements eccentricity-dependent sampling, mimicking the high-resolution fovea and blurred periphery of the human eye.
  • CNNF (Convolutional Neural Network with Feedback): Incorporates feedback loops, allowing higher-level features to iteratively modulate and "clean" lower-level representations.
  • MiceV1: A model regularized to align its internal activations with neural recordings from the primary visual cortex (V1) of mice.
  • VOneNet: Replaces initial DCNN layers with a "VOneBlock" containing Gabor filters and stochasticity to simulate neural firing variability.

Models were evaluated using the APGD white-box adversarial attack and the ImageNet-C/CIFAR-10-C benchmarks for common corruptions.

Key Insights: The Frequency "Sweet Spot"

A central finding of this research is the identification of a frequency-based explanation for robustness. Standard DCNNs rely heavily on high-frequency details, which are the most susceptible to corruption. Conversely, adversarially trained (AT) models often over-correct, relying almost exclusively on low-frequency shapes.

Model accuracy as a function of frequency bandwidth, highlighting the reliance of bio-inspired models on mid-frequency information.

Bio-inspired models occupy a unique middle ground, leveraging a mix of low and mid-frequency information. As shown in the frequency analysis, these models show a significant performance surge when mid-frequency data is available, suggesting they are tuned to features that are more stable across different types of noise and perturbations.

Key Results

Robustness Without Augmentation

One of the most compelling arguments for bio-inspired architectures is "robustness for free." Unlike standard models that require expensive adversarial training (AT) to resist attacks, bio-inspired models exhibit inherent resistance due to their architectural inductive biases.

Accuracy of bio-inspired models vs. baselines and adversarially trained (AT) models across different adversarial perturbation strengths (ε).

As illustrated in the adversarial benchmarks, bio-inspired models (e.g., CNNF, Retina) maintain high accuracy under small perturbations (ε = 0.25/255) where baseline models fail. While they may still lag behind heavily adversarially trained models at high perturbation magnitudes (ε = 8/255), they provide a much better balance of clean accuracy and baseline robustness.

Superior Generalization to Corruptions

Bio-inspired models significantly outperform AT models on real-world corruptions, particularly weather-related noise. For instance, on the "Fog" corruption benchmark, the CNNF model achieved ~89.5% accuracy, whereas its adversarially trained counterpart dropped to ~62.3%. The feedback mechanisms in CNNF appear particularly effective at resolving occlusions and atmospheric noise by allowing the model to self-correct internal representations.

Conclusion

Bio-inspired architectures offer a promising path toward more reliable AI. By moving beyond simple feed-forward structures and incorporating biological features like retinal sampling and feedback loops, we can create models that are naturally more robust to the complexities of the real world. Future work will focus on scaling these bio-inspired "front-ends" to larger architectures like Vision Transformers (ViTs) to achieve human-level visual stability.

Summary updated

Abstract

Deep convolutional neural networks (DCNNs) have revolutionized computer vision and are often advocated as good models of the human visual system. However, there are currently many shortcomings of DCNNs, which preclude them as a model of human vision. There are continuous attempts to use features of the human visual system to improve the robustness of neural networks to data perturbations. We provide a detailed analysis of such bio-inspired models and their properties. To this end, we benchmark the robustness of several bio-inspired models against their most comparable baseline DCNN models. We find that bio-inspired models tend to be adversarially robust without requiring any special data augmentation. Additionally, we find that bio-inspired models beat adversarially trained models in the presence of more real-world common corruptions. Interestingly, we also find that bio-inspired models tend to use both low and mid-frequency information, in contrast to other DCNN models. We find that this mix of frequency information makes them robust to both adversarial perturbations and common corruptions.
BibTeX
@misc{machiraju2022empirical,
  author = {Harshitha Machiraju and Oh-Hyeon Choung and Michael H. Herzog and Pascal Frossard},
  title = {{Empirical Advocacy of Bio-inspired Models for Robust Image Recognition}},
  year = {2022},
  eprint = {2205.09037},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.09037},
}
Figure