On the Blindspots of Convolutional Networks

Elad Hoffer, Shai Fine, Daniel SoudryarXiv 2018 · arXiv

Abstract

Convolutional Neural Networks (ConvNets) are the standard for computer vision, yet their architectural priors—locality and stationarity—may impose significant limitations. This study demonstrates that ConvNets possess inherent "blindspots," remaining oblivious to simple, perfectly predictive signals that traditional, "weaker" models like Decision Trees or Shallow Perceptrons can easily exploit. We show that while a ResNet-56 maintains baseline performance, simpler models achieve 100% accuracy when presented with these injected signals.

Introduction

We often view ConvNets as universal function approximators that eventually converge on the most efficient solution given sufficient depth and data. However, ConvNets are not blank slates; they are built on specific inductive biases, primarily locality (pixels near each other are related) and stationarity (features like "eyes" look the same regardless of location). This research investigates whether these strengths are also fundamental weaknesses, identifying types of information that remain "invisible" to state-of-the-art deep learning architectures while being easily recognizable by simpler models.

Methods

The study employs a "competitive analysis" framework. Standard datasets, such as CIFAR-10 for vision and Yelp/Amazon reviews for text, are modified by injecting "truth-revealing signals"—subtle, perfectly predictive modifications that encode the class label. If a model can "see" the signal, it should achieve near-perfect accuracy.

Example of a bird image from the CIFAR-10 dataset used in the experiments.

The signals tested include:

  • Low-dimensional signals: Encoding labels in a single pixel or a fixed pattern of pixels.
  • Global signals: Adding class-specific orthogonal noise vectors or encoding labels in the image's mean value.
  • Moving targets: Encoding labels in a pixel whose location varies randomly per image.

A ResNet-56 was compared against traditional models including Decision Trees, Random Forests, Naive Bayes, and Shallow Neural Networks.

Results and Analysis

The quantitative results reveal a massive gap between the "blind" ConvNet and "seeing" traditional models. While the ResNet-56 remained at its baseline accuracy of 84.9% across all injected signal types, simpler models reached near-perfection.

Truth Signal (CIFAR-10) ConvNet Accuracy Best Traditional Model Accuracy
Clean Data (Baseline) 84.9% - -
One Pixel 84.9% Decision Tree 100%
Pattern Pixels 84.9% Decision Tree 100%
Noise Encoding 84.9% Perceptron 100%
Mean Encoding 84.9% Shallow NN 100%
Multiple Locations 84.9% QDA 98.4%
Example of a truck image from the CIFAR-10 dataset, illustrating the natural image domain.

The Smoothing Trap and Locality Bias

The failure of ConvNets to utilize these signals stems from their core operations. Convolutions and pooling aggregate information, which effectively "washes out" low-dimensional data. A single "magic pixel" is treated as noise and averaged away by the network's layers. Furthermore, the focus on local spatial patches prevents the capture of global statistical properties, such as the mean of all pixels or distributed noise signals, unless the architecture is explicitly modified to account for them.

The Paradox of Complexity

In a striking reversal of the typical performance hierarchy, a 1-layer Perceptron outperformed a 56-layer ResNet. Simpler models, having fewer architectural priors about data structure, are free to latch onto mathematical "cheat codes." Conversely, the ResNet's invariance to shifts and scales—usually a benefit—becomes a hindrance when the critical information is encoded within those very properties.

Conclusion

While the signals used in this study are artificial, they represent extreme cases of real-world phenomena like sensor noise or global artifacts from photography equipment. This work highlights the need for architectural awareness; we must ensure that the inductive biases we bake into deep learning models do not inadvertently discard valuable information. Future work should explore architectural modifications, such as global skip connections or hybrid models, to mitigate these blindspots without losing the benefits of convolutions.

Summary updated

Abstract

Deep convolutional network has been the state-of-the-art approach for a wide variety of tasks over the last few years. Its successes have, in many cases, turned it into the default model in quite a few domains. In this work, we will demonstrate that convolutional networks have limitations that may, in some cases, hinder it from learning properties of the data, which are easily recognizable by traditional, less demanding, models. To this end, we present a series of competitive analysis studies on image recognition and text analysis tasks, for which convolutional networks are known to provide state-of-the-art results. In our studies, we inject a truth-revealing signal, indiscernible for the network, thus hitting time and again the network's blind spots. The signal does not impair the network's existing performances, but it does provide an opportunity for a significant performance boost by models that can capture it. The various forms of the carefully designed signals shed a light on the strengths and weaknesses of convolutional network, which may provide insights for both theoreticians that study the power of deep architectures, and for practitioners that consider applying convolutional networks to the task at hand.
BibTeX
@misc{hoffer2018blindspots,
  author = {Elad Hoffer and Shai Fine and Daniel Soudry},
  title = {{On the Blindspots of Convolutional Networks}},
  year = {2018},
  eprint = {1802.05187},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1802.05187},
}
Figure