Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?

Anh Totti Nguyen, Ella M. Gale, Nicholas Martin, Ryan Blything, Anh Nguyen 0002, Jeffrey S. BowersarXiv 2020 · arXiv

TL;DR

This paper challenges the popular notion that deep convolutional neural networks (CNNs) develop "object detector" units in their hidden layers. By comparing various metrics, the authors demonstrate that common measures of selectivity are misleading; units previously labeled as detectors actually suffer from extremely high false-alarm rates or poor hit-rates when evaluated using standard signal-detection theory.

What Problem Does This Paper Address?

In the "standard model" of computer vision, it is often assumed that CNNs learn a hierarchy of features, culminating in hidden units that act as "object detectors"—neurons that fire specifically and exclusively when a certain object (like a dog or a lamp) is present. Previous research has used metrics like precision or Class-Conditional Mean Activity Selectivity (CCMAS) to claim these units exist.

However, different researchers have reached divergent conclusions about how selective these representations really are. The authors of this paper argue that the metrics used to define "object detectors" are flawed. They set out to determine if these units are truly selective or if we are simply seeing "mirages" caused by biased measurement techniques.

Methods and Datasets

The researchers performed a rigorous comparison of several selectivity measures on units within AlexNet, VGG-16, and GoogLeNet. The study utilized two primary large-scale datasets: ImageNet (ILSVRC 2012) and Places-365.

AlexNet architecture used in the study, showing the progression from convolutional layers to fully connected layers.

To rigorously test selectivity, the authors compared traditional metrics against Signal-Detection Theory (SDT) measures, including hit-rates (recall), false-alarm rates, and "informedness." They also employed Activation Maximization (AM) via Plug-and-Play Generative Networks (PPGNs) to visualize what maximally triggers units and used Jitterplots to examine the entire distribution of activations across datasets.

Key Insights: Deconstructing the Mirage

The study identifies several reasons why previous metrics overestimated unit selectivity:

  1. The Precision Trap: A unit might have 98% "precision" (meaning 59 of its top 60 images are butterflies), but this ignores the thousands of other butterfly images it failed to detect (low recall) and the thousands of non-butterfly images that might still trigger it at a slightly lower threshold.
  2. The "Lighthouse" Illusion: Just because a synthetic image generated to maximize a unit looks like a "lighthouse" to a human doesn't mean that unit is a reliable lighthouse detector. In behavioral tests, less than 5% of hidden units in AlexNet's convolutional layers were consistently interpreted as objects by humans.
  3. CCMAS is Misleading: This metric is heavily skewed by the fact that most units have zero activation for the vast majority of images. This "sparsity" makes a unit look highly selective for a category even if its response to that category is weak and unreliable.
Visual evidence of the 'object detector' mirage: Jitterplots (top left/middle) show overlapping activations; Activation Maximization (top right) creates human-interpretable 'lighthouses' that don't reflect unit reliability; Top-activating patches (bottom) can suggest high precision while masking low recall.

Key Results

  • High False Alarms: A unit in AlexNet identified as a "Monarch Butterfly" detector actually had a false-alarm rate of over 99% at its optimal informedness threshold. Most images that activated the unit were not butterflies.
  • Abysmal Recall: For many "detectors," the recall at perfect precision was as low as 8%. This means the unit ignores 92% of the objects it is supposedly detecting to avoid making a single mistake.
  • Consistency in Failure: These findings held true across more modern architectures like VGG-16 and GoogLeNet. Even units with the highest "Network Dissection" scores for categories like "bus" showed poor specificity and high false-alarm rates in jitterplots.
  • Human Interpretation vs. Reality: While humans could identify objects in 71.2% of AM images from the output layer (fc8), they could only do so for ~21% of images from hidden layers (conv5/fc6).

Conclusion and Future Work

The authors conclude that standard CNNs for image classification do not learn "localist" representations where one unit equals one concept. Instead, they likely use distributed representations.

Future directions include: * Applying these signal-detection measures to larger RNNs (like LSTMs) to see if "grandmother cells" emerge there, as RNNs face different pressures like the "superposition catastrophe." * Investigating whether generative models (like GANs or VAEs) develop higher selectivity than classification-only models. * Exploring the gap between how CNNs "see" and how the human visual cortex processes information.

Summary updated

BibTeX
@misc{nguyen2020are,
  author = {Anh Totti Nguyen and Ella M. Gale and Nicholas Martin and Ryan Blything and Anh Nguyen and Jeffrey S. Bowers},
  title = {{Are there any 'object detectors' in the hidden layers of CNNs trained to identify objects or scenes?}},
  year = {2020},
  eprint = {2007.01062},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2007.01062},
}
Figure