TL;DR
This paper challenges the popular notion that deep convolutional neural networks (CNNs) develop "object detector" units in their hidden layers. By comparing various metrics, the authors demonstrate that common measures of selectivity are misleading; units previously labeled as detectors actually suffer from extremely high false-alarm rates or poor hit-rates when evaluated using standard signal-detection theory.
What Problem Does This Paper Address?
In the "standard model" of computer vision, it is often assumed that CNNs learn a hierarchy of features, culminating in hidden units that act as "object detectors"—neurons that fire specifically and exclusively when a certain object (like a dog or a lamp) is present. Previous research has used metrics like precision or Class-Conditional Mean Activity Selectivity (CCMAS) to claim these units exist.
However, different researchers have reached divergent conclusions about how selective these representations really are. The authors of this paper argue that the metrics used to define "object detectors" are flawed. They set out to determine if these units are truly selective or if we are simply seeing "mirages" caused by biased measurement techniques.
Methods and Datasets
The researchers performed a rigorous comparison of several selectivity measures on units within AlexNet, VGG-16, and GoogLeNet. The study utilized two primary large-scale datasets: ImageNet (ILSVRC 2012) and Places-365.
To rigorously test selectivity, the authors compared traditional metrics against Signal-Detection Theory (SDT) measures, including hit-rates (recall), false-alarm rates, and "informedness." They also employed Activation Maximization (AM) via Plug-and-Play Generative Networks (PPGNs) to visualize what maximally triggers units and used Jitterplots to examine the entire distribution of activations across datasets.
Key Insights: Deconstructing the Mirage
The study identifies several reasons why previous metrics overestimated unit selectivity:
- The Precision Trap: A unit might have 98% "precision" (meaning 59 of its top 60 images are butterflies), but this ignores the thousands of other butterfly images it failed to detect (low recall) and the thousands of non-butterfly images that might still trigger it at a slightly lower threshold.
- The "Lighthouse" Illusion: Just because a synthetic image generated to maximize a unit looks like a "lighthouse" to a human doesn't mean that unit is a reliable lighthouse detector. In behavioral tests, less than 5% of hidden units in AlexNet's convolutional layers were consistently interpreted as objects by humans.
- CCMAS is Misleading: This metric is heavily skewed by the fact that most units have zero activation for the vast majority of images. This "sparsity" makes a unit look highly selective for a category even if its response to that category is weak and unreliable.
Key Results
- High False Alarms: A unit in AlexNet identified as a "Monarch Butterfly" detector actually had a false-alarm rate of over 99% at its optimal informedness threshold. Most images that activated the unit were not butterflies.
- Abysmal Recall: For many "detectors," the recall at perfect precision was as low as 8%. This means the unit ignores 92% of the objects it is supposedly detecting to avoid making a single mistake.
- Consistency in Failure: These findings held true across more modern architectures like VGG-16 and GoogLeNet. Even units with the highest "Network Dissection" scores for categories like "bus" showed poor specificity and high false-alarm rates in jitterplots.
- Human Interpretation vs. Reality: While humans could identify objects in 71.2% of AM images from the output layer (fc8), they could only do so for ~21% of images from hidden layers (conv5/fc6).
Conclusion and Future Work
The authors conclude that standard CNNs for image classification do not learn "localist" representations where one unit equals one concept. Instead, they likely use distributed representations.
Future directions include: * Applying these signal-detection measures to larger RNNs (like LSTMs) to see if "grandmother cells" emerge there, as RNNs face different pressures like the "superposition catastrophe." * Investigating whether generative models (like GANs or VAEs) develop higher selectivity than classification-only models. * Exploring the gap between how CNNs "see" and how the human visual cortex processes information.
Summary updated