Abstract: We present a method for discovering and exploiting object specific deep learning features and use face detection as a case study. Motivated by the observation that certain convolutional channels of a Convolutional Neural Network (CNN) exhibit object specific responses, we seek to discover and exploit the convolutional channels of a CNN in which neurons are activated by the presence of specific objects in the input image. A method for explicitly fine-tuning a pre-trained CNN to induce an object specific channel (OSC) and systematically identifying it for the human face object has been developed. Based on the basic OSC features, we introduce a multi-resolution approach to constructing robust face heatmaps for fast face detection in unconstrained settings. We show that multi-resolution OSC can be used to develop state of the art face detectors which have the advantage of being simple and compact.
Introduction
While Convolutional Neural Networks (CNNs) have become the standard for computer vision tasks, they are often criticized as "black boxes." Researchers have noted that specific neurons in deep layers naturally activate in response to certain objects, but these responses are often inconsistent or buried within hundreds of other channels. This research addresses two primary challenges: systematically identifying or inducing a specific convolutional channel to function as a reliable object detector, and applying these internal "heatmaps" to build a fast, accurate detection pipeline capable of handling real-world variability such as pose, lighting, and scale.
Methods: Inducing Object Specific Channels
The core of this framework is the Object Specific Channel (OSC). Rather than searching for a pre-existing "face neuron," the authors propose a method to force the network to create one through targeted fine-tuning.
The Masking Trick
A pre-trained AlexNet is fine-tuned on a binary classification task: Face vs. Masked Face. To ensure the network focuses on facial geometry rather than background context, negative samples are created by covering the facial region of a face image with random RGB noise. This "masking trick" incentivizes the network to dedicate specific filters to internal facial structures to distinguish between the two classes.
Following fine-tuning, the authors analyze the 256 channels in the conv5 layer. By calculating a "face-score" (the ratio of activation inside a face bounding box vs. outside), they identified Channel 196 as a potent Face OSC.
Detection Pipeline and Multi-Resolution Heatmaps
To detect faces across various scales, the system employs a multi-resolution approach. The input image is divided into overlapping sub-images at multiple scales, which are processed by the CNN. The resulting OSC activations are merged into a single, high-resolution heatmap using a max-pooling operation across scales.
The final detection pipeline consists of three steps:
- Proposals: Candidate windows are flagged where heatmap intensity exceeds a threshold.
- Refinement: These candidates are processed by a second binary CNN classifier trained with heavy data augmentation (blur, occlusion) to prune false positives.
- NMS: Standard Non-Maximum Suppression merges overlapping detections into final bounding boxes.
Evaluation and Results
The model was evaluated on the PASCAL Face and FDDB (Face Detection Data Set and Benchmark) datasets. On PASCAL Face, the method achieved an Average Precision (AP) of 83.99%. While slightly lower than more complex models like HeadHunter or DPM at the time, it significantly outperformed standard Viola-Jones and other baseline neural network approaches.
Qualitative analysis demonstrates that the detector is highly robust, successfully identifying faces in crowded scenes, extreme profiles, and under significant occlusion. Furthermore, by using the OSC heatmap to generate proposals, the system avoids exhaustive sliding-window searches, making the process computationally efficient.
Conclusion
This work demonstrates that internal CNN representations can be explicitly specialized for object detection through forced specialization. By leveraging the spatial correspondence preserved in deep layers like conv5, a single channel can serve as a high-quality localization map. While modern single-shot detectors have since emerged, this approach offers a simple, compact, and effective method for building robust detectors without the need for manually defined "parts."
Summary updated