Do More Dropouts in Pool5 Feature Maps for Better Object Detection

Zhiqiang Shen, Xiangyang XuearXiv 2014 · arXiv

Abstract

This paper introduces FeatureEdit, a method designed to improve object detection performance by selectively pruning feature channels in the $pool_5$ layer of Convolutional Neural Networks (CNNs). By identifying and "dropping" (zeroing out) "unfriendly" features—those that exhibit high variance within a class or lack discriminative power between classes—the authors achieve significant improvements in mean Average Precision (mAP) on the PASCAL VOC benchmarks compared to the baseline R-CNN architecture.

Introduction

While Deep Convolutional Neural Networks (CNNs) have revolutionized computer vision, they are frequently treated as "black boxes." The final layers produce high-dimensional feature vectors, but the semantic utility of individual channels is often overlooked. The authors of this study argue that not all features contribute equally to classification. For any given object category, some feature channels represent stable, helpful concepts (e.g., "eyes" for a dog), while others represent "unfriendly" or distracting noise (e.g., a specific metal texture that appears inconsistently). Using the entire feature set indiscriminately can lead to overfitting, particularly when training data is limited.

Methodology: The FeatureEdit Algorithm

The proposed FeatureEdit algorithm operates on the $pool_5$ layer, the final convolutional layer before fully connected layers. The method is based on the observation that individual channels in $pool_5$ often align with specific semantic concepts like shapes, textures, or object parts.

The FeatureEdit pipeline: original pool5 feature maps are edited via intra-class and inter-class dropout to produce a more discriminative feature set.

Instead of the random dropout typically used during training, FeatureEdit employs a statistical pruning approach:

  1. Intra-class Edit: The algorithm calculates the variance of a feature channel within a single class. High variance suggests the feature is "unsteady" and unreliable for representing that category.
  2. Inter-class Edit: The algorithm measures how much a feature's mean value varies across different classes. Low variation indicates the feature is not discriminative, as it appears similar regardless of the object type.
  3. Maximum Entropy Principle: This theoretical framework is used to justify dropping the least representative concepts.

The resulting "edited" features are used to train a linear Support Vector Machine (SVM). The authors found that merging the original features with these edited features (FE_M) yielded the most robust performance.

Concept Correspondence and Feature Selection

To validate the approach, the researchers visualized $pool_5$ units to confirm their semantic alignment. As shown in the visualization below, certain features are consistently associated with specific visual parts. FeatureEdit identifies which of these are beneficial for the classifier and which should be suppressed.

Semantic concept correspondence in pool5 units. Helpful features (e.g., eyes) are retained, while "unfriendly" or inconsistent features (e.g., specific metal textures) are dropped.

Experimental Results

The researchers evaluated FeatureEdit on the canonical PASCAL VOC 2007, 2010, and 2012 datasets. The models were pre-trained on ImageNet 2012 before undergoing domain-specific fine-tuning.

Key results include:

  • VOC 2007: Achieved 60.1% mAP, a +1.6% improvement over the standard R-CNN (58.5%).
  • VOC 2010: Achieved 56.4% mAP, a +2.7% improvement over R-CNN (53.7%).
  • VOC 2012: Achieved 56.3% mAP, a +3.0% improvement over R-CNN (53.3%).

The method showed particularly strong gains in specific categories, such as "cow" (+5.1% mAP) and "dog" (+3.5% mAP) on the VOC 2007 dataset.

Discussion and Limitations

The core insight of this work is that removing information can actually improve model performance by cleaning the decision boundary. However, some limitations remain. Both the baseline R-CNN and FeatureEdit struggle with small object detection (e.g., birds or bottles), likely because the $pool_5$ resolution is too coarse. Additionally, the algorithm relies on manual thresholds for dropout rates, which may require tuning for different architectures. Future work may involve applying these editing principles to more modern backbones beyond the AlexNet-style architecture used here.

Summary updated

Abstract

Deep Convolutional Neural Networks (CNNs) have gained great success in image classification and object detection. In these fields, the outputs of all layers of CNNs are usually considered as a high dimensional feature vector extracted from an input image and the correspondence between finer level feature vectors and concepts that the input image contains is all-important. However, fewer studies focus on this deserving issue. On considering the correspondence, we propose a novel approach which generates an edited version for each original CNN feature vector by applying the maximum entropy principle to abandon particular vectors. These selected vectors correspond to the unfriendly concepts in each image category. The classifier trained from merged feature sets can significantly improve model generalization of individual categories when training data is limited. The experimental results for classification-based object detection on canonical datasets including VOC 2007 (60.1%), 2010 (56.4%) and 2012 (56.3%) show obvious improvement in mean average precision (mAP) with simple linear support vector machines.
BibTeX
@misc{shen2014do,
  author = {Zhiqiang Shen and Xiangyang Xue},
  title = {{Do More Dropouts in Pool5 Feature Maps for Better Object Detection}},
  year = {2014},
  eprint = {1409.6911},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1409.6911},
}
Figure