Abstract
This paper introduces FeatureEdit, a method designed to improve object detection performance by selectively pruning feature channels in the $pool_5$ layer of Convolutional Neural Networks (CNNs). By identifying and "dropping" (zeroing out) "unfriendly" features—those that exhibit high variance within a class or lack discriminative power between classes—the authors achieve significant improvements in mean Average Precision (mAP) on the PASCAL VOC benchmarks compared to the baseline R-CNN architecture.
Introduction
While Deep Convolutional Neural Networks (CNNs) have revolutionized computer vision, they are frequently treated as "black boxes." The final layers produce high-dimensional feature vectors, but the semantic utility of individual channels is often overlooked. The authors of this study argue that not all features contribute equally to classification. For any given object category, some feature channels represent stable, helpful concepts (e.g., "eyes" for a dog), while others represent "unfriendly" or distracting noise (e.g., a specific metal texture that appears inconsistently). Using the entire feature set indiscriminately can lead to overfitting, particularly when training data is limited.
Methodology: The FeatureEdit Algorithm
The proposed FeatureEdit algorithm operates on the $pool_5$ layer, the final convolutional layer before fully connected layers. The method is based on the observation that individual channels in $pool_5$ often align with specific semantic concepts like shapes, textures, or object parts.
Instead of the random dropout typically used during training, FeatureEdit employs a statistical pruning approach:
- Intra-class Edit: The algorithm calculates the variance of a feature channel within a single class. High variance suggests the feature is "unsteady" and unreliable for representing that category.
- Inter-class Edit: The algorithm measures how much a feature's mean value varies across different classes. Low variation indicates the feature is not discriminative, as it appears similar regardless of the object type.
- Maximum Entropy Principle: This theoretical framework is used to justify dropping the least representative concepts.
The resulting "edited" features are used to train a linear Support Vector Machine (SVM). The authors found that merging the original features with these edited features (FE_M) yielded the most robust performance.
Concept Correspondence and Feature Selection
To validate the approach, the researchers visualized $pool_5$ units to confirm their semantic alignment. As shown in the visualization below, certain features are consistently associated with specific visual parts. FeatureEdit identifies which of these are beneficial for the classifier and which should be suppressed.
Experimental Results
The researchers evaluated FeatureEdit on the canonical PASCAL VOC 2007, 2010, and 2012 datasets. The models were pre-trained on ImageNet 2012 before undergoing domain-specific fine-tuning.
Key results include:
- VOC 2007: Achieved 60.1% mAP, a +1.6% improvement over the standard R-CNN (58.5%).
- VOC 2010: Achieved 56.4% mAP, a +2.7% improvement over R-CNN (53.7%).
- VOC 2012: Achieved 56.3% mAP, a +3.0% improvement over R-CNN (53.3%).
The method showed particularly strong gains in specific categories, such as "cow" (+5.1% mAP) and "dog" (+3.5% mAP) on the VOC 2007 dataset.
Discussion and Limitations
The core insight of this work is that removing information can actually improve model performance by cleaning the decision boundary. However, some limitations remain. Both the baseline R-CNN and FeatureEdit struggle with small object detection (e.g., birds or bottles), likely because the $pool_5$ resolution is too coarse. Additionally, the algorithm relies on manual thresholds for dropout rates, which may require tuning for different architectures. Future work may involve applying these editing principles to more modern backbones beyond the AlexNet-style architecture used here.
Summary updated