Abstract
Traditional Convolutional Neural Networks (CNNs) often rely on global image features, which can struggle with the complexity of multi-label classification in messy, real-world scenes. This paper introduces the MsDPD (Multi-scale and Discriminative Part Detectors) framework. By identifying discriminative local object parts across multiple scales and employing a task-driven pooling scheme, MsDPD achieves state-of-the-art performance on PASCAL VOC benchmarks, effectively handling intra-class variations such as occlusion and viewpoint changes.
Introduction
Standard CNN architectures excel at single-label classification by summarizing an entire image into a global feature vector. However, multi-label scenes often contain multiple objects of varying sizes, partially occluded or viewed from different angles. Global features lack the geometric invariance required to distinguish these nuances. For instance, a small object like a bottle might be overlooked when the network focuses on the entire scene. To address this, we propose focusing on local, discriminative parts that are more robust to environmental noise and occlusion.
Methodology
The MsDPD framework is built upon the Single Shot MultiBox Detector (SSD) architecture using a VGG-16 backbone. The approach integrates three core components:
1. Discriminative Part Detectors (DPD)
To identify relevant local features, the model samples thousands of CNN features and applies an entropy-rank scheme. This filter selects "discriminative" parts—those with low entropy that appear consistently in specific categories (e.g., wheels for vehicles)—while discarding high-entropy "background" features like sky or grass.
2. Multi-scale Feature Extraction
Objects and their parts appear at different scales. MsDPD extracts features from five distinct layers (Conv7 through Conv12). Early, high-resolution layers capture small details (e.g., a bird's beak), while deeper, low-resolution layers capture larger structures (e.g., a train engine).
3. Task-Driven Pooling
Unlike standard heuristic pooling (max or average), MsDPD utilizes a task-driven pooling scheme. The part detectors and the classification task are trained jointly using a unified objective function, ensuring that the feature summarization is directly optimized for final classification accuracy.
Experimental Results
The framework was evaluated on the PASCAL VOC 2007 and 2012 datasets.
- PASCAL VOC 2007: MsDPD achieved a mean Average Precision (mAP) of 93.5%, outperforming global CNN approaches (89-90%).
- PASCAL VOC 2012: The model reached an mAP of 92.8%, setting a new benchmark.
Analysis revealed that different classes benefit from specific scales; for example, "bottles" are best detected at smaller scales, whereas "persons" require larger scales. The model showed significant improvements in "difficult" categories such as chair, cow, and sofa, which are prone to occlusion.
Conclusion
MsDPD demonstrates that local, multi-scale discriminative parts provide a more robust foundation for multi-label classification than global features. While the current optimization process is complex and prevents full end-to-end training, the significant gains in mAP on standard benchmarks validate the effectiveness of task-driven part detection. Future work will focus on streamlining the training pipeline for broader applicability.
Summary updated