Abstract
The increasing prominence of weakly labeled data nurtures a growing demand for object detection methods that can cope with minimal supervision. We propose an approach that automatically identifies discriminative configurations of visual patterns that are characteristic of a given object class. We formulate the problem as a constrained submodular optimization problem and demonstrate the benefits of the discovered configurations in remedying mislocalizations and finding informative positive and negative training examples. Together, these lead to state-of-the-art weakly-supervised detection results on the challenging PASCAL VOC dataset.
Introduction
Weakly-supervised object detection (WSOD) aims to train models using only image-level labels, avoiding the high cost of manual bounding box annotation. A significant challenge in WSOD is mislocalization: models often focus on the most discriminative part of an object (e.g., a face) while ignoring the full extent of the object (e.g., the body). This research addresses this by identifying consistent configurations of multiple visual parts to define object boundaries more accurately.
Methodology
The proposed approach utilizes a multi-stage pipeline based on discrete optimization:
- Discriminative Patch Discovery: The method identifies patches that frequently appear in positive images but rarely in negative ones. This is framed as a constrained submodular optimization problem. By maximizing a covering function subject to a matroid intersection constraint, the system ensures the selection of spatially diverse and non-redundant patches.
- Configuration Identification: Candidate parts are evaluated for consistent spatial relationships (relative position, scale, and viewpoint) across the dataset. A graph-based approach identifies the most frequent and stable configurations.
- Hard Negative Mining: Configurations help identify "hard negatives." For instance, if a configuration requires both a head and a torso, a localization containing only one is treated as a negative example to refine the detector's precision.
Key Results
The method was evaluated on the PASCAL VOC 2007 benchmark, achieving a Mean Average Precision (mAP) of 24.6%, surpassing the previous state-of-the-art of 22.7%.
The improvement was most significant in the "person" category, where high pose variability typically hinders weakly-supervised methods. Qualitative results show that the model successfully groups disparate parts—such as bike wheels with frames or locomotive bases with carriages—resulting in tighter, more accurate bounding boxes.
Discussion and Future Work
While effective, the current approach is limited to pairwise configurations. Extending this to N-tuples could capture more complex object structures. Additionally, the model occasionally confuses co-occurring objects (e.g., a lamp and a sofa) as parts of a single entity. Future work will focus on scaling the initial patch search and refining the distinction between co-occurrence and part-whole relationships.
Summary updated