Learning to Localize Objects with Noisy Labeled Instances

Xiaopeng Zhang, Yang Yang, Jiashi Feng

Abstract

Weakly Supervised Object Localization (WSOL) aims to identify object extents using only image-level labels. A significant hurdle in WSOL is the "classification-localization gap," where models focus on discriminative parts rather than the whole object. We present SD-LocNet, a framework that treats object locations as latent variables and optimizes them via a self-directed strategy. By leveraging a Localization Stability Score (LSS) to identify reliable instances and propagating their features as priors, SD-LocNet achieves state-of-the-art results on PASCAL VOC and MS COCO datasets.

Introduction

The primary challenge in WSOL is that standard classifiers often converge on the most discriminative parts of an object (e.g., a bird's beak) rather than its full spatial extent. This leads to noisy initializations and local minima during training. SD-LocNet addresses this by identifying "easy" images where localization is stable and using them to guide the learning process for more "difficult" or cluttered scenes.

Methodology

SD-LocNet introduces two core innovations to refine noisy object instances:

1. Adaptive Sampling via Localization Stability Score (LSS)

Traditional methods often rely on high-confidence classification scores, which can be misleading. Instead, SD-LocNet measures temporal stability. If a predicted bounding box remains consistent across multiple training epochs, it is assigned a high LSS and considered a "reliable" instance.

2. Reliability Induced Prior Propagation

Features from reliable instances are clustered to form category-specific priors. These priors then regularize the features of "unreliable" instances, forcing them to align with representative category characteristics. This "easy-to-hard" curriculum helps the model discover the full extent of objects in complex images.

Overview of the Self-Directed Localization process, showing how reliable instances guide the localization of more difficult cases.

Additionally, the network incorporates Multi-context Features using auxiliary convolutional branches to capture varying scales of contextual information, improving background-foreground separation.

Experimental Results

The model was evaluated on three major benchmarks, demonstrating significant improvements over previous state-of-the-art methods like OICR.

  • PASCAL VOC 2007: Achieved 70.9% CorLoc (Correct Localization) and 51.3% mAP, surpassing previous methods by approximately 6.6% in CorLoc.
  • PASCAL VOC 2012: Reached 69.1% CorLoc, a significant increase over the previous best of 62.1%.
  • MS COCO: On this challenging dataset, SD-LocNet improved the baseline localization accuracy from 26.1% to 40.3%.

Conclusion and Future Work

SD-LocNet demonstrates that temporal stability is a more effective signal than raw confidence for weakly supervised learning. By automatically generating a curriculum from "easy" to "hard" images, the model narrows the gap between weakly and fully supervised localization. Future research will focus on improving localization for deformable objects (e.g., humans) and reducing the remaining 15% performance gap compared to fully supervised models.

Summary updated

Abstract

This paper addresses Weakly Supervised Object Localization (WSOL) with only image-level supervision. We model the missing object locations as latent variables, and contribute a novel self-directed optimization strategy to infer them. With the strategy, our developed Self-Directed Localization Network (SD-LocNet) is able to localize object instance whose initial location is noisy. The self-directed inference hinges on an adaptive sampling method to identify reliable object instance via measuring its localization stability score. In this way, the resulted model is robust to noisy initialized object locations which we find is important in WSOL. Furthermore, we introduce a reliability induced prior propagation strategy to transfer object priors of the reliable instances to those unreliable ones by promoting their feature similarity, which effectively refines the unreliable object instances for better localization. The proposed SD-LocNet achieves 70.9% Cor-Loc and 51.3% mAP on PASCAL VOC 2007, surpassing the state-of-the-arts by a large margin.
BibTeX
@inproceedings{zhang2019learning,
  author = {Xiaopeng Zhang and Yang Yang and Jiashi Feng},
  title = {{Learning to Localize Objects with Noisy Labeled Instances}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019219},
}
Figure