Weakly Supervised Object Localization as Domain Adaption

Lei Zhu, Qi She, Qian Chen, Yunfei You, Boyu Wang, Yanye Lu

Abstract

Weakly Supervised Object Localization (WSOL) aims to identify object locations using only image-level labels, bypassing the need for expensive bounding box or pixel-level annotations. However, traditional methods based on Class Activation Mapping (CAM) often suffer from the "discriminative part" problem, where the model focuses only on the most salient features (e.g., a bird's beak) rather than the entire object. This paper introduces DA-WSOL, a novel framework that reframes WSOL as a Domain Adaptation (DA) task. By treating image-level features as a source domain and pixel-level features as a target domain, the proposed method forces the model to generalize across domains, leading to more comprehensive object localization. Experimental results across ImageNet, CUB-200, and OpenImages benchmarks demonstrate that DA-WSOL significantly outperforms existing state-of-the-art methods.

Introduction

The primary challenge in WSOL is the inherent gap between the training objective (image-level classification) and the testing requirement (pixel-level localization). Standard approaches rely on Multi-Instance Learning (MIL), where a model is rewarded for identifying any part of an object that confirms its class. This often leads to "lazy" models that ignore the full extent of an object once the classification confidence is high enough.

To address this, the authors propose viewing the transition from image-level features to pixel-level features as a domain shift. By applying techniques from Domain Adaptation—traditionally used to bridge gaps between synthetic and real-world data—the model can be encouraged to align its classification capabilities with the broader spatial distribution of the object.

Methodology: The DA-WSOL Pipeline

The DA-WSOL framework introduces two primary components to bridge the gap between image-level and pixel-level representations.

1. Target Sampling Strategy (TSA)

Because the pixel-level target domain is significantly larger and noisier than the image-level source domain, the authors implement a Target Sampling Strategy. Using a "cache matrix" and K-means clustering, pixels are categorized into three distinct subsets:

  • Fake Target Set ($T^f$): Pixels highly correlated with the image label, representing the easily identified discriminative parts.
  • Universum Target Set ($T^u$): Background pixels that do not belong to any source class.
  • Real Target Set ($T^t$): The remaining pixels that represent the full extent of the object.

2. Domain Adaptation Localization (DAL) Loss

The core of the method is the DAL loss, which consists of two main functions:

  • Alignment: This uses standard DA techniques, such as Maximum Mean Discrepancy (MMD) or Adversarial Learning, to align the feature distributions of the source (images) and target (pixels). This alignment forces the estimator trained on image labels to maintain performance when applied to individual pixels.
  • Universum Regularization: By utilizing the background pixels ($T^u$) as a "Universum" set, the model learns to tighten its decision boundaries. Minimizing the feature strength of these background samples allows the model to suppress noise and more precisely define the edges of the object.

Experimental Results

The researchers evaluated DA-WSOL on three major computer vision benchmarks, comparing it against traditional CAM and other state-of-the-art (SOTA) methods.

Dataset Metric DA-WSOL Performance Improvement over Baseline
ImageNet GT-known Loc 70.27% +5.55% over CAM
OpenImages pIoU 49.68% +7.48% over previous SOTA
CUB-200 GT-known Loc Competitive Significant qualitative coverage

Qualitative analysis confirms that while standard CAM highlights small "hotspots," DA-WSOL produces heatmaps that cover the entire body of the subject, resulting in much more accurate bounding boxes.

Key Insights

  • WSOL as a Generalization Problem: The fundamental contribution of this work is the realization that the gap between image-level training and pixel-level testing is essentially a domain gap. This allows the field to leverage mature tools from Domain Adaptation to solve localization issues.
  • The Value of Background: Rather than treating the background as simple noise, DA-WSOL uses it as a "Universum" set to define what the object is not, which effectively sharpens the localization of what the object is.
  • End-to-End Architecture: Unlike previous multi-stage methods that require separate structures for classification and region generation, DA-WSOL provides a unified, end-to-end pipeline.

Limitations and Future Work

Despite its success, the authors note that forcing a model to focus on the "whole" object can occasionally lead to a slight decrease in image-level classification accuracy, as the model is no longer permitted to focus solely on the most reliable discriminative features. Additionally, the K-means clustering within the TSA increases computational overhead during training. Future research may explore more efficient sampling strategies or the application of this DA perspective to weakly supervised semantic segmentation.

Summary updated

Abstract

Weakly supervised object localization (WSOL) focuses on localizing objects only with the supervision of image-level classification masks. Most previous WSOL methods follow the classification activation map (CAM) that localizes objects based on the classification structure with the multi-instance learning (MIL) mechanism. However, the MIL mechanism makes CAM only activate discriminative object parts rather than the whole object, weakening its performance for localizing objects. To avoid this problem, this work provides a novel perspective that models WSOL as a domain adaption (DA) task, where the score estimator trained on the source/image domain is tested on the target/pixel domain to locate objects. Under this perspective, a DA-WSOL pipeline is designed to better engage DA approaches into WSOL to enhance localization performance. It utilizes a proposed target sampling strategy to select different types of target samples. Based on these types of target samples, domain adaption localization (DAL) loss is elaborated. It aligns the feature distribution between the two domains by DA and makes the estimator perceive target domain cues by Universum regularization. Experiments show that our pipeline outperforms SOTA methods on multi benchmarks. Code are released at \url{https://github.com/zh460045050/DA-WSOL_CVPR2022}.
BibTeX
@inproceedings{zhu2022weakly,
  author = {Lei Zhu and Qi She and Qian Chen and Yunfei You and Boyu Wang and Yanye Lu},
  title = {{Weakly Supervised Object Localization as Domain Adaption}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
}
Figure