Instance-Level Contrastive Learning for Weakly Supervised Object Detection

Ming Zhang, Bing ZengSensors 2022 · DOI

TL;DR

This paper introduces an Instance-Level Contrastive Learning (ICL) framework for Weakly Supervised Object Detection (WSOD). By leveraging instance correlations across multiple images and a novel memory bank, their method significantly improves object detection accuracy and localization, addressing a key limitation of existing WSOD approaches that focus on individual images.

What Problem Does This Paper Address?

Object detection is a crucial task in computer vision, but traditional methods require detailed bounding box annotations for every object in every image. This is incredibly time-consuming and expensive. Weakly Supervised Object Detection (WSOD) aims to alleviate this by using only image-level labels (i.e., knowing if an object class is present or absent in an image) to train detectors. However, a major challenge in WSOD is that current methods often struggle with object appearance variations, occlusions, and scale differences because they primarily learn from individual images. This can lead to incomplete or inaccurate object instance representations, resulting in false detections. The paper tackles this by proposing a method that learns from the collective knowledge of instances across all training images, not just the current one.

Methods

The core of the proposed solution is the Instance-Level Contrastive Learning (ICL) framework. Here's a breakdown of its key components:

  • Instance-Level Contrastive Learning (ICL): This is the overarching framework. It uses a contrastive learning mechanism to pull representations of similar object instances (from different images) closer together in an embedding space, while pushing apart representations of different object classes. This helps learn more robust and discriminative instance representations.
  • Instance-Diverse Memory Updating (IMU): To capture the diversity of instances (different appearances, sizes, shapes), IMU maintains a memory bank for each class. Instead of storing a single representation per class, it stores multiple representation vectors. This memory bank is updated dynamically, incorporating background information to further enhance foreground representations.
  • Memory-Aware Instance Mining (MIM): This algorithm leverages the memory bank to identify more reliable object instances. Unlike methods that solely rely on proposal confidence, MIM combines this with the similarity of proposal features to the diverse instances stored in the memory bank. This helps in mining more complete and accurate object instances.
  • Memory-Aware Proposal Sampling (MPS): To address the common issue of imbalanced positive and negative samples in WSOD, MPS uses the memory bank to strategically select more positive proposals and discard unreliable negative ones. This helps in creating a more balanced training set for the detector.

The ICL framework integrates these components into an end-to-end trainable system.

Datasets

The experiments were conducted on two widely used benchmark datasets for weakly supervised object detection:

  • PASCAL VOC2007
  • PASCAL VOC2012

These datasets provide image-level annotations for training and are standard for evaluating WSOD methods.

Key Insights and Ideas

  • Cross-Image Instance Correlation is Key: The paper's central insight is that learning from individual images is insufficient for robust WSOD. By explicitly modeling correlations between instances across different images, the model can learn more generalized and complete representations, overcoming issues like occlusion.
  • Diversity in Memory is Crucial: Storing only one representation per object class is too simplistic. The IMU algorithm's approach of using multiple diverse representations in a memory bank is vital for capturing the variability of real-world objects.
  • Memory as a Knowledge Base: The memory bank acts as a cumulative knowledge base, allowing the model to learn from past experiences (other images) to improve current predictions. This is a powerful concept for data-scarce scenarios like WSOD.
  • Balancing the Learning Signal: The MPS algorithm highlights the importance of carefully managing positive and negative samples. Simply relying on high-confidence proposals can lead to bias; intelligently sampling based on memory can create a more effective training signal.

Key Results

The proposed ICL framework demonstrated significant improvements over its baseline (OICR) on both datasets:

  • PASCAL VOC2007:
    • mAP: 14.2% improvement (from 41.2% to 55.4%)
    • CorLoc: 13.4% improvement (from 60.6% to 74.0%)
  • PASCAL VOC2012:
    • mAP: 12.2% improvement (from 37.9% to 50.1%)
    • CorLoc: 8.3% improvement (from 62.1% to 70.4%)

The paper also showed that each component (IMU, MIM, MPS) contributed positively to the overall performance, with IMU providing a substantial boost.

Limitations and Future Work

The authors acknowledge that the performance of WSOD methods is inherently limited by the quality of stored representations. If the memory bank contains noisy representations, it can negatively impact the learned object instances.

Future work suggested by the authors includes:

  • Exploring contextual information from region proposals.
  • Leveraging segmentation information from images to better perceive object boundaries and locate instances.
  • Investigating how to further improve the mining of reliable instance representations, especially in challenging scenarios like highly overlapping objects or distant small objects.

Summary updated

Abstract

Weakly supervised object detection (WSOD) has received increasing attention in object detection field, because it only requires image-level annotations to indicate the presence or absence of target objects, which greatly reduces the labeling costs. Existing methods usually focus on the current individual image to learn object instance representations, while ignoring instance correlations between different images. To address this problem, we propose an instance-level contrastive learning (ICL) framework to mine reliable instance representations from all learned images, and use the contrastive loss to guide instance representation learning for the current image. Due to the diversity of instances, with different appearances, sizes or shapes, we propose an instance-diverse memory updating (IMU) algorithm to mine different instance representations and store them in a memory bank with multiple representation vectors per class, which also considers background information to enhance foreground representations. With the help of memory bank, we further propose a memory-aware instance mining (MIM) algorithm that combines proposal confidence and instance similarity across images to mine more reliable object instances. In addition, we also propose a memory-aware proposal sampling (MPS) algorithm to sample more positive proposals and remove some negative proposals to balance the learning of positive-negative samples. We conduct extensive experiments on the PASCAL VOC2007 and VOC2012 datasets, which are widely used in WSOD, to demonstrate the effectiveness of our method. Compared to our baseline, our method brings 14.2% mAP and 13.4% CorLoc gains on PASCAL VOC2007 dataset, and 12.2% mAP and 8.3% CorLoc gains on PASCAL VOC2012 dataset.
BibTeX
@inproceedings{zhang2022instance,
  author = {Ming Zhang and Bing Zeng},
  title = {{Instance-Level Contrastive Learning for Weakly Supervised Object Detection}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22197525},
}
Figure