Weakly Supervised Object Detection and Localization

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2322 papers345 conceptsfrozen 2026-10-04

The standard recipe for training an object detector requires a human to draw thousands of precise bounding boxes. This "strong supervision" is the primary bottleneck in computer vision; while image-level tags (e.g., "this image contains a dog") are abundant and cheap, spatial annotations are expensive and difficult to scale.

Weakly Supervised Object Detection (WSOD) and Localization (WSOL) aim to bridge this gap. The goal is to teach a model to find where an object is using only the information about what is in the image.

From Classification to Localization

The intuition behind weak supervision begins with a simple observation: to classify an image correctly, a neural network must focus on the object of interest. If a model identifies a "goldfinch," it likely did so by looking at the bird, not the sky.

Bolei Zhou and colleagues formalized this in Learning Deep Features for Discriminative Localization by introducing Class Activation Mapping (CAM). By replacing the dense, fully-connected layers at the end of a network with Global Average Pooling (GAP), they forced the network to maintain spatial information right up until the final classification decision. The weights of the final layer then act as a map, showing exactly which regions of the image contributed most to the classification. This effectively turns any standard classifier into a localizer "for free."

The Discriminative Part Problem

While CAM provides a starting point, it suffers from a fundamental flaw: classifiers are "lazy." To identify a "person," a network might only need to see a face. Consequently, the activation maps often highlight only the most discriminative part of an object rather than its full extent.

Researchers have developed several strategies to force the model to see the "whole" object:

  1. Iterative Refinement: Bin Wang proposed a "low-shot" correction approach in Low Shot Box Correction for Weakly Supervised Object Detection, using a tiny amount of strong supervision to train a network that expands "part" boxes into "full body" boxes.
  2. Cyclic Guidance: In Cyclic Guidance for Weakly Supervised Joint Detection and Segmentation, Yunhang Shen and Rongrong Ji showed that detection and segmentation can "teach" each other. A rough bounding box helps refine a pixel mask, and a better mask helps tighten the bounding box.
  3. Curriculum Learning: Miaojing Shi and Vittorio Ferrari demonstrated in Weakly Supervised Object Localization Using Size Estimates that starting with "easy" images (where objects are large and obvious) and moving to "hard" ones significantly improves the stability of the learning process.

Multiple Instance Learning and EM

Another dominant framing treats the image as a "bag" of potential object regions (proposals). This is known as Multiple Instance Learning (MIL). If an image is labeled "cat," at least one of the proposed regions must contain a cat.

Ziang Yan and Jian Liang utilized the Expectation-Maximization (EM) algorithm to solve this in Weakly- and Semi-Supervised Object Detection with Expectation-Maximization Algorithm. In the E-step, the model guesses which regions are most likely to be the object; in the M-step, the model updates its weights to better recognize those regions. This iterative "guessing and checking" allows the model to eventually converge on the correct spatial locations.

Alternative Sources of Weakness

Weak supervision isn't limited to just binary presence/absence tags. Researchers are increasingly looking at other "low-cost" signals:

By moving away from the "one box per object" paradigm, these methods allow us to train vision models on the vast, uncurated datasets of the web, bringing us closer to how humans learn to recognize the world: by seeing, hearing, and describing, rather than by drawing boxes.

Papers in this version

Showing 60 of 322 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 204 papers
  2. Version 1 · 2026-10-04 · 322 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_annotation_cost_bottleneck_f550f90e,
  title = {annotation cost bottleneck},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/annotation-cost-bottleneck/rdr_f550f90e702147b3a98bfde6d80082f3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_weakly_supervised_object_detection_and_l_f550f90e_v1,
  title = {Weakly Supervised Object Detection and Localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/weakly-supervised-object-detection-and-localization/rdr_f550f90e702147b3a98bfde6d80082f3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}