Training Object Detectors with Alternative and Weak Supervision

254 concepts217 papers1 proposal

Traditional object detection models require large datasets annotated with precise bounding boxes. This coordinate-level labeling process is incredibly slow and expensive; annotating a single image from the standard MS-COCO dataset can take up to six minutes, as noted by Pei Wang et al. in Omni-DETR: Omni-Supervised Object Detection with Transformers. When scaling models to specialized domains like remote sensing, agriculture, or medical imaging, this manual annotation bottleneck becomes a severe barrier. The challenge intensifies for oriented object detection (OOD), where drawing rotated bounding boxes requires significantly more cognitive effort and time than standard axis-aligned boxes.

To bypass this bottleneck, a prominent research direction focuses on training coordinate-level object detectors using cheaper, alternative forms of supervision. Instead of relying entirely on exhaustive, precise bounding boxes, these methods leverage single points, horizontal boxes, image-level tags, or intelligent active selection to achieve high-performance localization at a fraction of the cost.

Point Supervision: From a Single Click to a Bounding Box

How can a single point click provide enough geometric information to predict a precise bounding box? The intuition is that a point click identifies the presence and approximate center of an object, while external spatial priors or geometric constraints can infer its boundaries.

Dim P. Papadopoulos et al. demonstrated in Training Object Class Detectors with Click Supervision that center-clicks integrated into a Multiple Instance Learning (MIL) framework can drastically reduce annotation time (by 9x to 18x) while approaching full-supervision accuracy.

[Point Click] ───► [Spatial Prior / Foundation Model] ───► [Refined Bounding Box]

In the more complex domain of oriented object detection in aerial imagery, Guangming Cao et al. introduced P2RBox: A Single Point is All You Need for Oriented Object Detection. This framework uses the Segment Anything Model (SAM) to turn point annotations into mask proposals, subsequently refining them via semantic and spatial guidance. To handle dense configurations where masks might overlap or fail, Yi Yu et al. proposed Point2RBox-v2: Rethinking Point-supervised Oriented Object Detection with Spatial Layout Among Instances, which exploits the geometric relationships and mutual exclusivity between neighboring objects to infer precise boundaries. Further expanding this, Teng Zhang et al. developed Point2RBox-v3: Self-Bootstrapping from Point Annotations via Integrated Pseudo-Label Refinement and Utilization, dynamically routing images between SAM and traditional watershed algorithms based on scene density to bootstrap high-quality rotated boxes from minimal input.

Cross-Dimensional Transfer: Leveraging Axis-Aligned Labels

Another powerful strategy is to use cheap, standard horizontal bounding boxes to train detectors capable of predicting complex, rotated bounding boxes. This eliminates the need for annotators to calculate precise angles for every object.

The underlying intuition is that the model can learn the core appearance and localization of an object from horizontal boxes, while transferring the concept of orientation from a small, fully annotated source dataset or self-supervised cues. In Knowledge Combination to Learn Rotated Detection without Rotated Annotation, Tianyu Zhu et al. proposed a co-training framework that successfully transfers rotation concepts across domains, allowing a model trained on horizontal target labels to predict precise rotated boxes.

Similarly, Wei Zhang et al. introduced SPWOOD: Sparse Partial Weakly-Supervised Oriented Object Detection, combining sparse horizontal boxes or points with abundant unlabeled data through pseudo-label filtering to achieve competitive oriented detection without full rotated annotations.

Weak Supervision: Extracting Localization from Image Tags

Can we get object localization for "free" using only image-level labels (e.g., simply stating whether an object is present in an image)? Deep convolutional networks trained on image classification naturally focus on the most discriminative regions of an image to make their predictions. Weakly supervised object detection (WSOD) exploits this behavior.

In their seminal work, Maxime Oquab et al. showed in [Is object localization for free?

  • Weakly-supervised learning with convolutional neural networks](/work/is-object-localization-for-free-weakly-supervised-learning) that global pooling layers allow CNNs to output coarse object locations without coordinate-level training. To refine these coarse regions into tight bounding boxes, Baisheng Lai et al. developed a saliency-guided end-to-end framework in Saliency Guided End-to-End Learning for Weakly Supervised Object Detection, using class-specific saliency maps to generate high-confidence "seed" proposals that guide the detector's bounding box regressions.

Alternatively, instead of drawing boxes, humans can act as a rapid verification loop. The authors of We Don't Need No Bounding-Boxes: Training Object Class Detectors Using Only Human Verification proposed an iterative scheme where the model generates candidate boxes and humans provide rapid "Yes/No" feedback. This prunes the search space and reduces total annotation time by up to 9x while matching fully supervised performance.

Active and Semi-Supervised Synergy: Labeling Only What Matters

When some bounding box annotations are unavoidable, how do we minimize the number of images humans must label? The solution lies in combining Active Learning (AL)—which identifies the most informative samples for human annotation—with Semi-Supervised Learning (SSL), which automatically labels the remaining "easy" samples.

The intuition behind active selection in object detection is to look beyond simple classification uncertainty and focus on localization confusion. Chieh-Chi Kao et al. demonstrated this in Localization-Aware Active Learning for Object Detection, selecting training images based on where the model struggles to locate objects, which slashes labeling costs by 25%. Similarly, Hamed H. Aghdam et al. utilized spatial prediction divergence across network layers in Active Learning for Deep Detection Neural Networks to pinpoint highly informative images.

                  ┌───► High Uncertainty ───► Human Bounding Box Label
                  │
[Unlabeled Data] ─┼───► Low Uncertainty  ───► Automated Pseudo-Label

Once the most informative images are manually labeled, semi-supervised frameworks scale this knowledge across unlabeled data. Yen-Cheng Liu et al. introduced Unbiased Teacher for Semi-Supervised Object Detection, utilizing a teacher-student mutual learning loop to generate reliable pseudo-labels for unlabeled images while overcoming class bias. By pairing these two paradigms, frameworks like the one proposed by Peng Mi et al. in Active Teacher for Semi-Supervised Object Detection allow detectors to reach 100% of their fully supervised performance using less than half of the labeled data.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 225 papers
  2. Version 1 · 2026-10-04 · 175 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_manual_annotation_bottleneck_annotation__bf8a2e47,
  title = {manual annotation bottleneck (annotation bottleneck)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manual-annotation-bottleneck-annotation-bottleneck/rdr_bf8a2e47d32e4f9096ed821c3aebdc21},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_manual_annotation_bottleneck_annotation__bf8a2e47_v2,
  title = {manual annotation bottleneck (annotation bottleneck)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/manual-annotation-bottleneck-annotation-bottleneck/rdr_bf8a2e47d32e4f9096ed821c3aebdc21/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake