CPR++: Object Localization via Single Coarse Point Supervision

Xuehui Yu, Pengfei Chen, Kuiran Wang, Xumeng Han, Guorong Li, Zhenjun Han, Qixiang Ye, Jianbin Jiao

Abstract

Training object localization models typically requires expensive bounding box annotations. Point-based Object Localization (POL) offers a cheaper alternative but suffers from semantic variance due to inconsistent human clicks. This paper presents CPR++, an advanced framework that refines coarse human points into stable "semantic centers" using Multiple Instance Learning (MIL) and a cascaded refinement structure. CPR++ achieves state-of-the-art performance on benchmarks like COCO and Pascal VOC, even outperforming models trained on geometric centers, while maintaining minimal computational overhead.

Introduction

Object localization is a fundamental computer vision task, traditionally reliant on precise bounding box annotations. However, the high cost of box annotation has led to interest in Point-based Object Localization (POL), where a single click per object suffices. The primary challenge in POL is semantic variance: different annotators may click different parts of an object (e.g., the neck vs. the tail of a bird), leading to inconsistent labels for similar semantic regions and confusing the learning model.

Methodology

The authors introduce Coarse Point Refinement (CPR) and its successor, CPR++.

Coarse Point Refinement (CPR)

CPR treats a circular region around the initial human click as a "bag" of candidate points. Using Multiple Instance Learning (MIL), it identifies the most representative points to calculate a "semantic center," which serves as a more stable training target than the raw human click. This process significantly reduces training ambiguity by ensuring that similar semantic regions across different images receive consistent labels.

Refinement process (b) reducing training ambiguity compared to standard label assignment (a)

CPR++ Enhancements

CPR++ builds upon the base refinement logic with three key features:

  • Sampling Region Estimation: The model dynamically adjusts the search radius based on an estimated object scale derived from the spatial distribution of identified points.
  • Cascade Structure: Refinement occurs in stages, progressively narrowing the search area from a coarse region to a fine center point.
  • Variance Regularization: A specialized loss function encourages the model to produce sharp probability peaks at the object center, preventing blurry localization results.

Results and Analysis

CPR++ was evaluated on four major benchmarks: COCO, DOTA-v1.0, Pascal VOC, and a new dataset, SeaPerson, which focuses on tiny person detection from UAV footage. On the COCO dataset, CPR++ achieved a 58.22 mAP₁₀₀, a substantial improvement over the P2PNet baseline of 38.85.

Impact of search radius r on localization performance across different object scales

A critical insight from the ablation studies is the impact of the search radius $r$. As illustrated in the performance charts, smaller $r$ values are optimal for small objects to avoid interference from adjacent instances, whereas larger $r$ values are necessary for large objects to capture sufficient semantic context. CPR++'s dynamic scale estimation effectively manages this trade-off, allowing it to outperform models trained even on "perfect" geometric centers.

Conclusion

CPR++ demonstrates that algorithmic refinement can effectively overcome the inherent noise in human point annotations. By focusing on "semantic centers" rather than rigid geometric ones, it achieves superior localization accuracy with minimal additional training cost. Future work will focus on improving performance in extremely dense or occluded scenes where point-based methods still face challenges.

Summary updated

Abstract

Point-based object localization (POL), which pursues high-performance object sensing under low-cost data annotation, has attracted increased attention. However, the point annotation mode inevitably introduces semantic variance due to the inconsistency of annotated points. Existing POL heavily rely on strict annotation rules, which are difficult to define and apply, to handle the problem. In this study, we propose coarse point refinement (CPR), which to our best knowledge is the first attempt to alleviate semantic variance from an algorithmic perspective. CPR reduces the semantic variance by selecting a semantic centre point in a neighbourhood region to replace the initial annotated point. Furthermore, We design a sampling region estimation module to dynamically compute a sampling region for each object and use a cascaded structure to achieve end-to-end optimization. We further integrate a variance regularization into the structure to concentrate the predicted scores, yielding CPR++. We observe that CPR++ can obtain scale information and further reduce the semantic variance in a global region, thus guaranteeing high-performance object localization. Extensive experiments on four challenging datasets validate the effectiveness of both CPR and CPR++. We hope our work can inspire more research on designing algorithms rather than annotation rules to address the semantic variance problem in POL.
BibTeX
@article{yu2024cpr,
  author = {Xuehui Yu and Pengfei Chen and Kuiran Wang and Xumeng Han and Guorong Li and Zhenjun Han and Qixiang Ye and Jianbin Jiao},
  title = {{CPR++: Object Localization via Single Coarse Point Supervision}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2024},
  url = {https://openreview.net/forum?id=D2aPdmacIy},
}
Figure