Abstract
Point-Supervised Object Detection (PSOD) in a discriminative style has recently gained significant attention for its impressive detection performance and cost-effectiveness. However, accurately predicting high-quality pseudo-box labels for drone-view images, which often feature densely packed small objects, remains a challenge. This difficulty arises primarily from the limitation of rigid sampling strategies, which hinder the pseudo-box optimization process. To address this, we propose PointSR, an effective and robust point-supervised object detection framework with self-regularized sampling that integrates temporal and informative constraints throughout the pseudo-box generation process. Specifically, the framework comprises three key components: Temporal-Ensembling Encoder (TE Encoder), Coarse Pseudo-box Prediction, and Pseudo-box Refinement. Experimental results on benchmark datasets demonstrate that PointSR significantly outperforms state-of-the-art methods, achieving up to 2.6%–7.2% higher $AP_{50}$ using only point supervision. Additionally, it exhibits strong robustness to perturbation in human-labeled points.
Introduction
Detecting objects from drones is essential for applications ranging from urban planning to disaster response. However, the standard training paradigm requires precise bounding box annotations, which are notoriously expensive and time-consuming to produce for drone-view imagery. In these scenes, objects are typically small, numerous, and densely packed. Point-level supervision—where an annotator simply clicks the center of an object—is roughly 18 times faster than drawing boxes, but it presents a significant technical challenge: how do we accurately infer the size and shape of an object from a single pixel?
Existing Point-Supervised Object Detection (PSOD) methods often rely on "rigid" sampling strategies. These methods use fixed anchors that do not adapt to the specific scene and often sample background areas randomly. This lack of adaptability makes it difficult for models to distinguish between closely spaced objects or to accurately estimate boundaries in complex drone environments.
Methodology: Self-Regularized Sampling
PointSR addresses these limitations through a self-regularized sampling framework that moves beyond static anchors. The system is built on three primary components:
- Temporal-Ensembling Encoder (TE Encoder): Instead of relying on pre-defined box shapes, the TE Encoder aggregates information from previous training epochs to build a "prototype library" of object aspect ratios. Since an object's aspect ratio is more consistent than its absolute scale in drone shots, this allows the model to dynamically adjust its anchor predictions as training progresses.
- Informative-Sample Collector (IS Collector): Rather than picking random background samples, PointSR identifies "informative" negatives—areas where the model's current predictions have high loss. By focusing on these difficult regions near the objects, the model learns to better define object boundaries.
- Pseudo-box Refinement: The framework employs a two-stage refinement process. It first generates coarse pseudo-boxes using the learned prototypes and then uses the informative negative samples to suppress low-confidence candidates, resulting in high-quality training labels.
Key Insights and Results
A critical insight of PointSR is the prioritization of aspect ratio over scale. While the apparent size of a vehicle changes with drone altitude, its proportions remain relatively stable. By ensembling this information over time, PointSR avoids the instability and "jitter" common in weakly supervised learning.
The framework was evaluated on three major benchmarks: DroneVehicle, VisDrone, and UAVDT. PointSR achieved state-of-the-art performance, with an $AP_{50}$ of 42.5% on DroneVehicle, 38.7% on VisDrone, and 35.0% on UAVDT. This represents a significant improvement of 2.6% to 7.2% over previous point-supervised methods. Furthermore, the mean Intersection over Union (mIoU) of the generated pseudo-boxes improved by up to 9.5%, indicating much higher label quality. PointSR also demonstrated remarkable robustness to human error; even when point labels were intentionally shifted, the model maintained stable detection performance.
Conclusion
PointSR significantly narrows the performance gap between point-level and box-level supervision for drone-view object detection. By leveraging temporal ensembling and informative negative sampling, it provides a robust solution for training high-quality detectors at a fraction of the annotation cost. Future research may explore extending these temporal constraints to video data to further enhance boundary estimation through motion cues.
Summary updated