Abstract
Object detection is a core problem in computer vision. With the development of deep ConvNets, the performance of object detectors has been dramatically improved. The deep ConvNets based object detectors mainly focus on regressing the coordinates of bounding box, e.g., Faster-R-CNN, YOLO and SSD. Different from these methods that considering bounding box as a whole, we propose a novel object bounding box representation using points and links and implemented using deep ConvNets, termed as Point Linking Network (PLN). Specifically, we regress the corner/center points of bounding-box and their links using a fully convolutional network; then we map the corner points and their links back to multiple bounding boxes; finally an object detection result is obtained by fusing the multiple bounding boxes. PLN is naturally robust to object occlusion and flexible to object scale variation and aspect ratio variation. In the experiments, PLN with the Inception-v2 model achieves state-of-the-art single-model and single-scale results on the PASCAL VOC 2007, the PASCAL VOC 2012 and the COCO detection benchmarks without bells and whistles. The source code will be released.
Introduction
Traditional object detectors like Faster R-CNN, YOLO, and SSD typically treat a bounding box as a single entity, regressing its coordinates $(x, y, w, h)$ directly. While effective, this approach faces challenges with extreme aspect ratios, significant occlusion, and limited global context. The Point Linking Network (PLN) addresses these issues by shifting the detection paradigm from global box regression to a point-localization and association task. By predicting key points and the geometric links between them, PLN provides a more flexible and robust framework for object localization.
Methodology
The core of PLN is a fully convolutional architecture based on Inception-v2 that decomposes a bounding box into a set of points and their relationships.
The Point-Link Representation
Instead of regressing a box directly, the network predicts a center point ($O$) and four corner points ($C_1$ to $C_4$). A "link" is defined as a geometric association that identifies which corner belongs to which center. This representation allows the model to handle objects with unusual shapes or those partially hidden from view.
Architecture and Training
The input image is processed through an Inception-v2 backbone. The network is then split into four parallel branches, each responsible for a specific center-corner pair (e.g., center to top-left corner). For each grid cell in an $S \times S$ output, the network predicts:
- Probability ($P$): The likelihood of a point existing in the cell.
- Class Distribution ($Q$): The object category.
- Precise Location ($x, y$): Local offsets for sub-pixel accuracy.
- Link ($L$): The probability that a point in this cell links to a point in another specific grid cell.
To capture sufficient context for linking distant points, the architecture employs dilated convolutions, which expand the receptive field without increasing parameter count or sacrificing spatial resolution. During inference, pairs with high joint probability are linked to form candidate bounding boxes, which are then fused using Non-Maximum Suppression (NMS).
Experimental Results
PLN was evaluated on several standard benchmarks, demonstrating competitive performance against established baselines.
- PASCAL VOC 2007: PLN640 (640x640 input) achieved 78.8% mAP, outperforming YOLOv2 (78.6%) and Faster R-CNN (76.4%).
- MS COCO: On the test-dev2015 set, PLN512 reached 28.9% mAP@[0.5:0.95], significantly higher than SSD512 (26.8%) and YOLOv2 (21.6%).
- Generalization: On the People-Art dataset, PLN achieved 47% AP, surpassing YOLO (45%) and R-CNN (26%), indicating strong cross-domain robustness.
- Efficiency: While the unoptimized Python implementation runs at 10 fps, a simplified version achieves 24 fps with minimal accuracy loss (0.5%).
Discussion and Conclusion
The Point Linking Network offers several key advantages. First, predicting center points is inherently more stable than corners because centers are typically surrounded by rich object features. Second, the four-branch structure provides redundancy; if one corner is occluded, the remaining branches can still reconstruct the bounding box. Finally, the anchor-free nature of the model allows it to adapt dynamically to any object scale or aspect ratio.
Future work will focus on optimizing the computational overhead of the linking logic and extending the framework to tasks like human pose estimation or instance segmentation.
Summary updated