Point Linking Network for Object Detection

Xinggang Wang, Kaibing Chen, Zilong Huang, Cong Yao, Wenyu LiuarXiv 2017 · OpenReview

Abstract

Object detection is a core problem in computer vision. With the development of deep ConvNets, the performance of object detectors has been dramatically improved. The deep ConvNets based object detectors mainly focus on regressing the coordinates of bounding box, e.g., Faster-R-CNN, YOLO and SSD. Different from these methods that considering bounding box as a whole, we propose a novel object bounding box representation using points and links and implemented using deep ConvNets, termed as Point Linking Network (PLN). Specifically, we regress the corner/center points of bounding-box and their links using a fully convolutional network; then we map the corner points and their links back to multiple bounding boxes; finally an object detection result is obtained by fusing the multiple bounding boxes. PLN is naturally robust to object occlusion and flexible to object scale variation and aspect ratio variation. In the experiments, PLN with the Inception-v2 model achieves state-of-the-art single-model and single-scale results on the PASCAL VOC 2007, the PASCAL VOC 2012 and the COCO detection benchmarks without bells and whistles. The source code will be released.

Introduction

Traditional object detectors like Faster R-CNN, YOLO, and SSD typically treat a bounding box as a single entity, regressing its coordinates $(x, y, w, h)$ directly. While effective, this approach faces challenges with extreme aspect ratios, significant occlusion, and limited global context. The Point Linking Network (PLN) addresses these issues by shifting the detection paradigm from global box regression to a point-localization and association task. By predicting key points and the geometric links between them, PLN provides a more flexible and robust framework for object localization.

Methodology

The core of PLN is a fully convolutional architecture based on Inception-v2 that decomposes a bounding box into a set of points and their relationships.

Instead of regressing a box directly, the network predicts a center point ($O$) and four corner points ($C_1$ to $C_4$). A "link" is defined as a geometric association that identifies which corner belongs to which center. This representation allows the model to handle objects with unusual shapes or those partially hidden from view.

The Point-Link representation defines an object via its center (O) and four corners (C1-C4) connected by geometric links.

Architecture and Training

The input image is processed through an Inception-v2 backbone. The network is then split into four parallel branches, each responsible for a specific center-corner pair (e.g., center to top-left corner). For each grid cell in an $S \times S$ output, the network predicts:

  1. Probability ($P$): The likelihood of a point existing in the cell.
  2. Class Distribution ($Q$): The object category.
  3. Precise Location ($x, y$): Local offsets for sub-pixel accuracy.
  4. Link ($L$): The probability that a point in this cell links to a point in another specific grid cell.
The PLN architecture utilizes an Inception-v2 backbone followed by four parallel branches for center-corner inference and final NMS fusion.

To capture sufficient context for linking distant points, the architecture employs dilated convolutions, which expand the receptive field without increasing parameter count or sacrificing spatial resolution. During inference, pairs with high joint probability are linked to form candidate bounding boxes, which are then fused using Non-Maximum Suppression (NMS).

Experimental Results

PLN was evaluated on several standard benchmarks, demonstrating competitive performance against established baselines.

  • PASCAL VOC 2007: PLN640 (640x640 input) achieved 78.8% mAP, outperforming YOLOv2 (78.6%) and Faster R-CNN (76.4%).
  • MS COCO: On the test-dev2015 set, PLN512 reached 28.9% mAP@[0.5:0.95], significantly higher than SSD512 (26.8%) and YOLOv2 (21.6%).
  • Generalization: On the People-Art dataset, PLN achieved 47% AP, surpassing YOLO (45%) and R-CNN (26%), indicating strong cross-domain robustness.
  • Efficiency: While the unoptimized Python implementation runs at 10 fps, a simplified version achieves 24 fps with minimal accuracy loss (0.5%).

Discussion and Conclusion

The Point Linking Network offers several key advantages. First, predicting center points is inherently more stable than corners because centers are typically surrounded by rich object features. Second, the four-branch structure provides redundancy; if one corner is occluded, the remaining branches can still reconstruct the bounding box. Finally, the anchor-free nature of the model allows it to adapt dynamically to any object scale or aspect ratio.

Future work will focus on optimizing the computational overhead of the linking logic and extending the framework to tasks like human pose estimation or instance segmentation.

Summary updated

Abstract

Object detection is a core problem in computer vision. With the development of deep ConvNets, the performance of object detectors has been dramatically improved. The deep ConvNets based object detectors mainly focus on regressing the coordinates of bounding box, e.g., Faster-R-CNN, YOLO and SSD. Different from these methods that considering bounding box as a whole, we propose a novel object bounding box representation using points and links and implemented using deep ConvNets, termed as Point Linking Network (PLN). Specifically, we regress the corner/center points of bounding-box and their links using a fully convolutional network; then we map the corner points and their links back to multiple bounding boxes; finally an object detection result is obtained by fusing the multiple bounding boxes. PLN is naturally robust to object occlusion and flexible to object scale variation and aspect ratio variation. In the experiments, PLN with the Inception-v2 model achieves state-of-the-art single-model and single-scale results on the PASCAL VOC 2007, the PASCAL VOC 2012 and the COCO detection benchmarks without bells and whistles. The source code will be released.
BibTeX
@misc{wang2017point,
  author = {Xinggang Wang and Kaibing Chen and Zilong Huang and Cong Yao and Wenyu Liu},
  title = {{Point Linking Network for Object Detection}},
  year = {2017},
  eprint = {1706.03646},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1706.03646},
}
Figure