Polygon-to-Polygon Distance Loss for Rotated Object Detection

Yang Yang, Jifeng Chen, Xiaopin Zhong, Yuanlong DengAAAI 2022 · DOI

Abstract

Rotated object detection is critical for applications such as aerial imagery analysis and scene text recognition. However, traditional regression-based methods for rotated bounding boxes (RBBoxes) suffer from boundary discontinuities and optimization asynchrony between loss functions and evaluation metrics. This paper introduces the Polygon-to-Polygon (P2P) Distance Loss, a novel approach that treats RBBoxes as convex polygons. By calculating distances based on vertex-to-polygon area sums, the P2P loss eliminates dependency on specific angle definitions and provides stable gradients even for non-overlapping boxes. Experimental results on the DOTA and HRSC2016 datasets demonstrate that P2P loss achieves state-of-the-art accuracy (79.15% mAP on DOTA) with significantly faster convergence than existing specialized loss functions.

Introduction and Problem Statement

In standard object detection, horizontal bounding boxes are typically defined by four parameters. For tasks involving tilted objects, researchers utilize Rotated Bounding Boxes (RBBoxes), adding an angle parameter $\theta$. This parameterization introduces two significant challenges:

  1. Boundary Discontinuity: Angles are periodic. A small physical rotation can cause a numerical jump at the domain boundary (e.g., from $-90^\circ$ to $90^\circ$), leading to loss spikes that destabilize training.
  2. Optimization Asynchrony: Standard regression losses like Smooth $L_1$ do not always correlate with the Intersection over Union (IoU) metric. A model might minimize parameter error while producing a box with poor spatial overlap.
Figure 5: Comparison of different angle definitions ($D_{oc}$ and $D_{le}$) illustrating how the same physical orientation can result in different numerical parameters, leading to boundary issues.

Methodology: The P2P Distance Loss

The proposed P2P Loss shifts the focus from regressing raw parameters to calculating the geometric distance between two convex polygons.

Vertex-to-Polygon Area

The core of the method is the distance based on the area sum of triangles formed by the vertices of one box and the edges of another. If a vertex lies inside the target polygon, the area sum equals the target's area; if it lies outside, the area sum increases proportionally to the distance. By summing these areas for all vertices in both directions, the metric captures both spatial distance and morphological similarity.

Differentiability and Refinements

The distance is calculated using cross products, ensuring it is fully differentiable and compatible with standard deep learning frameworks. To enhance robustness against outliers and prevent gradient underflow, the authors incorporated two auxiliary terms: an Additional Center Loss and a Semiperimeter Loss.

Key Results and Discussion

The P2P Loss was integrated into a RetinaNet baseline and evaluated against major benchmarks. On the DOTA aerial imagery dataset, the model achieved 79.15% mAP, outperforming specialized alternatives like Gaussian Wasserstein Distance (GWD).

Figure 3: Qualitative comparison showing the reduction in boundary discontinuity errors. The left panel shows messy detections from standard methods, while the right panel shows the stabilized, accurate detections achieved with P2P loss.

Advantages of the P2P Approach

  • Definition Independence: Because the loss converts boxes to vertices before calculation, it is immune to the specific coordinate system or angle definition used (e.g., OpenCV vs. long-edge definitions).
  • Non-Overlapping Gradients: Unlike IoU-based losses that provide zero gradient for non-overlapping boxes, P2P Loss provides a constant "pull" toward the ground truth, facilitating earlier training stages.
  • Efficiency: The model achieved superior results in just 12 training epochs, whereas competing methods often required over 40 epochs to converge.

Conclusion

The Polygon-to-Polygon Distance Loss effectively resolves the boundary discontinuity problem in rotated object detection by treating boxes as geometric entities rather than independent parameters. Its mathematical elegance and performance gains on aerial datasets suggest it is a robust alternative for complex detection tasks. Future research may extend this logic to irregular polygons, potentially benefiting medical imaging and advanced robotics.

Summary updated

BibTeX
@inproceedings{yang2022polygon,
  author = {Yang Yang and Jifeng Chen and Xiaopin Zhong and Yuanlong Deng},
  title = {{Polygon-to-Polygon Distance Loss for Rotated Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i3.20214},
}
Figure