Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation

Zhaohui Zheng, Ping Wang, Dongwei Ren, Wei Liu, Rongguang Ye, Qinghua Hu, Wangmeng Zuo

Abstract

This paper introduces Complete-IoU (CIoU) loss and Cluster-NMS to address fundamental limitations in bounding box regression and redundant prediction suppression. By incorporating overlap area, central point distance, and aspect ratio into the loss function, and parallelizing Non-Maximum Suppression (NMS) via a clustering approach on GPUs, the proposed methods achieve significant improvements in detection accuracy and inference speed across various state-of-the-art architectures.

Introduction

In modern computer vision, object detection and instance segmentation rely heavily on the precision of bounding box regression and the efficiency of post-processing. Current methodologies often face bottlenecks during training, where loss functions may fail to provide informative gradients, and during inference, where sequential NMS processes create computational delays. This research, published in IEEE Transactions on Cybernetics, proposes a robust framework to enhance geometric modeling in both stages, boosting performance without sacrificing speed.

Geometric Modeling in Bounding Box Regression

Complete-IoU (CIoU) Loss

Traditional loss functions like $\ell_n$-norm or simple Intersection over Union (IoU) exhibit significant flaws. For instance, if a predicted box and a target box do not overlap, the IoU gradient becomes zero, leaving the model without a clear direction for optimization. While Generalized IoU (GIoU) attempted to solve this, it often suffers from slow convergence and struggles with varying aspect ratios.

The authors propose that a "complete" loss function must account for three critical geometric factors:

  1. Overlap Area: The standard IoU term.
  2. Central Point Distance: A penalty based on the normalized distance between the centers of the predicted and target boxes, encouraging rapid overlap.
  3. Aspect Ratio: A novel term ensuring the predicted box's proportions match the target.
Geometric representation of the distance between central points (d) and the diagonal length of the smallest enclosing box (c) used in CIoU loss.

By integrating these factors, CIoU loss provides a smoother loss landscape and clearer optimization instructions, leading to superior localization and faster model convergence.

Cluster-NMS for Efficient Inference

To address the inference bottleneck, the authors move away from traditional sequential NMS. They introduce Cluster-NMS, which frames the suppression of redundant boxes as a clustering problem solvable through matrix operations on a GPU. This parallelization allows for the simultaneous suppression of multiple redundant boxes. Furthermore, because Cluster-NMS is matrix-based, it can easily incorporate the same geometric factors used in the CIoU loss to refine box selection, particularly in crowded scenes.

Experimental Evaluation

Convergence and the "Basin" Effect

Through extensive simulation experiments, the researchers demonstrated that CIoU loss significantly expands the "basin" of effective gradients. Even when anchor boxes are far from the target or possess disparate shapes, CIoU provides a reliable path to the optimal solution where other losses fail.

Simulation experiment showing the distribution of anchor boxes (left) and the resulting convergence error curves for various IoU-based losses (right).

As shown in the simulation results, CIoU reaches near-zero error in significantly fewer iterations compared to IoU and GIoU. This efficiency translates to real-world training gains across datasets like MS COCO and PASCAL VOC.

Performance Gains and Speed

The proposed methods were integrated into several top-tier models, including YOLO v3, SSD, and Faster R-CNN. Key results include:

  • Accuracy: YOLACT on MS COCO achieved a gain of +1.7 AP for object detection and +1.1 AP for instance segmentation.
  • Efficiency: Cluster-NMS drastically reduced processing time. For YOLO v3, NMS latency dropped from 103.9ms to 15.3ms while simultaneously improving accuracy.

Conclusion

By addressing the geometric deficiencies in both the learning and inference phases of object detection, CIoU loss and Cluster-NMS provide a powerful toolkit for modern computer vision. These advancements allow models to converge faster and operate more efficiently in real-time applications, setting a new standard for geometric modeling in the field.

Summary updated

BibTeX
@article{zheng2022enhancing,
  author = {Zhaohui Zheng and Ping Wang and Dongwei Ren and Wei Liu and Rongguang Ye and Qinghua Hu and Wangmeng Zuo},
  title = {{Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation}},
  journal = {IEEE Trans. Cybernetics},
  year = {2022},
  doi = {10.1109/tcyb.2021.3095305},
}
Figure