Fuzzy-NMS: Improving 3D Object Detection With Fuzzy Classification in NMS

Huaping Liu(刘华平), Li Wang, Xinyu Zhang, Fachuan Zhao, Chuze Wu, Yichen Wang, Ziying Song, Lei Yang, Bin Xu, Jun Li, Shuzhi Sam GeIEEE Trans. Intell. veh 2023

TL;DR

This paper introduces Fuzzy-NMS, a novel post-processing module for 3D object detection that uses fuzzy logic to dynamically adjust Non-Maximum Suppression (NMS) thresholds. By considering object volume and clustering density, Fuzzy-NMS significantly improves the detection of small, overlapping objects, offering a plug-and-play solution that enhances existing 3D detectors without retraining.

What Problem Does This Paper Address?

3D object detection is crucial for applications like autonomous driving, providing richer information than 2D detection (category, location, size, and movement). A key challenge in 3D object detection is accurately identifying and localizing objects, especially in complex scenes with many overlapping instances. Non-Maximum Suppression (NMS) is a standard post-processing step to eliminate redundant bounding boxes. However, traditional NMS relies on a fixed Intersection over Union (IoU) threshold, which is often suboptimal. This is because the ideal threshold varies greatly depending on object size and density: small, closely packed objects (like pedestrians) require higher IoU thresholds to avoid merging, while large, sparse objects (like cars) need lower thresholds to prevent false positives. Existing methods struggle to adapt these thresholds dynamically, leading to missed detections or false positives, particularly for small objects.

Methods

The core innovation is the Fuzzy-NMS module, which acts as a plug-and-play component after the initial detection head. It addresses the NMS thresholding problem by:

  1. Volume and Density Estimation: For each candidate bounding box, the module calculates its 3D volume and its clustering density using the DBSCAN algorithm. DBSCAN is chosen for its ability to handle arbitrary cluster shapes and its efficiency.
  2. Fuzzy Classification: The estimated volume and density are fed into a fuzzy classification system. This system, based on a Mamdani-type fuzzy inference engine, categorizes bounding boxes into three classes:
  3. LVHD (Large-Volume High-Density): Large objects that are densely packed.
  4. SVHD (Small-Volume High-Density): Small objects that are densely packed.
  5. LD (Low-Density): Objects that are sparse, regardless of size. This classification is guided by fuzzy rules derived from human prior knowledge and statistical analysis of object distributions.

  6. Dynamic Threshold Assignment: Based on the fuzzy classification, Fuzzy-NMS assigns different NMS thresholds to each category. This allows for more precise filtering, ensuring that densely packed small objects are not suppressed too aggressively, while sparse large objects are handled appropriately.

  7. Final Filtering: The final bounding box selection is then performed using the predicted scores from the detection head, combined with the dynamically adjusted NMS thresholds.

Datasets

The proposed Fuzzy-NMS module was evaluated on two prominent 3D object detection benchmarks:

  • KITTI 3D Object Detection Dataset: This dataset is widely used for autonomous driving research and includes training, validation, and testing sets with car, pedestrian, and cyclist categories across different difficulty levels.
  • Waymo Open Dataset: A larger-scale dataset also used for autonomous driving, providing further validation of the method's scalability.

The authors used official evaluation tools and benchmarked against official model zoo submissions, replacing only the NMS module without retraining the entire network to isolate the impact of Fuzzy-NMS.

Key Insights and Ideas

  • Fuzzy Logic for Adaptive Thresholding: The paper elegantly demonstrates how fuzzy logic can be used to bridge the gap between discrete, fixed thresholds and the continuous, varying nature of object detection challenges.
  • Volume and Density as Key Features: Recognizing that object volume and spatial density are critical indicators for determining appropriate NMS thresholds is a crucial insight. This moves beyond simple IoU calculations.
  • Improved Small Object Detection: The method shows a particular strength in improving the detection of small objects (pedestrians, cyclists), which are often missed by traditional NMS due to their tendency to be densely packed and overlap.
  • Plug-and-Play Generality: The fact that Fuzzy-NMS can be integrated into existing 3D detectors without retraining is a significant practical advantage, making it easily adoptable.

Key Results

  • Significant Performance Gains: Fuzzy-NMS consistently improved the performance of various baseline 3D object detection methods (e.g., PointPillars, PV-RCNN, IA-SSD) on both KITTI and Waymo datasets.
  • Average 3D mAP Improvements: For IA-SSD and PointPillars on KITTI, average 3D mAP improvements of 1.29% and 0.71% were observed, respectively.
  • BEV mAP Improvements: Correspondingly, BEV mAP improvements of 1.16% and 0.85% were noted.
  • Enhanced Small Object Detection: Improvements were particularly pronounced for pedestrians and cyclists, with some categories seeing increases of 0.50% or more, and even exceeding 2-3% in some cases.
  • Qualitative Improvements: Visualizations (Fig. 9) showed Fuzzy-NMS effectively removing false detections and improving the recall of missed objects, especially in challenging scenarios with sparse or overlapping objects.
  • Runtime: While slightly slower than traditional NMS (10.08ms vs. 6.30ms for PointPillars), Fuzzy-NMS is still competitive and significantly faster than some other advanced NMS methods like DIoU-NMS. The overall detection pipeline with Fuzzy-NMS added to PointPillars took 55.38ms per frame, compared to 48.10ms without it.

Limitations and Future Work

  • Performance on Large Objects: The authors acknowledge that Fuzzy-NMS's performance gains are less pronounced for large objects like cars, as these objects tend to be less frequently overlapping. In some cases, performance on cars was even slightly lower than the original baselines.
  • Future Work: The authors suggest further research into lightweight networks and NMS for efficient object detection on resource-constrained devices, aiming for more accurate solutions with wider practical implications. They also plan to explore how to further optimize NMS for different object categories and scenarios.

Summary updated

BibTeX
@article{liu2023fuzzy,
  author = {Huaping Liu(刘华平) and Li Wang and Xinyu Zhang and Fachuan Zhao and Chuze Wu and Yichen Wang and Ziying Song and Lei Yang and Bin Xu and Jun Li and Shuzhi Sam Ge},
  title = {{Fuzzy-NMS: Improving 3D Object Detection With Fuzzy Classification in NMS}},
  journal = {IEEE Trans. Intell. veh},
  year = {2023},
}
Figure