FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection

Yi Wei, Shang Su, Jiwen Lu, Jie Zhou

TL;DR

This paper introduces FGR, a novel method for 3D vehicle detection that bypasses the need for expensive 3D annotations by leveraging only 2D bounding boxes and sparse LiDAR point clouds. FGR achieves performance comparable to fully supervised methods, demonstrating the potential for efficient 3D data annotation.

What Problem Does This Paper Address?

Accurate 3D object detection is crucial for applications like autonomous driving and robotics. However, traditional methods rely on extensive 3D ground truth annotations (e.g., precise 3D bounding boxes), which are incredibly time-consuming and costly to acquire. This paper tackles the challenge of performing 3D vehicle detection with significantly weaker supervision, specifically using readily available 2D bounding boxes and the sparse point cloud data from LiDAR sensors, thereby reducing the annotation burden.

Methods

FGR employs a two-stage approach:

  1. Coarse 3D Segmentation:
    • Ground Plane Estimation: RANSAC is used to remove ground points, which can otherwise interfere with object segmentation.
  2. Context-Aware Adaptive Region Growing: This novel algorithm segments objects within a 2D bounding box's frustum. It adaptively adjusts growth thresholds based on local point density and uses contextual information to filter out points belonging to other objects, making it more robust to occlusion and varying point densities.
  3. 3D Bounding Box Estimation:
    • Anti-noise Key Vertex Localization: This stage focuses on estimating the 3D bounding box from the segmented points. It identifies a "key vertex" (the right-angle vertex of the triangle containing the most points) and corresponding "key edges" in an anti-noise manner. An iterative refinement process is used to ensure the key vertex is stable even with noisy segmentation.
  4. Frustum Intersection: The identified key vertex and edges are then used to intersect with the 2D bounding box's frustum. This step leverages the semantic information from the 2D box to refine the 3D bounding box prediction, especially in cases of occlusion.

The generated pseudo-3D labels are then used to train a standard 3D detector.

Datasets

  • KITTI Dataset: Used for both training and evaluation. The authors follow standard splits for training and validation. The KITTI benchmark is divided into Easy, Moderate, and Hard difficulty levels based on object height, occlusion, and truncation.

Key Insights and Ideas

  • 2D + Sparse 3D is Sufficient: The paper demonstrates that semantic information from 2D bounding boxes combined with the geometric structure from sparse LiDAR point clouds is enough for accurate 3D vehicle detection, challenging the necessity of full 3D annotations.
  • Context Matters for Segmentation: The proposed "context-aware adaptive region growing" highlights the importance of considering surrounding points and adaptively adjusting parameters for robust segmentation in complex scenes.
  • Key Vertex Stability for Robustness: Identifying a stable "key vertex" and using an iterative refinement process is crucial for robust 3D bounding box estimation, especially when dealing with noisy segmentation masks.
  • Frustum Intersection for Occlusion Handling: Leveraging the 2D bounding box frustum for intersection is a clever way to constrain and refine 3D predictions, effectively handling partial views and occlusions.
  • Potential as a 3D Annotator: The method's ability to generate high-quality pseudo-3D labels suggests its significant potential as an automated annotation tool, drastically reducing manual labeling effort.

Key Results

  • FGR achieves performance comparable to fully supervised methods on the KITTI dataset, despite using only 2D bounding boxes and sparse point clouds.
  • Ablation studies confirm the necessity of both ground plane estimation and the adaptive region growing for coarse segmentation, and the anti-noise key vertex localization and frustum intersection for accurate 3D bounding box estimation.
  • Qualitative results show FGR can handle both non-occluded and occluded cases effectively.

Limitations and Future Work

  • Segmentation Noise: While FGR incorporates anti-noise mechanisms, the paper acknowledges that segmentation masks can still contain noise, which can influence bounding box estimation.
  • Future Work: The authors suggest that FGR can be applied as a 3D annotator, implying future work could involve scaling this annotation process and exploring its application to other object categories or more complex scenes. The paper also implicitly suggests further improvements to the segmentation and localization modules could yield even better results.

Summary updated

Abstract

In this paper, we investigate the problem of weakly supervised 3D vehicle detection. Conventional methods for 3D object detection usually require vast amounts of manually labelled 3D data as supervision signals. However, annotating large datasets needs huge human efforts, especially for 3D area. To tackle this problem, we propose a frustum-aware geometric reasoning (FGR) method to detect vehicles in point clouds without any 3D annotations. Our method consists of two stages: coarse 3D segmentation and 3D bounding box estimation. For the first stage, a context-aware adaptive region growing algorithm is designed to segment objects based on 2D bounding boxes. Leveraging predicted segmentation masks, we develop an anti-noise approach to estimate 3D bounding boxes in the second stage. Finally 3D pseudo labels generated by our method are utilized to train a 3D detector. Independent of any 3D groundtruth, FGR reaches comparable performance with fully supervised methods on the KITTI dataset. The findings indicate that it is able to accurately detect objects in 3D space with only 2D bounding boxes and sparse point clouds.
BibTeX
@inproceedings{wei2021fgr,
  author = {Yi Wei and Shang Su and Jiwen Lu and Jie Zhou},
  title = {{FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  doi = {10.1109/icra48506.2021.9561245},
}
Figure