Shape-Prior-Based Point Cloud Completion for Single-Stage Fully Sparse 3d Object Detection

K. Wang, Mingqian Ji, Jian Yang, Shanshan ZhangSSRN 2025 · DOI

TL;DR

This paper introduces a novel shape-prior-based point cloud completion method designed to boost the performance of single-stage, fully sparse 3D object detectors. By effectively addressing the challenges of point cloud sparsity and incompleteness, their approach significantly improves detection accuracy without compromising computational efficiency.

What Problem Does This Paper Address?

Autonomous driving systems heavily rely on LiDAR-based 3D object detection to perceive their surroundings. While recent advancements have led to efficient single-stage, fully sparse detectors, a critical limitation remains: the inherent sparsity and incompleteness of LiDAR point clouds. These issues, especially in occluded or distant areas, significantly hinder the detectors' ability to accurately identify and localize objects. Existing completion methods often target two-stage detectors, are computationally expensive, or depend on proposal boxes, making them unsuitable for the faster, single-stage paradigm. This paper aims to bridge this gap by proposing a completion method that seamlessly integrates with single-stage, fully sparse detectors.

Methods

The proposed method, termed Shape-Prior-Based Point Cloud Completion (SPPCC), consists of two core modules:

  • Instance Selection: This module is designed to identify foreground objects even when the baseline detector doesn't generate proposals. It employs a multi-layer perceptron (MLP) as a classifier to predict category scores for each point. A filter then removes background points, and a clusterer groups the remaining foreground points into distinct instances. This approach is novel because it operates independently of proposal boxes, a common requirement in prior methods.
  • Alignment-Based Point Completion: This module addresses the incompleteness of the identified instances. It first aligns the instance point clouds with pre-generated "prototype" point clouds (representing ideal object shapes) in terms of both their centers and orientations. This alignment is achieved through a Centroid Offset Network and an Angle Network. Subsequently, it uses a completion coefficient (CC) to intelligently sample points from the prototype to fill in missing parts of the instance point cloud. The CC is dynamically calculated based on the ratio of points in the instance to the prototype, ensuring a balance between adding new points and maintaining the original data's characteristics.

The overall framework integrates these modules into a baseline single-stage fully sparse detector. The final loss function combines the baseline detector's loss with losses from the Instance Selection and Alignment-Based Point Completion modules.

Datasets

The proposed method is evaluated on the KITTI dataset, a widely used benchmark for autonomous driving research. Experiments are conducted on both the validation and test sets. Additionally, a subset named "waymo-tiny" is constructed from the Waymo validation set to evaluate the method's performance on a broader range of scenarios.

Key Insights and Ideas

  • Bridging the Gap for Single-Stage Detectors: The most significant insight is the successful adaptation of point cloud completion techniques to the single-stage, fully sparse detection paradigm, which was previously a major hurdle due to the absence of proposal boxes.
  • Shape Priors as a Powerful Tool: Leveraging pre-defined "prototype" shapes of objects proves effective in guiding the completion process, allowing for more realistic and contextually appropriate filling of missing point cloud data.
  • Instance Selection without Proposals: The novel Instance Selection module demonstrates that it's possible to reliably identify object instances in sparse point clouds without relying on traditional bounding box proposals, which is crucial for single-stage detectors.
  • Alignment is Key: Aligning instance point clouds with prototypes in both position and orientation is critical for effective shape completion, ensuring that the added points conform to the object's expected structure.

Key Results

The proposed SPPCC method demonstrates significant performance improvements when integrated with state-of-the-art single-stage fully sparse detectors like FSDV2 and VoxelNeXt.

  • On the KITTI dataset, improvements in 3D Average Precision (AP) are observed across various difficulty levels (easy, moderate, hard) for both validation and test sets. For instance, on the KITTI test set, FSDV2 with SPPCC shows an improvement of up to +0.98% in average 3D AP.
  • The method also shows consistent gains for pedestrian and cyclist categories.
  • Evaluations on the "waymo-tiny" subset further validate the enhanced detection performance for vehicles, pedestrians, and cyclists.
  • Qualitative results (Figure 1 and Figure 7) show that the completion module leads to feature visualizations with higher quality in occluded regions and detection bounding boxes that align more closely with ground truth.

Limitations and Future Work

  • Inference Time: While the method significantly improves accuracy, the ablation studies indicate an increase in inference time when the alignment step is incorporated. The authors note that this increase remains within an acceptable range, but it's a trade-off to consider.
  • Prototype Generation: The generation of high-quality prototype point clouds is crucial. While the paper outlines a method for this, further research into more robust and automated prototype generation could be beneficial.
  • Generalizability: While tested on KITTI and Waymo, exploring the method's effectiveness on datasets with different sensor characteristics or object distributions could be a future direction.

The paper concludes that the proposed SPPCC method offers a good balance between detection accuracy and inference speed, highlighting its potential for real-world autonomous driving applications.

Summary updated

BibTeX
@misc{wang2025shape,
  author = {K. Wang and Mingqian Ji and Jian Yang and Shanshan Zhang},
  title = {{Shape-Prior-Based Point Cloud Completion for Single-Stage Fully Sparse 3d Object Detection}},
  year = {2025},
  doi = {10.2139/ssrn.5079469},
  howpublished = {SSRN},
}
Figure