Abstract
In autonomous driving, LiDAR sensors often produce incomplete point clouds due to self-occlusion, external obstacles, and limited sensor resolution. While existing point cloud completion models show promise, they typically rely on synthetic datasets like ShapeNet, resulting in a significant domain gap when applied to real-world traffic scenes. This paper introduces TraPCC (Traffic Point Cloud Completion), a self-supervised framework designed to complete missing geometries in real-world LiDAR scans without the need for synthetic CAD models. By leveraging temporal data aggregation and a scene-aware bottom-up mechanism, TraPCC effectively bridges the gap between synthetic training and messy, sparse real-world data.
Introduction
LiDAR sensors serve as the primary "eyes" for autonomous vehicles, yet they are fundamentally limited by line-of-sight constraints. Vehicles often appear as "hollow" shells because the sensor cannot see through the car's own body or through other objects in the environment. While deep learning has advanced the field of point cloud completion, most models struggle with the specific patterns of real-world scans, such as ring sweeps and varying sparsity, because they are trained on perfectly clean synthetic data.
Furthermore, obtaining ground truth (GT) data for moving vehicles in the real world is a significant challenge. TraPCC addresses these issues by avoiding synthetic data entirely, instead utilizing the temporal and spatial context inherent in traffic scenes to supervise the learning process.
Methodology
TraPCC employs three primary strategies to achieve high-fidelity completion in a self-supervised manner:
1. Self-Supervised Ground Truth (GT) Generation
Rather than relying on CAD models, the framework constructs a "GT-Pool" by aggregating consecutive frames of the same vehicle as it moves through a scene. By combining multiple viewpoints over time and applying vehicle symmetry priors, the system creates relatively complete real-world scans. These aggregated scans serve as the training targets, providing a natural supervision signal derived from the environment itself.
2. Bottom-Up Mechanism
The network architecture is divided into two specialized stages:
- P-Net (Partial Network): Focuses on local geometry by predicting dense points for partial bodies, such as the front or back of a vehicle.
- C-Net (Completion Network): Takes the local features from P-Net and refines them into a global shape. This ensures that the final output respects both fine-grained local details and overall vehicle proportions.
3. Scene-Graph Integration and Loss Functions
Vehicles in traffic do not exist in isolation. TraPCC utilizes a scene-graph to incorporate features from neighboring vehicles, which often share similar viewpoints or cause mutual occlusions. To handle the inherent sparsity of the data, the authors introduced the Modified Chamfer Distance (MCD) Loss. This one-way distance metric ensures the predicted dense cloud accurately covers the sparse input points without being unfairly penalized for the input's lack of density.
Experimental Results
Quantitative Performance
The researchers evaluated TraPCC on two major autonomous driving datasets: KITTI Tracking and nuScenes. On the KITTI dataset, TraPCC achieved a mean Chamfer Distance (meanCD) of 0.0973, demonstrating high fidelity to the reconstructed ground truth.
Boosting 3D Detection
One of the most significant findings is that TraPCC can serve as a post-processing filter for existing 3D detectors. When integrated with popular detectors like SECOND and PointPillars, TraPCC improved detection precision, particularly in "Hard" scenarios involving highly occluded or distant objects. For instance, the "Hard" detection accuracy for the SECOND detector improved from 65.91% to 65.98% on the KITTI benchmark.
Cross-Dataset Robustness
TraPCC demonstrated remarkable generalization capabilities. Models trained on the nuScenes dataset and tested on KITTI (and vice versa) maintained high performance levels. This suggests that the method is less sensitive to specific sensor "fingerprints"—such as the number of LiDAR beams—compared to supervised methods that rely on synthetic-to-real domain adaptation.
Discussion and Limitations
While TraPCC significantly advances real-world point cloud completion, it faces a "sparsity floor." If a vehicle is extremely distant or almost entirely occluded, the input data may be too sparse for the model to generate a reliable completion. Additionally, the quality of the GT-Pool depends on the accuracy of the temporal aggregation.
Future work will focus on two areas: refining the GT-Pool generation through advanced optimization techniques and deeper temporal integration. By incorporating temporal features directly into the neural network's inference process, the model could potentially "remember" vehicle parts seen in previous frames to improve real-time completion.
Conclusion
TraPCC provides a robust, self-supervised solution for the domain gap in point cloud completion. By utilizing temporal memory and scene context, it transforms sparse, occluded LiDAR scans into complete 3D representations, ultimately enhancing the reliability of 3D object detection in complex traffic environments.
Summary updated