Abstract
Completing the 3D shape of an object from a partial and noisy point cloud is a fundamental challenge in computer vision. While existing models often assume perfectly aligned and dense inputs, real-world LiDAR data frequently suffers from "pose disturbance"—small but significant errors in rotation and translation. We present a temporal framework that fuses information across multiple frames using Gated Recurrent Units (GRUs) and attention mechanisms. Our approach demonstrates robustness to alignment errors and produces high-fidelity, temporally consistent 3D shapes, outperforming current state-of-the-art methods on both synthetic and real-world datasets.
Introduction
In applications like autonomous driving and robotic navigation, 3D point cloud completion is critical for understanding the environment. However, standard models typically operate under "lab-perfect" conditions, expecting inputs to be centered and oriented correctly. In practice, LiDAR sensors produce sparse data, and object crops from 3D detectors often contain pose disturbances. These misalignments lead to distorted reconstructions when using single-frame methods. This research addresses the need for a model that can accumulate knowledge over time to overcome these real-world sensing limitations.
Methodology
The proposed architecture consists of a two-stage pipeline designed for online processing:
1. Alignment and Coarse Completion
To handle pose disturbance, the model employs a specialized alignment unit. Unlike standard T-Nets that may produce non-orthogonal matrices, this unit explicitly outputs 6D rotations and 3D translations, ensuring geometric validity. This stage aligns the noisy input to a uniform coordinate system and generates an initial coarse shape.
2. Temporal Fusion and Refinement
The core of the framework is its temporal fusion capability. A Global Fusion module uses a GRU to maintain a "long-term memory" of the object's global structure across a sequence. Simultaneously, a Local Fusion module utilizes a Squeeze-and-Excitation (SE) attention block within a sliding window to incorporate fine-grained details from recent views. Finally, a Graph Convolutional Network (GCN) refines the coarse shape into a detailed final configuration.
Experimental Results
The model was evaluated on Augmented ShapeNet (with introduced pose noise), the PCN dataset, and real-world KITTI data.
Quantitative Performance
On the Augmented ShapeNet dataset, our model achieved a Chamfer Distance (CD) of 6.64, a significant improvement over the previous state-of-the-art, GCC (9.19), and PCN (14.28). The results indicate that the model's performance improves as it processes more frames, with the CD dropping from 6.64 at the first frame to 5.26 by the tenth frame.
Real-World Consistency and Efficiency
Evaluations on the KITTI dataset highlighted the model's superior temporal consistency compared to single-frame methods. While other models produced fluctuating shapes for the same object across frames, our approach maintained a stable reconstruction. Furthermore, the model operates at approximately 10Hz, making it suitable for real-time integration into autonomous systems.
Conclusion
By moving beyond the single-frame paradigm and explicitly addressing pose disturbance, this work provides a robust solution for real-world 3D point cloud completion. The integration of 6D rotation alignment and temporal fusion allows for high-quality shape reconstruction from sparse, noisy sequences. Future work will explore end-to-end integration with tracking systems and extending the framework to dynamic, deforming objects.
Summary updated