Temporal Point Cloud Completion With Pose Disturbance

Jieqi Shi, Lingyun Xu, Peiliang Li, Xiaozhi Chen, Shaojie Shen

Abstract

Completing the 3D shape of an object from a partial and noisy point cloud is a fundamental challenge in computer vision. While existing models often assume perfectly aligned and dense inputs, real-world LiDAR data frequently suffers from "pose disturbance"—small but significant errors in rotation and translation. We present a temporal framework that fuses information across multiple frames using Gated Recurrent Units (GRUs) and attention mechanisms. Our approach demonstrates robustness to alignment errors and produces high-fidelity, temporally consistent 3D shapes, outperforming current state-of-the-art methods on both synthetic and real-world datasets.

Introduction

In applications like autonomous driving and robotic navigation, 3D point cloud completion is critical for understanding the environment. However, standard models typically operate under "lab-perfect" conditions, expecting inputs to be centered and oriented correctly. In practice, LiDAR sensors produce sparse data, and object crops from 3D detectors often contain pose disturbances. These misalignments lead to distorted reconstructions when using single-frame methods. This research addresses the need for a model that can accumulate knowledge over time to overcome these real-world sensing limitations.

Methodology

The proposed architecture consists of a two-stage pipeline designed for online processing:

1. Alignment and Coarse Completion

To handle pose disturbance, the model employs a specialized alignment unit. Unlike standard T-Nets that may produce non-orthogonal matrices, this unit explicitly outputs 6D rotations and 3D translations, ensuring geometric validity. This stage aligns the noisy input to a uniform coordinate system and generates an initial coarse shape.

2. Temporal Fusion and Refinement

The core of the framework is its temporal fusion capability. A Global Fusion module uses a GRU to maintain a "long-term memory" of the object's global structure across a sequence. Simultaneously, a Local Fusion module utilizes a Squeeze-and-Excitation (SE) attention block within a sliding window to incorporate fine-grained details from recent views. Finally, a Graph Convolutional Network (GCN) refines the coarse shape into a detailed final configuration.

The proposed two-stage architecture featuring pose alignment and temporal fusion units.

Experimental Results

The model was evaluated on Augmented ShapeNet (with introduced pose noise), the PCN dataset, and real-world KITTI data.

Quantitative Performance

On the Augmented ShapeNet dataset, our model achieved a Chamfer Distance (CD) of 6.64, a significant improvement over the previous state-of-the-art, GCC (9.19), and PCN (14.28). The results indicate that the model's performance improves as it processes more frames, with the CD dropping from 6.64 at the first frame to 5.26 by the tenth frame.

Quantitative comparison of Chamfer Distance (CD) across various object categories on the Augmented ShapeNet dataset.

Real-World Consistency and Efficiency

Evaluations on the KITTI dataset highlighted the model's superior temporal consistency compared to single-frame methods. While other models produced fluctuating shapes for the same object across frames, our approach maintained a stable reconstruction. Furthermore, the model operates at approximately 10Hz, making it suitable for real-time integration into autonomous systems.

Conclusion

By moving beyond the single-frame paradigm and explicitly addressing pose disturbance, this work provides a robust solution for real-world 3D point cloud completion. The integration of 6D rotation alignment and temporal fusion allows for high-quality shape reconstruction from sparse, noisy sequences. Future work will explore end-to-end integration with tracking systems and extending the framework to dynamic, deforming objects.

Summary updated

BibTeX
@article{shi2022temporal,
  author = {Jieqi Shi and Lingyun Xu and Peiliang Li and Xiaozhi Chen and Shaojie Shen},
  title = {{Temporal Point Cloud Completion With Pose Disturbance}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2022},
  url = {https://openreview.net/forum?id=PRiuJhMwWRz},
}
Figure