WalkFormer: Point Cloud Completion via Guided Walks

Mohang Zhang, Yushi Li, Rong Chen, Yushan Pan, Jia Wang, Yunzhe Wang, Rong Xiang2024 · DOI

TL;DR

WalkFormer tackles the challenge of reconstructing complete 3D point clouds from sparse, incomplete data by introducing a novel "guided walk" approach. Instead of moving all points, it selectively moves sampled points along learned paths, preserving existing structures while effectively filling in missing regions.

What Problem Does This Paper Address?

3D point clouds, generated by sensors like LiDAR and depth cameras, are crucial for understanding real-world environments and objects. However, they are often incomplete due to occlusions, sensor limitations, or object properties. This incompleteness hinders downstream tasks like object recognition, scene reconstruction, and virtual reality. Existing methods often rely on encoding the entire point cloud into a global feature vector and then decoding it, which can lead to a loss of fine-grained details and existing structures. Deformation-based methods, while promising, can sometimes struggle with preserving the integrity of the original points. This paper aims to improve point cloud completion by developing a method that can accurately reconstruct missing parts while robustly preserving the original shape.

Methods

WalkFormer employs a multi-step partial deformation approach. The core innovations are:

  • Neighbour Similarity Sampling: Instead of traditional sampling methods that might pick points from sparse outer regions, this technique selects starting points based on feature similarity within local neighborhoods. This ensures that the chosen points are representative of the existing structures, aiding in detail preservation.
  • Point Walk Module: This module generates "guided walks" through the point cloud's feature space. These walks are sequences of points that capture topological relationships and long-range dependencies, providing rich contextual information for deformation.
  • Route Transformer: This novel transformer module aggregates information from these guided walks across deformation steps. It incorporates previous displacement information to guide the current step, ensuring a more consistent and accurate moving path for the selected points.
  • Partial Deformation: Unlike methods that move all points at each step, WalkFormer only moves a subset of sampled points. This selective movement helps preserve the integrity of the original point cloud.

The overall process involves an encoder to extract per-point features, followed by multiple deformation steps where the Point Walk and Route Transformer modules guide the prediction of displacement vectors for the sampled points.

Datasets

The paper evaluates WalkFormer on the following datasets:

  • Completion3D: A widely-used benchmark for point cloud completion, containing 28,974 training, 800 validation, and 1,200 test models across 8 categories. Partial point clouds are generated by back-projecting 2.5D depth images.
  • PCN Dataset: Derived from ShapeNet, this dataset simulates incomplete point clouds by using 8 viewpoints. It's noted that WalkFormer is trained on lower-resolution inputs (2048 points) but tested on higher resolutions (16384 points), demonstrating generalization.
  • KITTI Dataset: This dataset consists of real-world sparse point clouds from LiDAR scans. Due to the absence of ground truth, Fidelity Distance (FD) and Minimal Matching Distance (MMD) are used for evaluation.

Key Insights and Ideas

  • Preserving Structure is Key: The paper strongly emphasizes that simply decoding from a global feature vector is insufficient. Maintaining the integrity of existing points is crucial for high-quality completion.
  • Guided Walks for Context: The "guided walk" concept is a clever way to encode long-range spatial and topological relationships. This allows the model to "understand" the context of missing regions better than purely local methods.
  • Selective Deformation is More Robust: Moving only a subset of points, guided by learned walks, proves more effective than moving all points, preventing the destruction of existing geometry.
  • Topology Matters: The Route Transformer's ability to leverage topological relationships from guided walks is vital for accurate and consistent deformation prediction.

Key Results

WalkFormer achieves state-of-the-art performance on the Completion3D dataset, outperforming previous methods in both Chamfer Distance (CD) and Earth Mover's Distance (EMD) metrics. For instance, it reduces average CD by 0.38 and average EMD by 0.11 compared to the second-best method, SeedFormer. On the PCN dataset, WalkFormer demonstrates strong generalization by being trained on lower-resolution inputs and achieving competitive results. Qualitative results show that WalkFormer can generate more complex topology structures and produce abundant points in missing areas, such as airplane wings, while preserving existing structures.

Limitations and Future Work

  • Sampling Strategy: While Neighbour Similarity Sampling is an improvement, the paper notes that reducing the number of sampled points (Nm) can sometimes lead to better performance, suggesting that a very dense set of moving points might not always be optimal. Future work could explore more sophisticated sampling strategies.
  • Computational Cost: Deformation-based methods, even with partial movement, can still be computationally intensive. While not explicitly detailed as a limitation, optimizing the efficiency of the walk generation and transformer modules could be a future direction.
  • Generalization to Extreme Sparsity: The paper demonstrates good performance, but extreme sparsity (very few input points) might still pose a significant challenge for any completion method. Further investigation into performance under such conditions would be valuable.

Summary updated

Abstract

Point clouds are often sparse and incomplete in real-world scenarios. The prevailing methods for point cloud completion typically rely on encoding the partial points and then decoding complete points from a global feature vector, which might lose the existing patterns and elaborate structures. To address these issues, we propose WalkFormer, a novel approach to predict complete point clouds through a partial deformation process. Concretely, our method samples locally dominant points based on feature similarity and moves the points to form the missing part. Since these points maintain representative information of the surrounding structures, they are appropriately selected as the starting points for multiple guided walks. Furthermore, we design a Route Transformer module to exploit and aggregate the walk information with topological relations. These guided walks facilitate the learning of long-range dependencies for predicting shape deformation. Qualitative and quantitative evaluations demonstrate that our proposed approach achieves superior performance compared to state-of-the-art methods in the 3D point cloud completion task.
BibTeX
@misc{zhang2024walkformer,
  author = {Mohang Zhang and Yushi Li and Rong Chen and Yushan Pan and Jia Wang and Yunzhe Wang and Rong Xiang},
  title = {{WalkFormer: Point Cloud Completion via Guided Walks}},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00326},
}
Figure