ASFM-Net: Asymmetrical Siamese Feature Matching Network for Point Completion

Yaqi Xia, Yan Xia, Wei Li, Rui Song, Kailang Cao, Uwe Stilla

Abstract

We tackle the problem of object completion from point clouds and propose a novel point cloud completion network employing an Asymmetrical Siamese Feature Matching strategy, termed as ASFM-Net. Specifically, the Siamese auto-encoder neural network is adopted to map the partial and complete input point cloud into a shared latent space, which can capture detailed shape prior. Then we design an iterative refinement unit to generate complete shapes with fine-grained details by integrating prior information. Experiments are conducted on the PCN dataset and the Completion3D benchmark, demonstrating the state-of-the-art performance of the proposed ASFM-Net. Our method achieves the 1st place in the leaderboard of Completion3D and outperforms existing methods with a large margin, about 12%.

Introduction

In real-world 3D scanning scenarios using LiDAR or depth cameras, captured point clouds are almost always incomplete due to self-occlusion or environmental obstructions. Inferring the complete 3D structure from these partial inputs is crucial for downstream tasks like autonomous driving and robotics. Most existing methods attempt to extract global features directly from partial inputs, which often lack the necessary geometric details for accurate reconstruction. ASFM-Net addresses this by leveraging "shape priors" to guide the completion process.

Methodology

ASFM-Net utilizes a coarse-to-fine architecture consisting of two main components: an Asymmetrical Siamese Auto-encoder and an Iterative Refinement Unit.

Asymmetrical Siamese Auto-encoder

This module is designed to learn global shape priors. It uses two identical auto-encoders. First, $AE_1$ is pre-trained on complete point clouds to map them into a high-dimensional latent space (a codeword), and its weights are then frozen. Next, $AE_2$ processes partial point clouds. During training, $AE_2$'s encoder is forced to generate a codeword that matches $AE_1$'s codeword using a Euclidean distance feature matching loss. This forces the network to map incomplete data into a structured dictionary of valid 3D shapes.

The Asymmetrical Siamese Auto-encoder architecture showing the feature matching between complete and partial point cloud latent spaces.

Iterative Refinement Unit

Because the Siamese auto-encoder focuses on global structure, it may lose fine local details. The refinement unit concatenates the original partial input with the coarse output (incorporating XY-plane mirror symmetry) to form a synthetic point cloud. It then uses a 2D grid generator and Multi-Layer Perceptrons (MLPs) to predict coordinate offsets, shifting points to reconstruct fine-grained geometric details.

Experimental Results

ASFM-Net was evaluated on standard 3D benchmarks including the PCN dataset and the Completion3D benchmark.

Quantitative Performance

ASFM-Net achieved 1st place on the Completion3D public leaderboard with an average Chamfer Distance (CD-T) of 6.68, outperforming the second-best method by a significant 17.7% margin. On the PCN dataset, it achieved state-of-the-art results across all categories, improving the average Chamfer Distance by 6.1% compared to previous leaders.

Qualitative Analysis

Visual results demonstrate that ASFM-Net produces significantly sharper and more complete reconstructions compared to baselines like PCN or TopNet, which often produce blurry or incomplete structures.

Qualitative comparison of ASFM-Net against other methods on the PCN dataset across various object categories.

Robustness and Generalization

The model exhibits extreme robustness, successfully reconstructing plausible shapes even when only 20% of the original object is visible. Furthermore, tests on real-world KITTI LiDAR data show a fidelity error of 0.025, significantly better than the 0.041 achieved by TopNet and PCN, proving the model's ability to generalize from synthetic training to real-world sensor data.

Conclusion

ASFM-Net introduces a novel paradigm for point cloud completion by using latent space matching to inject shape priors into the reconstruction process. By decoupling global structure from local refinement, the network achieves superior accuracy and robustness. Future work will explore the utility of these high-fidelity completions in improving 3D object detection and classification tasks.

Summary updated

Abstract

We tackle the problem of object completion from point clouds and propose a novel point cloud completion network employing an Asymmetrical Siamese Feature Matching strategy, termed as ASFM-Net. Specifically, the Siamese auto-encoder neural network is adopted to map the partial and complete input point cloud into a shared latent space, which can capture detailed shape prior. Then we design an iterative refinement unit to generate complete shapes with fine-grained details by integrating prior information. Experiments are conducted on the PCN dataset and the Completion3D benchmark, demonstrating the state-of-the-art performance of the proposed ASFM-Net. Our method achieves the 1st place in the leaderboard of Completion3D and outperforms existing methods with a large margin, about 12%. The codes and trained models are released publicly at https://github.com/Yan-Xia/ASFM-Net.
BibTeX
@inproceedings{xia2021asfm,
  author = {Yaqi Xia and Yan Xia and Wei Li and Rui Song and Kailang Cao and Uwe Stilla},
  title = {{ASFM-Net: Asymmetrical Siamese Feature Matching Network for Point Completion}},
  booktitle = {ACM Multimedia},
  year = {2021},
  url = {https://openreview.net/forum?id=T7hJZg0ayO},
}
Figure