TL;DR
This paper introduces RGM, a novel deep graph matching framework for robust 3D point cloud registration. By leveraging graph structures and a transformer-based edge generator, RGM significantly improves correspondence accuracy, especially in the presence of outliers and partial point clouds, achieving state-of-the-art performance.
What Problem Does This Paper Address?
3D point cloud registration, the task of aligning two 3D point clouds, is a fundamental problem in computer vision and robotics with applications in areas like autonomous driving and surgical navigation. A key challenge in this task is accurately finding correspondences between points in the two clouds, especially when dealing with noisy data, outliers (points with no true match), or partial overlaps between the clouds. Existing learning-based methods, while fast, often struggle with outliers because they rely solely on local point features, which can be ambiguous. This paper aims to address this limitation by developing a more robust registration framework that considers both local geometry and the broader structural context of the point clouds.
Methods
The proposed framework, RGM (Robust Graph Matching), consists of several key components:
- Local Feature Extractor: Similar to existing methods, RGM first extracts local geometric features for each point using its K-nearest neighbors. These features are then mapped to a high-dimensional space.
- Edge Generator (Transformer-based): This is a novel component. Instead of relying on fixed graph structures (like full connections or Delaunay triangulations), RGM uses a transformer-based module to dynamically learn "soft" graph edges. This allows the model to capture more relevant structural relationships between points, especially for partial-to-partial registration where attention mechanisms can effectively identify overlapping regions.
- Graph Feature Extractor & AIS Module: The extracted local features become node features in a graph. A graph feature extractor then aggregates information from neighboring nodes using intra-graph convolutions to refine these features. The core of the correspondence finding is the AIS (Affinity Layer, Instance Normalization, and Sinkhorn) module. It computes an affinity matrix between nodes of the two graphs and uses instance normalization and the Sinkhorn algorithm to derive a soft correspondence matrix. This approach considers not just local features but also the graph's structure and topology, making it more robust to outliers.
- LAP Solver and SVD: During testing, the soft correspondences are converted into hard, one-to-one correspondences using a Linear Assignment Problem (LAP) solver (based on the Hungarian algorithm). Finally, Singular Value Decomposition (SVD) is used to compute the rigid transformation. The framework also supports iterative refinement, similar to ICP.
Datasets
The experiments were conducted on the ModelNet40 dataset. This dataset contains 12,311 meshed CAD models from 40 categories. For evaluation, the authors used:
- Clean Point Clouds: Generated by sampling points from meshes and applying random transformations.
- Noisy Point Clouds: Created by adding Gaussian noise to clean point clouds.
- Partial-to-Partial Point Clouds: Generated by retaining only 70% of points after applying a random plane cut.
- Unseen Categories: The model was tested on categories not seen during training to evaluate generalization capabilities.
Key Insights and Ideas
- Graph Matching for Robustness: The core insight is that by representing point clouds as graphs and employing graph matching, RGM can leverage higher-order structural information beyond local geometry. This is crucial for distinguishing true correspondences from outliers.
- Transformer for Dynamic Edge Generation: Using a transformer to learn graph edges is a novel approach that allows the model to adaptively build relevant connections, particularly beneficial for challenging partial-to-partial scenarios where identifying overlapping regions is key.
- AIS Module for Accurate Correspondences: The AIS module's combination of affinity calculation, instance normalization, and Sinkhorn iteration effectively handles outliers and produces more reliable soft correspondences compared to simpler methods.
- End-to-End Training with Correspondence Loss: Training directly on the ground-truth correspondences (using cross-entropy loss) allows the network to learn features that are directly optimized for finding accurate matches, rather than indirectly through transformation parameters.
Key Results
RGM achieved state-of-the-art performance across various challenging scenarios:
- Clean Point Clouds: Achieved 100% success rate with minimal error metrics (MIE(R) < 0.001°, MIE(t) < 0.00001).
- Noisy Point Clouds: Demonstrated strong robustness, with recall close to 100% and significantly lower errors compared to other methods.
- Partial-to-Partial Point Clouds: Outperformed other methods with a success rate higher than 90%, showing its effectiveness in handling incomplete data.
- Unseen Categories: Showed superior generalization capabilities compared to other learning-based methods.
Limitations and Future Work
- Computational Cost: While faster than traditional methods, deep learning approaches can still be computationally intensive. The paper mentions that inference time comparisons are in the supplementary material, suggesting it's a factor.
- Generalization to Deformable Registration: The authors suggest that the deep graph matching approach has potential for other registration problems, including 2D-3D and deformable registration, indicating future research directions.
- Ablation Studies: The paper includes ablation studies that highlight the importance of the AIS module and the transformer-based edge generator, confirming their significant contributions to the overall performance. The variant without the AIS module (RGMVar1) showed a drastic drop in accuracy, and the variant with full connection edges (RGMVar2) also performed worse than the proposed transformer-based edge generator.
Summary updated