Iterative Feedback Network for Unsupervised Point Cloud Registration

Yifan Xie, Boyu Wang, Shiqi Li, Jihua Zhu

TL;DR

This paper introduces IFNet, a novel iterative feedback network for unsupervised 3D point cloud registration. By incorporating a feedback mechanism that allows high-level features to guide low-level feature learning across time steps, IFNet significantly improves registration accuracy, especially in challenging scenarios like low overlap and noisy data.

What Problem Does This Paper Address?

Aligning 3D point clouds, known as point cloud registration, is crucial for applications like 3D reconstruction, augmented reality, and autonomous driving. Traditional methods like ICP struggle with noisy data, low overlap, and sensitivity to initial alignment. While deep learning has shown promise, collecting ground truth transformations for training is expensive. Existing learning-based methods are typically feedforward, meaning information only flows in one direction. This limits their ability to leverage richer, more abstract information from later stages of processing to refine earlier, more detailed feature representations. The authors hypothesize that, similar to how higher brain areas inform lower ones, high-level features from later iterations could guide low-level feature learning in earlier ones, leading to more robust and accurate registration.

Methods

The core of IFNet is the Iterative Feedback Network (IFNet), which uses multiple Feedback Registration Block (FRB) modules. These FRBs are cascaded and unfolded over time, creating an iterative process.

  • Feedback Registration Block (FRB): Each FRB takes the output from the previous FRB (or initial input) and refines the rigid transformation (rotation and translation). Crucially, it also generates feedback information.
  • Iterative Feedback Mechanism: Information flows both forward (feedforward) and backward (feedback) across time steps. High-level features from a later time step are fed back to refine low-level features in an earlier time step. This allows the network to progressively improve its understanding of the point cloud geometry.
  • Feedback Transformer: This component is key to effectively integrating the high-level feedback information. It adaptively selects and enhances relevant information from the feedback features to refine the low-level source point cloud features.
  • Geometry-Aware Descriptor: To better utilize geometric information, the authors propose a descriptor that concatenates edge information, edge lengths, and normals derived from k-nearest neighbors. This descriptor is used as a positional embedding within the Feedback Transformer, enriching the network's understanding of local geometry.
  • Loss Functions: IFNet is trained using a combination of:
  • Global Registration Loss (Lgr): A distance-based loss to align the transformed source point cloud with the target.
  • Neighborhood Consistency Loss (Lnc): Encourages consistency between transformed source points and their neighbors in the target point cloud.
  • Pseudo Consistency Loss (Lpc): Uses cross-entropy to enforce spatial consistency in pseudo-correspondences.

Datasets

IFNet was evaluated on several standard benchmarks:

  • ModelNet40: A large dataset of 3D CAD models used for training and testing, including experiments on "same" categories, "unseen" categories, and with added noise.
  • 7Scenes: An indoor dataset used for evaluating performance in real-world indoor environments.
  • ICL-NUIM: Another indoor dataset, known for its synchronized RGB-D sequences, used to assess performance in various indoor scenes.
  • KITTI: An outdoor dataset commonly used for autonomous driving tasks, featuring LIDAR scans.

Key Insights and Ideas

  • Feedback is Crucial for Point Cloud Registration: The paper strongly suggests that a top-down feedback mechanism, inspired by biological vision systems, can significantly enhance the performance of point cloud registration by allowing high-level semantic understanding to guide low-level geometric feature extraction.
  • Unsupervised Learning Benefits from Iterative Refinement: By unfolding the FRB modules over time, IFNet achieves progressive refinement, leading to more accurate results without requiring ground truth transformations for every iteration.
  • Geometry-Aware Descriptors Enhance Feature Representation: Explicitly encoding geometric information like edge lengths and normals into a descriptor helps the network better understand local structures, leading to improved registration.
  • The Feedback Transformer is Key to Effective Information Integration: Simply concatenating features isn't enough; the Feedback Transformer's ability to selectively attend to and fuse high-level feedback with low-level features is critical for IFNet's success.

Key Results

  • Superior Performance: IFNet consistently outperforms state-of-the-art traditional, supervised, and unsupervised methods across various datasets (ModelNet40, 7Scenes, ICL-NUIM, KITTI) in terms of RMSE and MAE for both rotation and translation.
  • Robustness: The method demonstrates strong robustness to varying numbers of points and different levels of noise.
  • Efficiency: While employing an iterative feedback mechanism, IFNet achieves competitive inference times (e.g., 146ms for IFNet vs. 62ms for RIENet, 295ms for CEMNet on ModelNet40).
  • Low Overlap Performance: IFNet shows significant advantages in low-overlap scenarios compared to methods like RIENet.

Limitations and Future Work

  • Computational Cost: While efficient, the iterative nature of IFNet inherently increases computation compared to single-pass methods. The paper notes that time efficiency is a consideration, and the chosen settings (s=3, t=3) represent a balance between performance and efficiency.
  • Generalization to Extreme Scenarios: While robust to noise and varying points, further exploration into extreme cases like severe occlusions or drastically different point densities could be beneficial.
  • Future Work: The authors suggest exploring more advanced attention mechanisms within the Feedback Transformer and investigating the application of this feedback paradigm to other 3D vision tasks. The current work focuses on rigid registration; extending it to non-rigid registration could be a natural next step.

Summary updated

Abstract

As a fundamental problem in computer vision, point cloud registration aims to seek the optimal transformation for aligning a pair of point clouds. In most existing methods, the information flows are usually forward transferring, thus lacking the guidance from high-level information to low-level information. Besides, excessive high-level information may be overly redundant, and directly using it may conflict with the original low-level information. In this letter, we propose a novel Iterative Feedback Network (IFNet) for unsupervised point cloud registration, in which the representation of low-level features is efficiently enriched by rerouting subsequent high-level features. Specifically, our IFNet is built upon a series of Feedback Registration Block (FRB) modules, with each module responsible for generating the feedforward rigid transformation and feedback high-level features. These FRB modules are cascaded and recurrently unfolded over time. Further, the Feedback Transformer is designed to efficiently select relevant information from feedback high-level features, which is utilized to refine the low-level features. What's more, we incorporate a geometry-awareness descriptor to empower the network for making full use of most geometric information, which leads to more precise registration results. Extensive experiments on various benchmark datasets demonstrate the superior registration performance of our IFNet.
BibTeX
@article{xie2024iterative,
  author = {Yifan Xie and Boyu Wang and Shiqi Li and Jihua Zhu},
  title = {{Iterative Feedback Network for Unsupervised Point Cloud Registration}},
  journal = {IEEE Robotics and Automation Letters},
  year = {2024},
  doi = {10.1109/lra.2024.3355784},
}
Figure