Top-I2P: Explore Open-Domain Image-to-Point Cloud Registration Using Topology Relationship

Pei An, Jiaqi Yang, Muyao Peng, You Yang, Qiong Liu, Jie Ma, Liangliang NanIJCAI 2025 · DOI

Introduction

Image-to-Point Cloud (I2P) registration is a fundamental task in computer vision, essential for visual localization, augmented reality, and autonomous navigation. It involves determining the 6-degree-of-freedom (6-DoF) camera pose by matching pixels in a 2D image to points in a 3D point cloud. Despite recent advances, existing deep learning models face two critical hurdles: poor generalization to unseen "open-domain" environments and extreme computational costs associated with dense pixel-to-point matching.

The Challenge of Open-Domain Registration

Most current I2P registration models overfit to the specific textures and geometries of their training datasets. When deployed in a new environment—an "open-domain" scenario—their performance often degrades significantly. Furthermore, achieving high-precision registration typically requires pixel-level cross-attention. For a standard image and a 100k-point cloud, this process generates a matching matrix with approximately $10^{10}$ elements, exceeding the memory and processing capabilities of modern GPUs.

Methodology: Leveraging Topology

To address these issues, Top-I2P introduces a framework that prioritizes scene topology over raw visual features. Topology—the spatial relationships and intersections between patches—remains more consistent across different environments than textures like color or lighting.

The Top-I2P framework: (a) Inspiration from SAM-based segmentation; (b) Pipeline involving topology graph construction, reasoning, and feature interaction.

The Top-I2P pipeline consists of three core stages:

  1. Topology Graph Construction: Utilizing the Segment Anything Model (SAM), the system partitions both the 2D image and 3D point cloud into semantic patches. These patches serve as nodes in topology graphs ($G_I$ and $G_P$).
  2. Correction-based Topology Reasoning (CTR): To bridge the modality gap between 2D and 3D data, the model uses Graph Convolutional Networks (GCNs) and a Sinkhorn layer to predict overlapping patches. This identifies sparse, meaningful connections between the two spaces.
  3. Fast Topology Feature Interaction (FTI): By performing patch-to-patch interaction rather than dense pixel-to-point matching, Top-I2P reduces the computational burden by a factor of $10^6$. Information is only exchanged between topologically related patches, ensuring both efficiency and accuracy.

Experimental Results

The authors evaluated Top-I2P on several benchmarks, including 7-Scenes, ScanNet, and a self-collected real-world dataset. The model was specifically tested in "open-domain" settings, where it was trained on one scene (e.g., a kitchen) and tested on entirely different ones (e.g., stairs or offices).

Comparison of open-domain registration performance. Top-I2P achieves a significantly higher inlier ratio (IR) compared to 2D3D-MATR in unseen scenes.

Top-I2P demonstrates superior performance compared to state-of-the-art (SOTA) methods like 2D3D-MATR and FreeReg. On the ScanNet dataset, Top-I2P achieved a Registration Recall (RR) of 0.901, significantly higher than FreeReg (0.780). Beyond accuracy, the model is highly efficient, running at approximately 5.6 FPS with manageable memory overhead, making it viable for real-time applications.

Conclusion

Top-I2P represents a significant step forward in robust I2P registration. By focusing on topology as a scene-independent cue and leveraging foundational models like SAM, it overcomes the generalization and efficiency barriers of previous methods. While challenges remain in textureless or extremely low-resolution environments, the shift toward topological reasoning provides a more reliable foundation for downstream tasks in general scenes.

Summary updated

BibTeX
@inproceedings{an2025top,
  author = {Pei An and Jiaqi Yang and Muyao Peng and You Yang and Qiong Liu and Jie Ma and Liangliang Nan},
  title = {{Top-I2P: Explore Open-Domain Image-to-Point Cloud Registration Using Topology Relationship}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/76},
}
Figure