Indoor Visual Localization Using Point and Line Correspondences in Dense Colored Point Cloud

Yuya Matsumoto, Gaku Nakano, Kazumine OguraWACV 2024

TL;DR

This paper introduces Loc-PL, a novel pipeline for indoor visual localization that leverages both point and line correspondences. By combining these complementary features, Loc-PL achieves robust localization in challenging indoor environments with repetitive patterns and texture-less areas, outperforming existing state-of-the-art methods on a large-scale dataset.

What Problem Does This Paper Address?

Visual localization, the task of estimating a camera's 6-DOF pose within a pre-built 3D map, is crucial for applications like augmented reality and robotics. However, indoor environments present unique challenges compared to outdoor ones. These include the prevalence of texture-less surfaces (making it hard to detect reliable keypoints) and repetitive patterns (leading to ambiguous matches and false correspondences). Existing point-based localization methods often struggle with these issues, leading to unstable or inaccurate pose estimations. While line features offer a potential solution due to their presence even in texture-less scenes, effectively integrating them with point features for robust indoor localization remains an open problem.

Methods

The proposed Loc-PL pipeline employs a coarse-to-fine strategy with two main camera pose estimation modules, both utilizing both points and lines.

  • Pose Estimation Module: This module aims to make localization robust to repetitive patterns. It incorporates geometric consistency between points and lines. Crucially, it uses a RANSAC-based approach to find line correspondences, which is effective for image pairs with larger viewpoint differences. This module also refines camera poses using PnP (Perspective-n-Point) with both point and line inliers.
  • Re-localization Module: This module refines the pose estimate and circumvents unstable localization caused by locally concentrated matches. It obtains line correspondences using rendered images from a dense point cloud and employs a Perspective-m-Point-n-Line (PmPnL) approach for stable pose refinement.

A key innovation is the use of different strategies for obtaining line correspondences in each module: RANSAC for the initial, potentially wider viewpoint gap in the first module, and feature line matching for the closer viewpoints in the re-localization module. Additionally, the authors developed a simple yet effective module for evaluating the correctness of camera poses based on matched point distances.

Datasets

The experiments were conducted on the InLoc dataset. This is a large-scale indoor dataset covering a 25,287m² area across five floors. It provides 10K images, with each pixel associated with a corresponding 3D point from a LiDAR sensor. The dataset includes 329 query images captured by an iPhone 7. The paper notes that InLoc is particularly suitable for this research because it is one of the few datasets that provides dense point clouds, which are necessary for rendering virtual viewpoints.

Key Insights and Ideas

  • Complementarity is Key: The core idea is that points and lines are spatially complementary. Points are good for texture-rich areas, while lines are robust in texture-less or repetitive environments. Combining them provides a more complete geometric representation.
  • Different Line Strategies for Different Stages: Using a RANSAC-based approach for line matching in the initial pose estimation (where viewpoint gaps can be large) and a feature-line matching approach in the re-localization step (where viewpoints are closer) is a smart way to leverage the strengths of each method.
  • RANSAC for Robustness: The RANSAC-based line correspondence search, combined with point inliers, provides a robust score that is less susceptible to false matches in repetitive patterns. The score is designed to penalize scenarios with many point inliers but few line inliers, indicating a potential mismatch.
  • PmPnL for Stable Refinement: The use of Perspective-m-Point-n-Line (PmPnL) in the re-localization step, which jointly optimizes points and lines, leads to more stable and accurate pose refinement, especially in areas where point-based methods might fail.

Key Results

The Loc-PL pipeline achieved state-of-the-art (SOTA) performance on the InLoc dataset, excelling in four out of six evaluation metrics. Specifically, for the most stringent accuracy criterion (< 0.25 meters), Loc-PL showed significant improvements over existing methods. Ablation studies demonstrated the effectiveness of each component:

  • Including line correspondences in the pose estimation module improved scores, particularly for less strict error thresholds (< 0.5m, < 1.0m).
  • The re-localization step using PmPnL with both points and lines tended to increase scores for stricter error thresholds (< 0.25m, < 0.50m).
  • The RANSAC-based method for obtaining line correspondences in the initial pose estimation was found to be more effective than feature line matching, especially when dealing with larger viewpoint gaps.

Limitations and Future Work

  • Limitations: The paper acknowledges that the RANSAC-based method for line correspondences in the re-localization step can be less accurate than feature line matching because it relies solely on 2D geometric properties and doesn't consider visual features or depth, potentially leading to false matches.
  • Future Work: The authors suggest that the combination of points and lines, along with different strategies for obtaining line correspondences at different stages, is a promising direction for improving indoor visual localization. Further exploration of these combined approaches could lead to even more robust and accurate systems.

Summary updated

BibTeX
@inproceedings{matsumoto2024indoor,
  author = {Yuya Matsumoto and Gaku Nakano and Kazumine Ogura},
  title = {{Indoor Visual Localization Using Point and Line Correspondences in Dense Colored Point Cloud}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure