LDL: Line Distance Functions for Panoramic Localization

Junho Kim, Changwoon Choi, Hojun Jang, Young Min KimICCV 2023 · DOI · OpenReview

Introduction

Visual localization—the task of determining a camera's precise 6-degree-of-freedom (6DoF) pose within a pre-mapped 3D environment—is fundamental for autonomous robotics and augmented reality. Traditional point-based methods, which rely on matching local texture features like SIFT or SuperPoint, often struggle in indoor environments characterized by repetitive structures, featureless walls, and dramatic changes in illumination or scene layout.

LDL (Line Distance Functions) introduces a robust alternative by leveraging the structural "skeleton" of a scene: its line segments. By shifting from fragile point-to-point correspondences to the spatial distribution of lines, LDL achieves high-speed, illumination-invariant localization in panoramic images.

Methodology

The core of the LDL framework is the Line Distance Function (LDF). Unlike traditional methods that attempt to establish discrete line-to-line correspondences—a process prone to ambiguity—LDL treats lines as a dense field of distance values, facilitating a more robust matching process.

The LDL pipeline: from feature extraction to pose refinement via distance function decomposition and PnP-RANSAC.

The pipeline consists of four primary stages:

  1. Principal Direction Extraction: The system identifies vanishing points in the 360° panorama to determine dominant structural axes (e.g., X, Y, and Z). This step significantly constrains the search space for camera rotation.
  2. Distance Function Decomposition: To improve expressiveness and reduce noise, the line distribution is decomposed into three separate channels based on their principal directions. This prevents overlapping structural elements from confusing the matching algorithm.
  3. Coarse Pose Selection: LDL evaluates thousands of candidate poses in milliseconds. It compares the 2D LDF of the query image with the projected 3D LDF of the map using a robust "inlier-counting" loss function, which remains resilient to scene changes like moved furniture.
  4. Refinement: Promising coarse poses are polished using local features (e.g., SuperPoint/SuperGlue) and PnP-RANSAC to achieve high-precision 6DoF estimates.

Experimental Evaluation

The authors validated LDL on two challenging indoor benchmarks: Stanford 2D-3D-S, which features large-scale office environments with repetitive structures, and OmniScenes, which includes significant temporal variations in lighting and object placement.

Key Results

  • Efficiency: The pose search phase operates at millisecond scales, outperforming global feature matching methods like NetVLAD by an order of magnitude in indoor scenarios.
  • Precision: On the Stanford 2D-3D-S dataset, LDL achieved a median translation error of 0.02m, demonstrating competitive accuracy with state-of-the-art point-based systems.
  • Robustness: LDL exhibits exceptional resilience to lighting changes. In synthetic tests with extreme gamma and white balance shifts, LDL maintained up to 91% accuracy, while competing methods like CPO and PICCOLO failed to localize effectively.

Discussion and Insights

A key insight of LDL is the move from distribution over correspondence. By matching the global distribution of lines, the system avoids the computationally expensive and error-prone task of proving exact matches between individual segments.

3D line segments and local features used for high-precision refinement.

Furthermore, the method offers Privacy by Design. Since the refinement step only requires local features located near structural lines, sensitive information in other parts of the image (e.g., faces or documents) can be discarded. This protects against "feature inversion" attacks that attempt to reconstruct original images from stored descriptors.

Limitations

LDL is optimized for man-made indoor environments. Its performance may degrade in organic or unstructured environments (e.g., forests) where straight lines are absent. Additionally, while the initial pose search is extremely fast, the final refinement step remains a computational bottleneck.

Conclusion

LDL presents a significant advancement in panoramic localization by prioritizing structural line distributions over point-based textures. Its combination of speed, accuracy, and robustness to illumination makes it a compelling solution for real-world indoor localization challenges. Future work will focus on optimizing the refinement stage and scaling the approach to building-wide environments.

Summary updated

Abstract

We introduce LDL, a fast and robust algorithm that localizes a panorama to a 3D map using line segments. LDL focuses on the sparse structural information of lines in the scene, which is robust to illumination changes and can potentially enable efficient computation. While previous line-based localization approaches tend to sacrifice accuracy or computation time, our method effectively observes the holistic distribution of lines within panoramic images and 3D maps. Specifically, LDL matches the distribution of lines with 2D and 3D line distance functions, which are further decomposed along principal directions of lines to increase the expressiveness. The distance functions provide coarse pose estimates by comparing the distributional information, where the poses are further optimized using conventional local feature matching. As our pipeline solely leverages line geometry and local features, it does not require costly additional training of line-specific features or correspondence matching. Nevertheless, our method demonstrates robust performance on challenging scenarios including object layout changes, illumination shifts, and large-scale scenes, while exhibiting fast pose search terminating within a matter of milliseconds. We thus expect our method to serve as a practical solution for line-based localization, and complement the well-established point-based paradigm. The code for LDL is available through the following link: https://github.com/82magnolia/panoramic-localization.
BibTeX
@inproceedings{kim2023ldl,
  author = {Junho Kim and Changwoon Choi and Hojun Jang and Young Min Kim},
  title = {{LDL: Line Distance Functions for Panoramic Localization}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01639},
}
Figure