Robust Visual SLAM with Point and Line Features

Xingxing Zuo, Xiaojia Xie, Yong Liu, Guoquan Huang

Abstract

In this paper, we develop a robust efficient visual SLAM system that utilizes heterogeneous point and line features. By leveraging ORB-SLAM [1], the proposed system consists of stereo matching, frame tracking, local mapping, loop detection, and bundle adjustment of both point and line features. In particular, as the main theoretical contributions of this paper, we, for the first time, employ the orthonormal representation as the minimal parameterization to model line features along with point features in visual SLAM and analytically derive the Jacobians of the re-projection errors with respect to the line parameters, which significantly improves the SLAM solution. The proposed SLAM has been extensively tested in both synthetic and real-world experiments whose results demonstrate that the proposed system outperforms the state-of-the-art methods in various scenarios.

Introduction

Most Visual SLAM (Simultaneous Localization and Mapping) systems rely heavily on point features like ORB or SIFT. While effective, point-based systems often fail in "feature-poor" environments—such as long hallways with white walls, glass buildings, or areas with dramatic lighting changes. Line features are abundant in these man-made settings and offer more geometric structure than points. However, integrating them into a real-time optimization framework is difficult due to over-parameterization and the computational cost of numerical derivatives.

Methodology

The proposed system is built upon the ORB-SLAM2 framework, extended to handle heterogeneous features. The system extracts both point and line features to build a consistent 3D map and track camera poses simultaneously.

The system extracts point and line features (left) to build a consistent 3D map and track camera poses (right).

Improved Line Detection

The standard Line Segment Detector (LSD) is enhanced with a merging step. This addresses the common issue where a single physical line is detected as multiple broken segments, which typically complicates data association and leads to numerical instability.

Comparison of raw line detection versus the proposed merged line segments for improved data association.

Orthonormal Representation and Analytical Jacobians

To avoid the numerical instability of over-parameterized 3D lines, the authors employ a 4-parameter orthonormal representation. This minimal parameterization ensures a lean optimization back-end. Furthermore, the paper provides the first derivation of analytical Jacobians for line re-projection errors, significantly improving speed and accuracy over traditional numerical approximations used in previous work.

Point-Line Loop Closing

A dedicated thread for loop detection and correction utilizes both point and line descriptors (LBD), enabling more reliable recognition of previously visited locations and correction of accumulated drift.

Experimental Evaluation

The system was evaluated on synthetic data, the it3f indoor dataset, and the KITTI odometry benchmark.

  • Accuracy: On the KITTI dataset, the proposed PL-SLAM consistently achieved lower Absolute Trajectory Error (ATE) than ORB-SLAM2 and other line-based competitors.
  • Robustness: In the it3f indoor dataset, characterized by reflective surfaces and low-texture walls, the point-line fusion showed significantly less drift than point-only systems.
  • Efficiency: Despite the added complexity of processing lines, the system maintains real-time performance at approximately 15.1 frames per second (FPS) on standard hardware.

Conclusion

By fusing point and line features through a mathematically rigorous framework, this work demonstrates a significant leap in SLAM robustness for structured environments. Future work will focus on integrating Inertial Measurement Units (IMUs) to handle rapid movements and completely featureless scenarios, creating an even more resilient SLAM solution.

Summary updated

Abstract

In this paper, we develop a robust efficient visual SLAM system that utilizes heterogeneous point and line features. By leveraging ORB-SLAM [1], the proposed system consists of stereo matching, frame tracking, local mapping, loop detection, and bundle adjustment of both point and line features. In particular, as the main theoretical contributions of this paper, we, for the first time, employ the orthonormal representation as the minimal parameterization to model line features along with point features in visual SLAM and analytically derive the Jacobians of the re-projection errors with respect to the line parameters, which significantly improves the SLAM solution. The proposed SLAM has been extensively tested in both synthetic and real-world experiments whose results demonstrate that the proposed system outperforms the state-of-the-art methods in various scenarios.
BibTeX
@inproceedings{zuo2017robust,
  author = {Xingxing Zuo and Xiaojia Xie and Yong Liu and Guoquan Huang},
  title = {{Robust Visual SLAM with Point and Line Features}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  doi = {10.1109/iros.2017.8205991},
}
Figure