Survey and Evaluation of RGB-D SLAM

Shishun Zhang, Longyu Zheng, Wenbing TaoIEEE Access 2021 · DOI

Abstract

This paper provides a systematic taxonomy and rigorous benchmarking of state-of-the-art RGB-D Simultaneous Localization and Mapping (SLAM) systems. By evaluating seven representative algorithms across three diverse datasets, the authors clarify the trade-offs between direct, indirect, and hybrid tracking methods. The study offers a definitive roadmap for selecting SLAM tools for robotics and AR/VR applications, highlighting the impact of sensor characteristics like rolling shutter and the critical role of loop closure in maintaining global consistency.

Introduction

Traditional visual SLAM (vSLAM) using monocular or stereo cameras faces significant challenges, including complex map initialization, computationally expensive triangulation, and scale ambiguity. The introduction of RGB-D sensors, such as the Microsoft Kinect, revolutionized the field by providing per-pixel depth information alongside color data. This effectively resolves the scale problem and simplifies dense reconstruction.

However, the proliferation of competing algorithms has made it difficult for researchers to select the optimal system for specific needs. Key questions remain regarding the choice between direct methods (using raw pixel intensities) and indirect methods (using geometric features), as well as how these systems handle real-world perturbations like motion blur or lighting changes. This paper addresses these gaps through a structured classification and head-to-head performance evaluation.

Methods and System Architecture

The authors categorize RGB-D SLAM into three primary components: Tracking, Mapping, and Loop Closing.

General architecture of an RGB-D SLAM system, divided into Front End tracking and Back End mapping/optimization.

1. Tracking (The Front End)

The front end is responsible for estimating the camera's pose between consecutive frames.

  • Direct Methods: Minimize photometric error (intensity differences). Examples include KinectFusion and DVO-SLAM.
  • Indirect Methods: Minimize geometric error between matched features (e.g., ORB or SIFT). ORB-SLAM2 is a prominent example.
  • Hybrid Methods: Combine both approaches for enhanced accuracy, as seen in BundleFusion.

2. Mapping (The Back End)

The back end maintains the global map of the environment.

  • Point-Based: Represents the world as raw points or "surfels" (surface elements).
  • Volumetric: Uses Truncated Signed Distance Functions (TSDF) or voxel grids to create smooth, watertight surfaces.

3. Loop Closing

Loop closing recognizes previously visited locations to correct accumulated "drift" in the camera's trajectory, ensuring the map remains globally consistent.

Experimental Evaluation

The authors evaluated seven open-source systems—DVO-SLAM, KinectFusion, ElasticFusion, RGBDSLAM_v2, ORB-SLAM2, BundleFusion, and BAD-SLAM—using three industry-standard benchmarks: TUM RGB-D, ICL-NUIM, and ETH3D.

Visualization of camera trajectory and sparse reconstruction, illustrating the tracking and loop closure process.

Key Insights and Results

The evaluation revealed several critical performance trade-offs:

  • Direct vs. Indirect: Direct methods are generally faster and excel in low-texture environments but are sensitive to lighting changes and rolling shutter effects. Indirect methods like ORB-SLAM2 are robust to motion blur but fail in feature-sparse "white wall" scenarios.
  • The Impact of Rolling Shutter: Most consumer RGB-D cameras use rolling shutters, introducing distortions that break geometric assumptions. BAD-SLAM stood out for its ability to mitigate these effects through superior calibration and gradient-based tracking.
  • Loop Closure Necessity: Without loop closure, trajectory drift accumulates linearly, eventually compromising global consistency.
  • Hybrid Advantages: BundleFusion’s "sparse-then-dense" strategy allows it to excel in complex environments where purely direct or indirect methods might struggle.

In terms of specific metrics, ORB-SLAM2 was the most consistently accurate in standard sequences, while BAD-SLAM dominated the high-quality ETH3D dataset. KinectFusion and ElasticFusion demonstrated the lowest tracking times, while BundleFusion provided the most complete and accurate 3D models.

Limitations and Future Work

Despite significant progress, several challenges remain. Handling dynamic environments with moving people or objects is a major hurdle for current static-world assumptions. Future research frontiers include integrating deep learning for semantic mapping, achieving real-time non-rigid reconstruction, and improving multi-sensor fusion (e.g., RGB-D with IMUs) for high-speed robotics.

Summary updated

BibTeX
@article{zhang2021survey,
  author = {Shishun Zhang and Longyu Zheng and Wenbing Tao},
  title = {{Survey and Evaluation of RGB-D SLAM}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2021.3053188},
}
Figure