Dropping the D: RGB-D SLAM Without the Depth Sensor

Mert Kiray, Alican Karaomer, Benjamin BusamarXiv 2025 · DOI · arXiv

Introduction

Simultaneous Localization and Mapping (SLAM) is a fundamental capability for autonomous robotics, enabling devices to construct a map of an unknown environment while simultaneously tracking their own location within it. Traditionally, researchers have faced a stark choice between two modalities: monocular SLAM and RGB-D SLAM.

Monocular SLAM utilizes a single, inexpensive camera but is plagued by "scale ambiguity"—the inability to distinguish between a small object nearby and a large object far away—and "scale drift" over time. Conversely, RGB-D SLAM employs specialized hardware like infrared projectors or LiDAR to capture metric depth. While highly accurate, these physical sensors are expensive, power-intensive, and frequently fail in outdoor sunlight or on reflective surfaces. DropD-SLAM introduces a third path: achieving the high-fidelity metric accuracy of RGB-D systems using only a standard monocular RGB camera.

The DropD-SLAM Architecture

The core innovation of DropD-SLAM lies in its modular front-end. Rather than training a single, end-to-end neural network that might struggle to generalize, the authors utilize a suite of pretrained, state-of-the-art AI models as "virtual sensors." This approach allows the system to benefit from the geometric robustness of traditional SLAM while leveraging the semantic power of modern deep learning.

Comparison of traditional RGB-D SLAM versus the proposed DropD-SLAM approach, highlighting the virtual sensor modules and trajectory accuracy.

Virtual Sensor Modules

  1. Metric Depth Estimation: By integrating foundation models such as DepthAnythingV2 or UniDepthV2, the system predicts dense depth maps from single RGB frames. These models provide "metric" depth (actual meters), effectively resolving the scale ambiguity inherent in monocular setups.
  2. Learned Keypoints: Instead of relying on hand-crafted features like ORB, which can fail under motion blur, the system uses Key.Net. This provides significantly more stable tracking in low-texture or high-dynamic environments.
  3. Dynamic Masking: To address the challenge of "dynamic noise" (e.g., moving people or vehicles), the system employs YOLOv11 to identify and mask out non-static objects. Morphological dilation is applied to these masks to ensure that edge pixels from moving objects do not corrupt the mapping process.

These virtual 3D features are then passed to an unmodified ORB-SLAM3 back-end, which performs the final geometric optimization.

Experimental Evaluation

The authors evaluated DropD-SLAM using the TUM RGB-D benchmark, testing it against both static and highly dynamic sequences (such as the freiburg3 walking sequences).

Key Results

  • Dynamic Robustness: On sequences with significant human motion, DropD-SLAM achieved a mean Absolute Trajectory Error (ATE) of 1.8 cm, notably outperforming specialized RGB-D methods like DynaSLAM (2.8 cm).
  • Static Stability: In static indoor environments, the system achieved a mean ATE of 7.4 cm, matching the performance of systems equipped with actual depth-sensing hardware.
  • Real-Time Performance: Despite the heavy AI workload, the pipeline maintains a processing speed of 22 FPS on an NVIDIA RTX 4090 GPU, making it viable for real-time robotic deployment.

Key Insights

A critical finding of the research is that temporal consistency—the stability of depth predictions across consecutive frames—is more vital for SLAM than absolute per-frame accuracy. A consistent model allows the SLAM back-end to optimize away minor biases, whereas a flickering model leads to trajectory drift. Furthermore, the AI-driven approach often outperformed physical sensors on dark or reflective surfaces where traditional infrared-based depth sensors typically produce data "holes."

Conclusion and Future Work

DropD-SLAM demonstrates that modern AI foundation models have reached a level of maturity where they can effectively replace specialized depth hardware in many SLAM applications. While the current implementation requires a high-end GPU, future work will focus on optimizing these models for edge devices and incorporating "uncertainty awareness" to allow the SLAM optimizer to dynamically weigh the reliability of AI-generated depth predictions.

Summary updated

Abstract

We present DropD-SLAM, a real-time monocular SLAM system that achieves RGB-D-level accuracy without relying on depth sensors. The system replaces active depth input with three pretrained vision modules: a monocular metric depth estimator, a learned keypoint detector, and an instance segmentation network. Dynamic objects are suppressed using dilated instance masks, while static keypoints are assigned predicted depth values and backprojected into 3D to form metrically scaled features. These are processed by an unmodified RGB-D SLAM back end for tracking and mapping. On the TUM RGB-D benchmark, DropD-SLAM attains 7.4 cm mean ATE on static sequences and 1.8 cm on dynamic sequences, matching or surpassing state-of-the-art RGB-D methods while operating at 22 FPS on a single GPU. These results suggest that modern pretrained vision models can replace active depth sensors as reliable, real-time sources of metric scale, marking a step toward simpler and more cost-effective SLAM systems.
BibTeX
@misc{kiray2025dropping,
  author = {Mert Kiray and Alican Karaomer and Benjamin Busam},
  title = {{Dropping the D: RGB-D SLAM Without the Depth Sensor}},
  year = {2025},
  eprint = {2510.06216},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.06216},
}
Figure