Abstract
We introduce DROID-SLAM, a new deep learning based SLAM system. DROID-SLAM consists of recurrent iterative updates of camera pose and pixelwise depth through a Dense Bundle Adjustment layer. DROID-SLAM is accurate, achieving large improvements over prior work, and robust, suffering from substantially fewer catastrophic failures. Despite training on monocular video, it can leverage stereo or RGB-D video to achieve improved performance at test time.
Introduction
Simultaneous Localization and Mapping (SLAM) is a foundational capability for robotics, autonomous vehicles, and AR/VR. Historically, visual SLAM has faced a strict trade-off: classical optimization-based systems (e.g., ORB-SLAM) offer high accuracy but are brittle, often failing due to motion blur or featureless environments. Conversely, recent end-to-end deep learning approaches improve robustness but typically lack the trajectory accuracy of classical methods.
DROID-SLAM introduces a novel deep learning architecture that combines recurrent optical flow updates with a differentiable Dense Bundle Adjustment (DBA) layer. This system achieves state-of-the-art performance across monocular, stereo, and RGB-D modalities while demonstrating exceptional zero-shot generalization from synthetic training data to real-world environments.
Methodology
DROID-SLAM builds upon the RAFT architecture for optical flow, adapting it for long-term 3D SLAM. The system maintains a "frame graph" of co-visible images and utilizes two asynchronous threads: a frontend for local tracking and a backend for global optimization and loop closure.
The core innovation lies in the iterative update step:
- Recurrent Updates: For incoming frames, the network extracts features and builds 4D correlation volumes. A ConvGRU iteratively updates a hidden state, predicting revisions to dense optical flow fields and outputting confidence weights.
- Differentiable Dense Bundle Adjustment (DBA): Flow revisions are passed into a differentiable DBA layer. This layer employs a Gauss-Newton solver to compute updates to camera poses and dense, per-pixel depth maps, ensuring geometric alignment with predicted flow.
By embedding a classical solver directly within the neural network, DROID-SLAM forces the model to respect multi-view geometric constraints, preventing the drift common in pure deep-learning SLAM.
Experiments and Results
The model was trained exclusively on monocular video from the synthetic TartanAir dataset and evaluated across several real-world benchmarks including EuRoC, TUM-RGBD, and ETH3D-SLAM.
Key Performance Metrics
- TartanAir: Reduced trajectory error by over 60% in both monocular and stereo settings compared to prior state-of-the-art, ranking 1st in the ECCV 2020 SLAM competition.
- EuRoC: Achieved an average Absolute Trajectory Error (ATE) of 2.2cm in monocular settings, a 43% improvement over ORB-SLAM3.
- Robustness: The system suffered zero catastrophic failures on TartanAir, EuRoC, and TUM-RGBD. On the ETH3D-SLAM benchmark, it successfully tracked 30 out of 32 datasets, significantly outperforming the next best method (19/32).
Generalization
A key insight is the system's remarkable zero-shot generalization. A single model trained on synthetic monocular video can process real-world stereo and RGB-D data at inference time without retraining. Unlike prior works that use a learned depth basis, DROID-SLAM optimizes dense per-pixel depth directly, preventing overfitting to training-specific depth distributions.
Discussion and Limitations
DROID-SLAM demonstrates that marrying deep learning's robustness with geometric optimization's accuracy is a viable path for next-generation SLAM. However, the system has a significant compute and memory footprint. Global bundle adjustment requires storing feature maps for all keyframes; processing long sequences (up to 5000 frames) necessitates high-end GPUs with 24GB of VRAM. Future research will likely focus on improving memory efficiency and reducing redundant computations to enable deployment on more resource-constrained hardware.
Code is available at: https://github.com/princeton-vl/DROID-SLAM
Summary updated