Abstract
This paper introduces a novel unsupervised method for training deep visual odometry (VO) and SLAM models using only single frames with depth information, rather than traditional video sequences. By synthesizing optical flow from simulated camera motions applied to 3D point clouds, the authors train a VO model without ground truth poses. This approach is extended into a full SLAM system using pose graph optimization, achieving state-of-the-art unsupervised results on the KITTI and EuRoC datasets. Notably, the method matches supervised performance on rigid indoor environments.
Introduction
Deep learning-based visual odometry (VO) and SLAM methods typically require large amounts of video data paired with precise ground-truth camera poses for supervised training. Collecting such data is expensive and often requires specialized hardware like high-end IMUs or motion capture systems. While unsupervised methods exist, they usually rely on sequential video frames to enforce photometric consistency and historically lag significantly behind supervised methods in accuracy.
This research addresses these challenges by shifting the training paradigm from video sequences to single frames with depth maps. The goal is to close the performance gap between supervised and unsupervised approaches while simplifying the training pipeline by removing the need for ground-truth trajectory data.
Methodology
Synthetic Optical Flow Generation
The core innovation of this work is the generation of synthetic training data from single images. The process begins with a single image's depth map (obtained from a sensor or estimated via stereo), which is projected into a 3D point cloud. The system then applies a randomly sampled 6DoF camera motion—based on a Student's t-distribution fitted to real-world motion profiles—and re-projects the points to create a virtual second frame.
The difference between these projections yields a synthetic optical flow field paired with a perfectly known, generated camera motion. This allows the model to learn the relationship between visual displacement and physical movement without ever seeing a real video sequence during training.
Dual VO Networks and SLAM Integration
The authors utilize PWC-Net to extract optical flow and train a neural network to predict 6DoF motion from it. A key insight is that a single network struggles to handle the vast range of motion scales encountered in SLAM. Consequently, the authors train two specialized models: $NN_{cons}$ for the small motions typical of consecutive frames, and $NN_{loops}$ for the wider, more diverse motion distributions seen during loop closures.
To upgrade the VO model to a full SLAM system, a Bag of Visual Words (BoVW) approach with SIFT features is used for relocalization and loop closure detection. These loop closures are integrated into a pose graph where edges represent relative motions predicted by the specialized networks. The final trajectory is optimized using the g2o library.
Experimental Results
The method was evaluated on two primary benchmarks: the KITTI Odometry 2012 dataset (outdoor driving) and the EuRoC MAV dataset (indoor drone flight).
- KITTI Results: The proposed unsupervised SLAM method achieved an Absolute Trajectory Error (ATE) of 3.24, significantly outperforming prior unsupervised state-of-the-art methods like SGANVO (5.12) and SfMLearner (28.14). The transition from pure VO to SLAM (via graph optimization) was crucial, reducing the ATE from 14.30 to 3.24.
- EuRoC Results: On the challenging EuRoC dataset, the model achieved an ATE of 0.84. This performance matched supervised models, which the authors attribute to the synthetic data's ability to act as a strong regularizer, preventing the overfitting often seen in supervised models trained on limited trajectories.
Discussion and Key Insights
A primary finding of this work is that single frames are sufficient for training motion estimation if depth is available. The synthetic data generation implicitly assumes a perfectly rigid scene. This explains the model's exceptional performance on the rigid EuRoC dataset. However, it also highlights a limitation: the model still trails supervised methods on KITTI because the synthetic flow does not yet account for independently moving objects like other vehicles.
Conclusion
By leveraging single-frame depth to synthesize infinite varieties of camera motion, this method provides a robust and simplified path for training deep SLAM systems. Future work will focus on incorporating dynamic object modeling into the synthetic flow generation to bridge the remaining performance gap in non-rigid, highly dynamic environments.
Summary updated