Training Deep SLAM on Single Frames

Igor Slinko, Anna Vorontsova, Dmitry Zhukov, Olga Barinova, Anton KonushinarXiv 2019 · arXiv

Abstract

This paper introduces a novel unsupervised method for training deep visual odometry (VO) and SLAM models using only single frames with depth information, rather than traditional video sequences. By synthesizing optical flow from simulated camera motions applied to 3D point clouds, the authors train a VO model without ground truth poses. This approach is extended into a full SLAM system using pose graph optimization, achieving state-of-the-art unsupervised results on the KITTI and EuRoC datasets. Notably, the method matches supervised performance on rigid indoor environments.

Introduction

Deep learning-based visual odometry (VO) and SLAM methods typically require large amounts of video data paired with precise ground-truth camera poses for supervised training. Collecting such data is expensive and often requires specialized hardware like high-end IMUs or motion capture systems. While unsupervised methods exist, they usually rely on sequential video frames to enforce photometric consistency and historically lag significantly behind supervised methods in accuracy.

This research addresses these challenges by shifting the training paradigm from video sequences to single frames with depth maps. The goal is to close the performance gap between supervised and unsupervised approaches while simplifying the training pipeline by removing the need for ground-truth trajectory data.

Methodology

Synthetic Optical Flow Generation

The core innovation of this work is the generation of synthetic training data from single images. The process begins with a single image's depth map (obtained from a sensor or estimated via stereo), which is projected into a 3D point cloud. The system then applies a randomly sampled 6DoF camera motion—based on a Student's t-distribution fitted to real-world motion profiles—and re-projects the points to create a virtual second frame.

Comparison of estimated optical flow from PWC-Net versus flow synthesized from depth maps and simulated motion.

The difference between these projections yields a synthetic optical flow field paired with a perfectly known, generated camera motion. This allows the model to learn the relationship between visual displacement and physical movement without ever seeing a real video sequence during training.

Dual VO Networks and SLAM Integration

The authors utilize PWC-Net to extract optical flow and train a neural network to predict 6DoF motion from it. A key insight is that a single network struggles to handle the vast range of motion scales encountered in SLAM. Consequently, the authors train two specialized models: $NN_{cons}$ for the small motions typical of consecutive frames, and $NN_{loops}$ for the wider, more diverse motion distributions seen during loop closures.

The architecture of the visual odometry model, featuring a series of convolutional layers followed by separate fully connected heads for translation (t) and rotation (r).

To upgrade the VO model to a full SLAM system, a Bag of Visual Words (BoVW) approach with SIFT features is used for relocalization and loop closure detection. These loop closures are integrated into a pose graph where edges represent relative motions predicted by the specialized networks. The final trajectory is optimized using the g2o library.

Experimental Results

The method was evaluated on two primary benchmarks: the KITTI Odometry 2012 dataset (outdoor driving) and the EuRoC MAV dataset (indoor drone flight).

  • KITTI Results: The proposed unsupervised SLAM method achieved an Absolute Trajectory Error (ATE) of 3.24, significantly outperforming prior unsupervised state-of-the-art methods like SGANVO (5.12) and SfMLearner (28.14). The transition from pure VO to SLAM (via graph optimization) was crucial, reducing the ATE from 14.30 to 3.24.
  • EuRoC Results: On the challenging EuRoC dataset, the model achieved an ATE of 0.84. This performance matched supervised models, which the authors attribute to the synthetic data's ability to act as a strong regularizer, preventing the overfitting often seen in supervised models trained on limited trajectories.

Discussion and Key Insights

A primary finding of this work is that single frames are sufficient for training motion estimation if depth is available. The synthetic data generation implicitly assumes a perfectly rigid scene. This explains the model's exceptional performance on the rigid EuRoC dataset. However, it also highlights a limitation: the model still trails supervised methods on KITTI because the synthetic flow does not yet account for independently moving objects like other vehicles.

Conclusion

By leveraging single-frame depth to synthesize infinite varieties of camera motion, this method provides a robust and simplified path for training deep SLAM systems. Future work will focus on incorporating dynamic object modeling into the synthetic flow generation to bridge the remaining performance gap in non-rigid, highly dynamic environments.

Summary updated

Abstract

Learning-based visual odometry and SLAM methods demonstrate a steady improvement over past years. However, collecting ground truth poses to train these methods is difficult and expensive. This could be resolved by training in an unsupervised mode, but there is still a large gap between performance of unsupervised and supervised methods. In this work, we focus on generating synthetic data for deep learning-based visual odometry and SLAM methods that take optical flow as an input. We produce training data in a form of optical flow that corresponds to arbitrary camera movement between a real frame and a virtual frame. For synthesizing data we use depth maps either produced by a depth sensor or estimated from stereo pair. We train visual odometry model on synthetic data and do not use ground truth poses hence this model can be considered unsupervised. Also it can be classified as monocular as we do not use depth maps on inference. We also propose a simple way to convert any visual odometry model into a SLAM method based on frame matching and graph optimization. We demonstrate that both the synthetically-trained visual odometry model and the proposed SLAM method build upon this model yields state-of-the-art results among unsupervised methods on KITTI dataset and shows promising results on a challenging EuRoC dataset.
BibTeX
@misc{slinko2019training,
  author = {Igor Slinko and Anna Vorontsova and Dmitry Zhukov and Olga Barinova and Anton Konushin},
  title = {{Training Deep SLAM on Single Frames}},
  year = {2019},
  eprint = {1912.05405},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.05405},
}
Figure