Abstract
We present a novel self-supervised algorithm named MotionHint for monocular visual odometry (VO) that takes motion constraints into account. A key aspect of our approach is to use an appropriate motion model that can help existing self-supervised monocular VO (SSM-VO) algorithms to overcome issues related to the local minima within their self-supervised loss functions. The motion model is expressed with a neural network named PPnet. It is trained to coarsely predict the next pose of the camera and the uncertainty of this prediction. Our self-supervised approach combines the original loss and the motion loss, which is the weighted difference between the prediction and the generated ego-motion. Taking two existing SSM-VO systems as our baseline, we evaluate our MotionHint algorithm on the standard KITTI benchmark. Experimental results show that our MotionHint algorithm can be easily applied to existing open-sourced state-of-the-art SSM-VO systems to greatly improve the performance by reducing the resulting ATE by up to 28.73%.
Introduction
Self-supervised monocular visual odometry (VO) has emerged as a powerful alternative to supervised methods, leveraging photometric consistency across video frames to learn depth and ego-motion. However, these systems often struggle with "local minima" in their loss functions. Real-world challenges such as texture-less regions, lighting variations, and motion blur create ambiguous photometric signals, leading to physically implausible trajectory predictions, particularly during sharp maneuvers where the system may drift significantly.
To address these limitations, we propose MotionHint, a framework that incorporates learned motion constraints into the self-supervised learning process. By teaching the model that vehicle motion follows predictable physical patterns, we provide a "hint" that guides the optimization process away from erroneous local minima.
Methodology
The core of MotionHint is PPnet, a motion prediction network based on a Long Short-Term Memory (LSTM) architecture. PPnet is designed to act as a "motion expert" that predicts the subsequent camera pose based on a sequence of prior poses.
PPnet Architecture and Training
PPnet is trained to output both a predicted pose and its associated uncertainty. The training process involves three distinct phases:
- VO Pre-training: An initial self-supervised VO system (e.g., SC-Depth) is trained using standard photometric loss.
- PPnet Pre-training: PPnet learns motion dynamics from pose sequences. Notably, it can be trained on noisy trajectories from traditional geometric methods like ORB-SLAM2, making it highly flexible and independent of ground truth for this stage.
- Joint Fine-tuning: The VO system is fine-tuned using a combined loss function that balances visual photometric loss with a new motion loss—the difference between the VO's predicted ego-motion and PPnet's expectation, weighted by the predicted uncertainty.
To ensure robustness, we employ Pose Centralization to prevent global drift from affecting local motion prediction and Scale Augmentation to make PPnet invariant to the scale drift common in monocular VO systems.
Experiments and Results
We evaluated MotionHint on the KITTI benchmark using sequences 00-08 for training and 09-10 for testing. We compared our approach against state-of-the-art baselines, including SC-Depth.
Performance Gains
Experimental results demonstrate that MotionHint significantly reduces the Absolute Trajectory Error (ATE). When applied to SC-Depth, MotionHint achieved a reduction in ATE of up to 28.73%.
A key finding was that training PPnet on unpaired data (sequences not used for VO training) often yielded superior results. This suggests that exposure to a broader variety of motion patterns enhances the model's ability to regularize the VO system effectively. Furthermore, the inclusion of uncertainty prediction allows the system to discount the motion "hint" when the predicted maneuver is highly unusual, preventing the motion model from overriding valid visual evidence during rare events.
Conclusion
MotionHint provides a robust, plug-and-play framework for improving self-supervised monocular VO. By integrating a learned motion prior via PPnet, we effectively mitigate the issues of local minima and photometric ambiguity. Future work will explore the application of these constraints to more diverse platforms, such as aerial vehicles, and investigate tighter integration between geometric perception and motion modeling to further stabilize navigation in challenging environments.
Summary updated