Dynamic Object-Aware Visual Odometry (VO) Estimation Based on Optical Flow Matching

Hae Min Cho, Euntai KimIEEE Access 2023 · OpenReview

Abstract

In this work, we propose a new visual odometry (VO) system that exploits the dynamic parts of an image. The key idea of our method is to identify the dynamic parts by combining semantic segmentation and optical flow and to suppress the dynamic parts in the process of VO estimation. First, movable objects are detected using the semantic segmentation. If an object contains many pixels of inconsistent optical flow, the object is considered as dynamic and merged with other dynamic objects to create a dynamic mask. Next, the ego-motion of the camera is estimated by using only the remaining static parts of the image and suppressing its dynamic parts. Unlike the other popular deep learning-based VO approaches, our method uses a geometric approach based on optimization to achieve high performance. The ego-motion of the camera is obtained by optimizing the correspondences obtained using the dynamic mask and optical flow consistency. Finally, the proposed method is applied to the KITTI Odometry benchmark dataset. Our method achieves an average of 7.67% translation error and 2.186°/100m rotation error, representing a performance improvement of 21% and 11% from the state-of-the-art baseline, DF-VO, respectively. In addition, our method yields a Relative Pose Error (RPE) of 0.186m, a 52% improvement against ORB-SLAM2. The experimental results show that the proposed VO method reliably predicts excellent visual odometry in the presence of dynamic objects.

Introduction

Most Visual Odometry (VO) and SLAM (Simultaneous Localization and Mapping) systems rely on a "static world assumption." They assume that the majority of the environment is stationary, allowing the system to calculate camera movement (ego-motion) based on how stationary landmarks shift between frames.

However, real-world environments—like busy city streets—are full of dynamic objects: cars, cyclists, and pedestrians. If a VO system inadvertently uses features from a moving car to calculate its own position, the resulting estimate will be skewed, leading to "drift" and significant localization errors. While deep learning approaches have tried to solve this by learning motion end-to-end, they often lack the mathematical precision of classical geometric methods. This paper seeks to bridge that gap by using deep learning for perception while retaining geometric optimization for estimation.

Proposed Method

The proposed pipeline uses three distinct deep neural networks to feed a classical geometric optimizer:

  1. Semantic/Instance Segmentation (Mask R-CNN): The system identifies "movable" objects (e.g., cars, buses, pedestrians). Crucially, it identifies individual instances rather than just masking categories.
  2. Optical Flow (LiteFlowNet): The system calculates the pixel-level motion between consecutive frames.
  3. Dynamic Masking: Instead of blindly removing all movable objects (which might be parked and thus useful for navigation), the system checks for flow consistency. It compares forward and backward optical flow; if the flow is inconsistent within a semantically identified "movable" object, that object is flagged as truly dynamic and masked out.
  4. Depth Estimation (Monodepth2): Used to recover the absolute scale of the motion, a common challenge in monocular VO.
Architecture of the proposed VO system, highlighting the Mask R-CNN-based semantic segmentation branch.

Finally, the system performs Geometric Optimization. It takes the remaining "static" pixels and uses epipolar geometry and Perspective-n-Point (PnP) methods to calculate the camera's rotation and translation. By only looking at the parts of the world that are truly standing still, the mathematical model becomes significantly more reliable. To ensure a stable pose estimate, the image is divided into a grid to select correspondences evenly across the domain, preventing the optimizer from becoming over-reliant on a single high-texture area.

Experimental Results

The researchers evaluated their method using the KITTI Odometry benchmark dataset, the industry standard for autonomous driving research. It consists of 11 driving sequences (00-10) with ground-truth camera poses provided by high-precision GPS/IMU systems.

The proposed method showed significant improvements over existing baselines:

  • Accuracy: On the KITTI benchmark, it achieved an average translation error of 7.67%, a 21% improvement over the previous state-of-the-art hybrid method, DF-VO.
  • Robustness: It yielded a Relative Pose Error (RPE) of 0.186m, representing a 52% improvement over ORB-SLAM2, a widely used geometry-based method that often struggles in dynamic traffic.
  • Qualitative Performance: Trajectory plots show that the system maintains a much tighter alignment with ground-truth paths in high-traffic sequences compared to methods that do not explicitly handle dynamic objects.

Conclusion

This work demonstrates that combining deep learning-based perception with classical geometric optimization yields a robust VO system. While the current implementation uses fragmented training for its sub-networks and operates on a frame-to-frame basis, it sets a strong foundation for future SLAM research. Future work will focus on end-to-end training and incorporating global mapping or loop closure to transition from visual odometry to a full SLAM system.

Summary updated

Abstract

In this work, we propose a new visual odometry (VO) system that exploits the dynamic parts of an image. The key idea of our method is to identify the dynamic parts by combining semantic segmentation and optical flow and to suppress the dynamic parts in the process of VO estimation. First, movable objects are detected using the semantic segmentation. If an object contains many pixels of inconsistent optical flow, the object is considered as dynamic and merged with other dynamic objects to create a dynamic mask. Next, the ego-motion of the camera is estimated by using only the remaining static parts of the image and suppressing its dynamic parts. Unlike the other popular deep learning-based VO approaches, our method uses geometric approach based on optimization to achieve high performance. That is, the ego-motion of the camera is obtained by optimizing the correspondences obtained using the dynamic mask and optical flow consistency. Finally, the proposed method is applied to the KITTI Odometry benchmark dataset, and its performance is compared with that of the previous VO methods. Our method achieves an average of 7.67(%) translation error and 2.186 rotation error(°/100m), which imply the performance improvement by 21% and 11% from the state-of-the-art baseline, DF-VO, respectively. In addition, our method yields the RPE (Relative Pose Error) of 0.186m, which is the performance improvement by 52% against ORB-SLAM2, a popular geometry-based method. The experimental results show that the proposed VO method reliably predicts excellent visual odometry in the presence of dynamic objects.
BibTeX
@inproceedings{cho2023dynamic,
  author = {Hae Min Cho and Euntai Kim},
  title = {{Dynamic Object-Aware Visual Odometry (VO) Estimation Based on Optical Flow Matching}},
  booktitle = {IEEE Access},
  year = {2023},
  url = {https://openreview.net/forum?id=7anAREcNxGI},
}
Figure