Abstract
Estimating motion from images is a fundamental task in robotics. While traditional visual odometry (VO) focuses on camera motion in static scenes, and object tracking often requires prior models, estimating both simultaneously remains a challenge. This paper introduces Multimotion Visual Odometry (MVO), a framework that estimates the full $SE(3)$ trajectories of a moving camera and multiple independent objects without prior knowledge. Evaluated on a real-world dataset with ground truth, MVO demonstrates robust performance in complex dynamic environments.
Introduction
In robotic vision, Visual Odometry (VO) typically assumes a static world to estimate a robot's ego-motion. However, real-world environments like busy urban streets are inherently dynamic. Traditional VO systems often treat moving objects as outliers to be filtered out, which can lead to failure if dynamic objects dominate the field of view. Conversely, multiple object tracking (MOT) systems often rely on pre-trained detectors or assume known camera motion.
MVO addresses this "chicken-and-egg" problem: accurate segmentation requires motion information, while motion estimation requires segmented objects. By treating moving objects as valuable data rather than noise, MVO simultaneously estimates the 6-DOF trajectories of the camera and all rigid bodies in the scene.
Methodology
The MVO pipeline extends the standard stereo VO workflow by incorporating a robust multimotion fitting stage.
1. Tracklet Generation and Graph Construction
The system extracts sparse 3D features and matches them across frames to create "tracklets." A geometric neighborhood graph is then constructed where vertices represent tracklets. Edges connect tracklets that maintain spatial proximity over time, indicating they likely belong to the same rigid body.
2. Iterative Labeling via Energy Minimization
Unlike sequential RANSAC approaches that greedily extract motions, MVO employs an energy minimization framework called CORAL. This process involves:
- Proposal: Generating motion hypotheses using RANSAC on small tracklet clusters.
- Assignment: Labeling tracklets (e.g., "Camera," "Object N," or "Outlier") by balancing data fit against spatial smoothness.
- Refinement: Dynamically merging similar motion models and splitting those that poorly explain the data.
3. Batch Estimation
Following segmentation, a full-batch bundle adjustment is performed for each identified motion to produce smooth, high-accuracy $SE(3)$ trajectories.
Experimental Results
The authors validated MVO using a custom dataset captured with a Bumblebee XB3 stereo camera, with ground truth provided by a Vicon motion capture system for both the camera and four moving blocks.
Segmentation and Tracking Accuracy
In experiments involving five distinct motions (camera + 4 objects), MVO correctly identified the number of motions in 96.8% of frames, significantly outperforming sequential RANSAC (31.2%). The camera's ego-motion drift was approximately 3.24% of the path length, comparable to state-of-the-art static VO systems. Object tracking achieved translational errors between 0.27m and 0.99m, depending on the duration the objects remained within the camera's frustum.
Key Insights
- Stability: Simultaneous estimation is far more stable than sequential methods, which often produce "ghost" motions from noise.
- Generality: Being model-free, MVO tracks any rigid body based on geometry rather than visual appearance (e.g., car vs. box).
- Smoothness: Graph-based regularization ensures clean segmentation even with sparse feature points.
Conclusion and Future Work
MVO provides a robust framework for understanding complex dynamic scenes without prior object models. Current limitations include a dependency on feature-rich surfaces and challenges with objects re-entering the field of view. Future work will focus on improving re-identification and optimizing the pipeline for real-time robotic applications.
Summary updated