Introduction
Monocular Visual Odometry (VO) is a critical component for autonomous navigation, enabling motion estimation using a single camera. While effective in static environments, urban settings present significant challenges due to "distractors"—independently moving objects such as buses, cars, and pedestrians. Traditional outlier rejection methods like RANSAC often fail when these distractors dominate the field of view, leading the system to incorrectly interpret the motion of a large vehicle as the static background. This paper presents a self-supervised approach to identify and ignore these ephemeral regions, ensuring robust, metric-scale motion estimation even in high-traffic scenarios.
Methodology
The proposed pipeline transitions from high-fidelity offline mapping to efficient online execution through a three-stage process:
- Self-Supervised Labeling: To eliminate the need for manual annotation, the authors utilize a survey vehicle equipped with LIDAR and stereo cameras. By performing multiple traversals of the same route, they calculate a structural entropy metric. Points that appear consistently across different days are classified as "static" (e.g., buildings), while those appearing infrequently are labeled "ephemeral" (e.g., traffic).
- Deep Learning Framework: A deep convolutional neural network, comprising an encoder and two decoders, is trained using monocular images. The network is tasked with predicting both metric depth—solving the inherent scale ambiguity of monocular systems—and an "ephemerality mask."
- Ephemerality-Aware VO: The predicted mask is integrated into VO algorithms. In sparse VO, it acts as a filter to exclude features on dynamic objects. In dense VO, it serves as a weight in the photometric error function to minimize the influence of ephemeral pixels.
Evaluation and Results
The system was rigorously tested using the Oxford RobotCar Dataset, involving over 400km of driving across 42 traversals, including unmapped locations to test generalization.
Key Insights
- Entropy-Based Ground Truth: Using repeated traversals to define static vs. dynamic regions bypasses the need for semantic labels, focusing purely on motion consistency.
- Metric Scale from Experience: Training against LIDAR-derived depth allows the monocular network to learn absolute scale, a significant hurdle for single-camera systems.
- Robustness to Occlusion: The system maintains accuracy even when distractors occupy the majority of the frame, provided a small portion of the static world remains visible.
Performance Metrics
The ephemerality-aware approach demonstrated superior performance over baseline VO methods:
- Drift Reduction: Across the 400km test set, translational drift in Sparse VO was reduced from 6.55% to 6.38%.
- Traffic Robustness: In distractor-heavy scenes, velocity estimation error for Sparse VO improved by 4.5x, dropping from 0.220 m/s to 0.0489 m/s when using the mask.
- Qualitative Success: The masks effectively identified diverse dynamic objects, including cyclists and pedestrians, regardless of their distance or orientation.
Discussion and Future Work
While the method significantly enhances VO robustness, it currently requires an initial survey phase with high-end sensors to generate training labels. Future research could explore online learning capabilities to adapt to new environments without prior data. Additionally, integrating semantic segmentation could provide more granular information about the types of distractors encountered, potentially improving tracking and safety in autonomous path planning.
Summary updated