Introduction
Event cameras represent a paradigm shift in computer vision, offering microsecond latency, high dynamic range (HDR), and immunity to motion blur by reporting only pixel-wise intensity changes. However, their unconventional data format makes traditional state estimation difficult. Stereo-DEVO is the first learning-based stereo visual odometry (VO) system for event cameras that operates in real-time at VGA resolution, effectively bridging the gap between the robustness of deep learning and the geometric precision of stereo vision.
The Challenge: State Estimation with Events
Traditional cameras fail in high-speed or low-light scenarios due to motion blur and limited dynamic range. Event cameras excel here but present unique challenges for visual odometry. Existing model-based methods often struggle with noise in nighttime scenes where artificial lights flicker. Conversely, monocular learning-based methods suffer from scale ambiguity—the inability to perceive absolute real-world dimensions.
Methodology: Stereo-DEVO
Stereo-DEVO addresses these issues through a tightly-coupled deep learning pipeline:
- Static-Stereo Association: To determine metric depth, the system matches features between left and right cameras. It utilizes a feature encoder to build a pyramid of learned representations, performing a robust 1D search along epipolar lines. This approach is far more resilient to noise than matching raw events.
- Tightly-Coupled Bundle Adjustment (BA): The system maintains a patch graph within a sliding window, simultaneously optimizing camera trajectory and sparse patch depth using a Gauss-Newton solver. This ensures mathematical consistency between temporal motion and stereo geometry.
- Real-Time Processing: By refactoring the pipeline into a C++/Python hybrid on ROS, the authors achieved real-time processing of VGA-resolution (640x480) data, a significant milestone for learning-based event systems.
Experimental Results
The system was evaluated across diverse benchmarks, including large-scale driving (DSEC) and high-speed drone maneuvers.
Key findings include:
- Nighttime Robustness: In the 2.5km Zurich City sequence, Stereo-DEVO maintained a mean error of just 4.52 meters. Model-based competitors like ESVO2 failed entirely due to the "junk" events generated by artificial lighting.
- High-Speed Stability: During aggressive drone flights reaching 3.9 m/s, the system maintained an error of only 0.13m over a 55m path.
- Metric Accuracy: By leveraging stereo geometry, the system provides accurate real-world scaling without requiring an Inertial Measurement Unit (IMU).
Conclusion
Stereo-DEVO demonstrates that combining deep feature extraction with classical geometric optimization enables robust, real-time, and scale-aware visual odometry for event cameras. While future work will focus on further optimization for low-power embedded hardware and integration with IMUs for zero-motion scenarios, this research marks a major step toward practical neuromorphic vision for autonomous robotics.