TL;DR
This paper introduces a novel multi-sensor fusion system for Micro-Aerial Vehicles (MAVs) that combines long-range stereo vision, IMU, GPS, and barometric data. It addresses key challenges in MAV state estimation, including aggressive motion, intermittent GPS, and high-altitude operation, by proposing a robust long-range stereo visual odometry and a stochastic cloning EKF for sensor fusion.
What Problem Does This Paper Address?
Accurate state estimation (position, orientation, velocity) is paramount for MAVs to navigate autonomously, avoid obstacles, and map their environment. However, MAVs face unique challenges: they are often equipped with lightweight sensors, exhibit aggressive 6-DOF motion, and can experience unreliable or unavailable GPS signals, especially in GPS-denied environments or at high altitudes. Existing visual odometry (VO) methods struggle with long-range scenarios where stereo vision's depth accuracy degrades significantly, and integrating various sensors with different data rates and uncertainties is complex. This paper aims to provide a robust and accurate state estimation solution that overcomes these limitations.
Methods
The paper presents a two-pronged approach:
- Long-Range Stereo Visual Odometry (VO):
- Adaptive Mode Switching: The VO system dynamically switches between a "stereo mode" for short-range, low-altitude flights and a "monocular mode" for long-range, high-altitude flights. This is crucial because at high altitudes, the baseline-to-depth ratio becomes too large for traditional stereo triangulation to be effective.
- Multi-View Stereo Triangulation & Inverse Depth Filtering: For long-range scenarios, it leverages multi-view stereo triangulation and an inverse depth filter. This allows it to refine depth estimates over time using multiple observations, even when direct stereo triangulation is unreliable.
- IMU Integration: The VO system uses IMU pre-integration data from the EKF to improve feature tracking and matching robustness, especially during aggressive motions.
- Stochastic Cloning EKF State Estimator:
- Loosely-Coupled Fusion: This EKF loosely fuses absolute measurements (GPS, barometer) with relative measurements (IMU, VO).
- Delayed State Cloning: To handle the asynchronous nature of sensor data and the different update rates, a "stochastic cloning" technique is employed. This augments the state with a delayed pose and its covariance, allowing for more consistent updates from the VO.
- Robustness to GPS Outages: The system is designed to maintain reasonable state estimation even when GPS is unavailable by relying more heavily on IMU and VO.
Datasets
- EuRoC Dataset: A benchmark dataset for MAV state estimation, containing nine stereo-IMU datasets recorded in three different indoor environments. The authors specifically used six datasets from the VICON room for evaluation due to the availability of 6-DOF ground truth.
- Real Flight Dataset: The authors also collected their own dataset using their developed MAV in Fort Indiantown Gap, Pennsylvania, featuring aggressive 6-DOF motions over a 12-minute flight.
Key Insights and Ideas
- Stereo VO's Long-Range Limitation is Solvable: The paper demonstrates that by intelligently switching between stereo and monocular-like modes and employing multi-view techniques, stereo VO can be made effective even at long ranges and high altitudes, where it traditionally fails.
- IMU as a "Helper" for VO: Integrating IMU data not just for state estimation but also to aid the VO process (feature matching, pose tracking) significantly improves its robustness, especially in challenging dynamic scenarios.
- Stochastic Cloning for Asynchronous Fusion: The "stochastic cloning" approach is a clever way to handle the asynchronous nature of sensor data in an EKF framework, allowing for more consistent and accurate fusion of measurements arriving at different rates.
- MAV State Estimation is a Multi-faceted Problem: The paper highlights that no single sensor is sufficient. Combining absolute (GPS, barometer) and relative (IMU, VO) measurements, each with its own strengths and weaknesses, is essential for robust MAV navigation.
Key Results
- Improved Long-Range VO Performance: The proposed long-range stereo VO showed improved performance on challenging datasets compared to state-of-the-art methods like ORBSLAM 2.0, particularly in long-range, high-altitude scenarios where ORBSLAM 2.0 failed.
- Robustness to GPS Outages: In simulated GPS outage scenarios, the fused system with VO maintained consistent and accurate state estimation, whereas systems without VO experienced significant drift and error increase due to noisy IMU data.
- Effective Multi-Sensor Fusion: The multi-sensor fusion EKF demonstrated consistent state estimation, with errors bounded within the 3-sigma covariance band, especially after transitioning to relative VO updates.
- Real-time Capability: The EKF implementation could run in real-time on various platforms, including a desktop PC and embedded systems like NVIDIA Jetson TX1 and Odroid XU4. The long-range stereo VO also achieved real-time performance on a desktop PC.
Limitations and Future Work
- VO Performance on Low-Cost Hardware: While the EKF can run in real-time on low-cost hardware, the stereo VO implementation was significantly slower on the Odroid XU4 compared to a desktop PC, primarily due to OpenCV function performance.
- Future Work: The authors plan to focus on code optimization to reduce stereo VO time consumption on low-cost hardware. They also aim to further improve the robustness and efficiency of their multi-sensor fusion system.
Summary updated