Abstract
It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. This paper presents a generic framework, dubbed Multi-Sensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates.
Introduction
Navigating a Micro Aerial Vehicle (MAV) through complex environments—such as transitioning from a GPS-denied indoor warehouse to a GPS-rich outdoor field—remains a fundamental challenge in robotics. While modern drones are equipped with a plethora of sensors including cameras, IMUs, pressure sensors, and GPS, integrating these disparate data streams in real-time is notoriously difficult. Researchers from ETH Zurich and NASA JPL have addressed this by introducing the MSF-EKF framework, designed to make sensor integration as modular as a "plug-and-play" system.
Traditional multi-sensor fusion faces three primary hurdles:
- Calibration Complexity: Sensors are rarely perfectly aligned, and extrinsic calibration usually requires tedious pre-flight procedures.
- Temporal Disparity: Sensors operate at vastly different rates and delays (e.g., 5Hz GPS with high latency vs. 1kHz IMU).
- Rigid Architectures: Many systems are hard-coded for specific sensor suites, making them brittle if a sensor fails or a new one is added.
The MSF-EKF Framework
The MSF-EKF addresses these issues through a modular architecture that treats sensors as independent components. The filter maintains "core" states—position, velocity, and attitude driven by the IMU—separately from "auxiliary" states, which contain sensor-specific parameters like camera scale or GPS offsets. This separation allows for the dynamic addition or removal of sensors at compile-time.
Handling Temporal Delays
To manage varying sensor latencies, the system employs a ring buffer that stores a history of states. When a delayed measurement arrives, the filter identifies the corresponding historical state, applies the update, and then "re-propagates" the physics forward to the current time.
Furthermore, the framework utilizes an Iterated EKF (IEKF) to re-calculate the drone's trajectory multiple times for each update. This significantly reduces linearization errors, which are common in high-speed, non-linear MAV maneuvers.
Experimental Validation
The approach was validated using an Asctec Firefly MAV equipped with an IMU, pressure sensor, monocular camera (running PTAM), and a GPS receiver. To ensure rigorous verification, the researchers used a Leica TotalStation 15i, a high-end surveying tool providing sub-millimeter precision, to establish ground truth. Experiments involved outdoor flights over trajectories exceeding 800 meters at speeds up to 4 m/s.
Experimental Results
The MSF-EKF demonstrated remarkable robustness and accuracy. A key insight from the study was the treatment of visual SLAM: by treating vision as a relative measurement rather than an absolute one, the system allows uncertainty to grow naturally, enabling GPS to correct long-term drift effectively.
The system successfully handled sensor outages and corrected for visual scale errors (often 3-5%) by fusing pressure and IMU data. Computationally, the framework proved highly efficient, with measurement updates taking approximately 65 microseconds and state re-propagation requiring only 21 microseconds.
Conclusion
The MSF-EKF framework provides a foundational tool for resilient autonomous systems. While the researchers noted that initialization of new sensors still requires specific drone movements to calibrate certain states, the modularity and efficiency of the system represent a significant step forward. Future work aims to integrate key-frame based SLAM more tightly to further reduce temporal drift and to open-source the framework for the broader robotics community.
Summary updated