A robust and modular multi-sensor fusion approach applied to MAV navigation

Simon Lynen, Markus W. Achtelik, Stephan Weiss, Margarita Chli, Roland SiegwartIROS 2013 · OpenReview

Abstract

It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. This paper presents a generic framework, dubbed Multi-Sensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates.

Introduction

Navigating a Micro Aerial Vehicle (MAV) through complex environments—such as transitioning from a GPS-denied indoor warehouse to a GPS-rich outdoor field—remains a fundamental challenge in robotics. While modern drones are equipped with a plethora of sensors including cameras, IMUs, pressure sensors, and GPS, integrating these disparate data streams in real-time is notoriously difficult. Researchers from ETH Zurich and NASA JPL have addressed this by introducing the MSF-EKF framework, designed to make sensor integration as modular as a "plug-and-play" system.

Traditional multi-sensor fusion faces three primary hurdles:

  1. Calibration Complexity: Sensors are rarely perfectly aligned, and extrinsic calibration usually requires tedious pre-flight procedures.
  2. Temporal Disparity: Sensors operate at vastly different rates and delays (e.g., 5Hz GPS with high latency vs. 1kHz IMU).
  3. Rigid Architectures: Many systems are hard-coded for specific sensor suites, making them brittle if a sensor fails or a new one is added.

The MSF-EKF Framework

The MSF-EKF addresses these issues through a modular architecture that treats sensors as independent components. The filter maintains "core" states—position, velocity, and attitude driven by the IMU—separately from "auxiliary" states, which contain sensor-specific parameters like camera scale or GPS offsets. This separation allows for the dynamic addition or removal of sensors at compile-time.

Handling Temporal Delays

To manage varying sensor latencies, the system employs a ring buffer that stores a history of states. When a delayed measurement arrives, the filter identifies the corresponding historical state, applies the update, and then "re-propagates" the physics forward to the current time.

State re-propagation mechanism for delayed measurements.

Furthermore, the framework utilizes an Iterated EKF (IEKF) to re-calculate the drone's trajectory multiple times for each update. This significantly reduces linearization errors, which are common in high-speed, non-linear MAV maneuvers.

Experimental Validation

The approach was validated using an Asctec Firefly MAV equipped with an IMU, pressure sensor, monocular camera (running PTAM), and a GPS receiver. To ensure rigorous verification, the researchers used a Leica TotalStation 15i, a high-end surveying tool providing sub-millimeter precision, to establish ground truth. Experiments involved outdoor flights over trajectories exceeding 800 meters at speeds up to 4 m/s.

Experimental Results

The MSF-EKF demonstrated remarkable robustness and accuracy. A key insight from the study was the treatment of visual SLAM: by treating vision as a relative measurement rather than an absolute one, the system allows uncertainty to grow naturally, enabling GPS to correct long-term drift effectively.

3D trajectory comparison between the MSF-EKF estimate and ground truth.

The system successfully handled sensor outages and corrected for visual scale errors (often 3-5%) by fusing pressure and IMU data. Computationally, the framework proved highly efficient, with measurement updates taking approximately 65 microseconds and state re-propagation requiring only 21 microseconds.

Conclusion

The MSF-EKF framework provides a foundational tool for resilient autonomous systems. While the researchers noted that initialization of new sensors still requires specific drone movements to calibrate certain states, the modularity and efficiency of the system represent a significant step forward. Future work aims to integrate key-frame based SLAM more tightly to further reduce temporal drift and to open-source the framework for the broader robotics community.

Summary updated

Abstract

It has been long known that fusing information from multiple sensors for robot navigation results in increased robustness and accuracy. However, accurate calibration of the sensor ensemble prior to deployment in the field as well as coping with sensor outages, different measurement rates and delays, render multi-sensor fusion a challenge. As a result, most often, systems do not exploit all the sensor information available in exchange for simplicity. For example, on a mission requiring transition of the robot from indoors to outdoors, it is the norm to ignore the Global Positioning System (GPS) signals which become freely available once outdoors and instead, rely only on sensor feeds (e.g., vision and laser) continuously available throughout the mission. Naturally, this comes at the expense of robustness and accuracy in real deployment. This paper presents a generic framework, dubbed MultiSensor-Fusion Extended Kalman Filter (MSF-EKF), able to process delayed, relative and absolute measurements from a theoretically unlimited number of different sensors and sensor types, while allowing self-calibration of the sensor-suite online. The modularity of MSF-EKF allows seamless handling of additional/lost sensor signals during operation while employing a state buffering scheme augmented with Iterated EKF (IEKF) updates to allow for efficient re-linearization of the prediction to get near optimal linearization points for both absolute and relative state updates. We demonstrate our approach in outdoor navigation experiments using a Micro Aerial Vehicle (MAV) equipped with a GPS receiver as well as visual, inertial, and pressure sensors.
BibTeX
@inproceedings{lynen2013robust,
  author = {Simon Lynen and Markus W. Achtelik and Stephan Weiss and Margarita Chli and Roland Siegwart},
  title = {{A robust and modular multi-sensor fusion approach applied to MAV navigation}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2013},
  url = {https://openreview.net/forum?id=d9EpLbYRVR},
}
Figure