Asynchronous Sensor Fusion for Robust State Estimation

This direction is merged into another direction: sensor fusion with EKF. Its last version stays readable and citable here.

Version 1 of 136 papers39 conceptsfrozen 2026-10-04

For an autonomous robot to navigate, it must answer a fundamental question: "Where am I right now?" In a perfect world, a single sensor would provide high-frequency, perfectly accurate position data. In reality, robots rely on a "sensor cocktail" where each ingredient is flawed. Some sensors are fast but drift over time; others are accurate but slow or prone to intermittent failure.

The core research direction across these papers is the development of asynchronous sensor fusion frameworks—primarily based on the Extended Kalman Filter (EKF)—that reconcile these conflicting data streams to provide a continuous, high-frequency estimate of a robot's state.

The Multi-Rate Dilemma

The central challenge in state estimation is the mismatch in sensor frequencies and latencies.

  1. High-Frequency, Relative Sensors: Inertial Measurement Units (IMUs) and wheel encoders provide data at hundreds of Hertz. They are excellent for capturing sudden movements, but because they measure changes (acceleration or rotation), their errors accumulate. This is known as "dead reckoning" drift.
  2. Low-Frequency, Absolute Sensors: Cameras, LiDAR, and GPS provide "global" or "landmark-based" references that do not drift. However, processing a camera frame to find a whale Willaim Selby or a racing gate Guido De Croon is computationally expensive, introducing significant latency.

As Jesús Pestana and Thomas Holzmann demonstrate in Graz Griffins ’ Solution to the European Robotics Challenges 2014, a micro aerial vehicle (MAV) cannot wait 100 milliseconds for a visual update to decide how to move; it would crash. The solution is a prediction-correction cycle.

The Prediction-Correction Cycle

The Extended Kalman Filter (EKF) acts as the bridge. It maintains a "belief" about the robot's state (position, velocity, orientation).

  • Prediction: Whenever high-frequency IMU data arrives, the filter "predicts" the next state. This keeps the control loop running at high speeds, allowing for stable hovering or aggressive maneuvers Willaim Selby.
  • Correction: When a delayed visual or LiDAR update finally arrives, the filter doesn't just accept it as the current position. Instead, it looks back at where the robot was when the image was captured, calculates the error, and "corrects" the current state estimate.

This mechanism allows systems like Swift to defeat human champions in drone racing Champion-level drone racing using deep reinforcement learning. By fusing visual-inertial odometry, the system maintains a high-frequency global pose even during high-speed turns where visual features might blur.

Robustness Through Redundancy

In "GPS-denied" environments—such as underwater Chensheng Cheng, inside highway tunnels Kyuwon Kim, or under dense crop canopies Arun Narenthiran Sivakumar—primary sensors often fail. A camera might be blinded by sun glint, or a LiDAR might see only repetitive, featureless walls.

The research shows that fusion provides graceful degradation. If the visual system loses track of a target, the EKF continues to propagate the state using the IMU and motion models. Carmen Scheidemann and colleagues at ETH Zurich illustrate this by using a "virtual leash" for quadruped robots, where occlusion-resistant radio-frequency (RF) signals bridge the gaps when high-precision visual tracking fails Obstacle-Avoidant Leader Following with a Quadruped Robot.

Incorporating Physical Constraints

Modern state estimation goes beyond simple sensor blending by incorporating the "laws of physics" as synthetic sensors. Wonkeun Youn demonstrates this with fixed-wing UAVs: by assuming the aircraft has near-zero "sideslip" during normal flight, this physical constraint can be fed into the EKF to bound velocity drift when GPS is lost Fuzzy Adaptive Attitude Estimation for a Fixed-Wing UAV.

Similarly, Ziming Ding in VID-Fusion treats external forces (like wind) not as noise to be filtered out, but as part of the dynamic model. By fusing visual and inertial data with the robot's own aerodynamic equations, the system can estimate external disturbances in real-time, leading to more accurate trajectory following.

Efficiency at the Edge

A recurring theme is the push for these complex fusion algorithms to run on "nano" hardware. Jonas Kühne and Michele Magno show that by optimizing the EKF to use rigid-body motion models and efficient visual feature tracking, high-precision navigation can be achieved on RISC-V processors consuming less than 68 mW Efficient and Accurate Downfacing Visual-Inertial Odometry.

Whether it is a drone tracking whales or a robot navigating a cornfield, the "magic" of the idea lies in the filter's ability to trust each sensor exactly as much as it should, exactly when it needs to, creating a whole that is significantly more reliable than the sum of its parts.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_asynchronous_sensor_fusion_for_robust_st_1997d051,
  title = {Asynchronous Sensor Fusion for Robust State Estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asynchronous-sensor-fusion-for-robust-state-estimation/rdr_1997d05163dd4403a42061119cd64488},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_asynchronous_sensor_fusion_for_robust_st_1997d051_v1,
  title = {Asynchronous Sensor Fusion for Robust State Estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asynchronous-sensor-fusion-for-robust-state-estimation/rdr_1997d05163dd4403a42061119cd64488/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}