Multi-Rate Sensor Fusion with Extended Kalman Filters

346 concepts292 papers1 proposal

Imagine navigating a dark, unfamiliar room. Your inner ear and muscle memory (proprioception) give you an immediate, high-frequency sense of how fast you are turning or stepping. However, if you rely on this internal tracking alone, small errors quickly compound, causing you to drift off course. To correct yourself, you periodically open your eyes or reach out to touch a familiar piece of furniture (exteroception).

Robots face this exact dilemma. High-frequency proprioceptive sensors, such as Inertial Measurement Units (IMUs) and wheel encoders, provide low-latency motion data but suffer from unbounded drift over time. Conversely, exteroceptive sensors, such as GPS, LiDAR, and cameras, offer absolute positioning but operate at much lower frequencies and introduce significant processing latencies.

The Extended Kalman Filter (EKF) serves as the mathematical framework that bridges these asynchronous sensor streams. By treating state estimation as a continuous process of prediction punctuated by discrete corrections, multi-rate EKF-based sensor fusion allows autonomous systems to maintain real-time, drift-free localization.


The Predict-and-Correct Cycle

The core intuition behind an EKF is its two-phase execution loop: prediction and correction. This architecture naturally maps to the multi-rate reality of robotic hardware.

 [High-Rate Proprioception] ---> ( Prediction Step ) ---> State Estimate (High Frequency, Drifting)
                                       |
                                       v
 [Low-Rate Exteroception]  ---> ( Correction Step ) ---> Refined Estimate (Drift Reset)
  1. The Prediction Step (High Frequency): Driven by fast, noisy inputs like IMUs or wheel speeds, the filter integrates physical equations of motion to project the robot's state forward in time. Because these sensors are noisy, the filter's uncertainty (covariance) grows with every prediction step. For example, in autonomous driving, Jiang et al. utilize high-frequency wheel speed and IMU data within an EKF to continuously project a vehicle's position along a road corridor (Estimation and prediction of vehicle dynamics states based on fusion of OpenStreetMap and vehicle dynamics models).
  2. The Correction Step (Low Frequency): When a slower exteroceptive measurement arrives—such as a camera frame or a LiDAR scan—the filter calculates an innovation. The innovation is the difference between what the sensor actually observes and what the filter expected it to observe based on the current prediction. The EKF weights this innovation against the current uncertainty to update the state, effectively snapping the robot back to reality and shrinking the uncertainty bounds.

Shen et al. demonstrated the necessity of this multi-rate synergy in quadrotor flight, fusing high-rate inertial data with lower-rate visual pose estimates (Vision-Based State Estimation and Trajectory Control Towards High-Speed Flight with a Quadrotor). The high-frequency prediction loop provides the low-latency feedback required by flight controllers to maintain stability during aggressive maneuvers, while the lower-frequency visual updates eliminate the long-term drift of the IMU. Similarly, Ort et al. combined discrete ground-penetrating radar updates with continuous wheel encoders to maintain stable state estimation through inclement weather (Autonomous Navigation in Inclement Weather Based on a Localizing Ground Penetrating Radar).


Tightly-Coupled Fusion and State Augmentation

A naive approach to sensor fusion might involve running separate tracking algorithms for each sensor and averaging their outputs. This is known as loosely-coupled fusion. However, if an IMU develops a temperature-induced bias, a loosely-coupled system cannot easily deduce which sensor is malfunctioning.

Modern architectures prefer tightly-coupled fusion via state augmentation. Instead of tracking only the robot's position and velocity, the EKF's internal state vector is expanded to include hidden parameters, such as sensor biases, calibration offsets, or the locations of external landmarks.

When a correction step occurs, the EKF uses the cross-covariances between the robot's position and these hidden parameters to back-propagate information. If the robot's predicted position consistently overshoots the visual landmarks, the filter doesn't just adjust the current position; it refines its estimate of the IMU's internal bias.

This mechanism is central to TLIO: Tight Learned Inertial Odometry, where Liu et al. integrated learned motion displacement vectors and their predicted uncertainties directly as EKF measurement updates. This tightly-coupled architecture allows the filter to constantly refine internal sensor biases, yielding highly accurate 3D tracking from a single IMU.

State augmentation can also bind external infrastructure into the estimation loop. Meger designed a joint state estimation framework that tracks both a mobile robot's pose and the 6-DoF poses of multiple static cameras simultaneously (Simultaneous Planning, Localization, and Mapping in a Camera Sensor Network). By incorporating both robot odometry and visual detections into a single augmented state vector, the system maintains a unified global coordinate frame without external calibration.


Managing Non-Linearities and Moving Reference Frames

The standard Kalman Filter assumes that the physics governing a system are strictly linear. However, robotic rotation and sensor geometry are inherently non-linear. The Extended Kalman Filter handles this by computing a first-order Taylor series approximation (using Jacobians) to linearize the system around the current state estimate.

When non-linearities are severe or sensor streams are highly irregular, standard linearization can introduce errors. In these scenarios, researchers deploy advanced variants:

  • The Unscented Kalman Filter (UKF): Instead of linearizing the equations, the UKF passes a carefully chosen set of sample points (sigma points) through the true non-linear equations, capturing the mean and covariance more accurately. This approach was critical to the success of Stanley, the autonomous vehicle that won the DARPA Grand Challenge; Montemerlo et al. utilized a UKF to reliably fuse asynchronous data streams from IMUs, wheel encoders, and GPS during extended periods of satellite signal loss (Winning the DARPA Grand Challenge with an AI Robot).
  • The Invariant EKF (InEKF): Standard EKFs can suffer from false observability, where the filter mistakenly believes it has gained information about unobservable states (like absolute heading) due to linearization errors. The InEKF solves this by ensuring that the error propagation equations are invariant to state transformations. He et al. leveraged an Invariant EKF to solve the problem of a legged robot operating inside a non-inertial reference frame, such as a moving ship (Legged Robot State Estimation within Non-inertial Environments). By fusing the moving platform's inertial data with the robot's kinematics, the relative pose and velocity become fully observable without requiring external cameras.

By structuring state estimation around a recursive predict-and-correct architecture, multi-rate Kalman filters transform a collection of imperfect, asynchronous sensors into a single, highly accurate, and low-latency stream of truth.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sensor_fusion_with_ekf_ba72cb06,
  title = {sensor fusion with EKF},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensor-fusion-with-ekf/rdr_ba72cb068fbe4cf4a083a71a8921813a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_sensor_fusion_with_ekf_ba72cb06_v1,
  title = {sensor fusion with EKF},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensor-fusion-with-ekf/rdr_ba72cb068fbe4cf4a083a71a8921813a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake