Differentiable Moving Horizon Estimation for Robust Flight Control

Bingheng Wang, Zhengtian Ma, Shupeng Lai, Lin Zhao, Tong Heng Lee

Abstract

Estimating and reacting to external disturbances is of fundamental importance for robust control of quadrotors. Existing estimators typically require significant tuning or training with a large amount of data, including the ground truth, to achieve satisfactory performance. This paper proposes a data-efficient differentiable moving horizon estimation (DMHE) algorithm that can automatically tune the MHE parameters online and also adapt to different scenarios. We achieve this by deriving the analytical gradient of the estimated trajectory from MHE with respect to the tuning parameters, enabling end-to-end learning for auto-tuning. Most interestingly, we show that the gradient can be calculated efficiently from a Kalman filter in a recursive form. Moreover, we develop a model-based policy gradient algorithm to learn the parameters directly from the trajectory tracking errors without the need for the ground truth. The proposed DMHE can be further embedded as a layer with other neural networks for joint optimization. Finally, we demonstrate the effectiveness of the proposed method via both simulation and experiments on quadrotors, where challenging scenarios such as sudden payload change and flying in downwash are examined.

Introduction

Quadrotors are agile aerial vehicles but remain highly sensitive to external disturbances such as wind gusts, aerodynamic ground effects, and sudden payload changes. To maintain stability and precision, flight controllers must accurately estimate these forces in real-time. Traditional estimation techniques, such as the Extended Kalman Filter (EKF) or Moving Horizon Estimation (MHE), rely heavily on weight matrices and noise covariance parameters that are notoriously difficult to tune manually. Furthermore, while deep learning approaches can predict disturbances, they typically require "ground truth" force data, which is rarely available in real-world flight scenarios.

This work introduces Differentiable Moving Horizon Estimation (DMHE), a framework that enables quadrotors to automatically tune their disturbance estimators online by learning directly from trajectory tracking errors, bypassing the need for hard-to-obtain ground truth data.

Methodology

The core innovation of DMHE is making the MHE optimization process differentiable, allowing it to function as a layer within a larger learning pipeline.

Analytical Gradients and the Kalman Connection

The authors derive the analytical gradient of the MHE output with respect to its tuning parameters ($\theta$) using Karush-Kuhn-Tucker (KKT) optimality conditions. A key mathematical insight of this paper is that calculating this gradient is equivalent to solving an auxiliary linear MHE problem. Crucially, this auxiliary problem can be solved efficiently in a recursive form using a Kalman Filter, ensuring that the "backward pass" for learning is computationally feasible for real-time robotics.

The DMHE framework architecture, illustrating the flow from MHE solving to gradient calculation via an auxiliary MHE block.

Model-Based Policy Gradient

Instead of requiring ground truth disturbance measurements, DMHE employs a model-based policy gradient algorithm. It defines a loss function based on the drone's tracking error—the deviation from its intended path. If the tracking error increases, the gradient provides the necessary information to adjust the MHE parameters, improving the disturbance estimate and allowing the controller to compensate more effectively.

Experimental Results

The DMHE framework was evaluated through high-fidelity simulations and real-world experiments on a 1.8kg quadrotor equipped with an Intel NUC and Pixhawk flight controller.

Performance and Accuracy

In payload release tests, DMHE achieved remarkable accuracy, estimating a force of -5.71N compared to a ground truth of -5.87N, despite never having seen ground truth data during training. The algorithm is also highly efficient; for a 20-step horizon, the gradient calculation takes only 5.6ms on onboard hardware, allowing for high-frequency control loops.

Experimental results: (a) convergence of mean loss across training episodes, (b) tracking performance improvements, and (c) comparison of disturbance estimation against ground truth.

Robustness to Downwash

The system demonstrated significant robustness in challenging environments. When subjected to "downwash" (turbulence from another drone), the DMHE-enhanced controller maintained altitude with minimal oscillation, whereas standard baseline controllers suffered instability and significant altitude loss of over 1 meter.

Conclusion

DMHE bridges the gap between classical control theory and modern machine learning. By treating the estimator as a differentiable layer, the system gains the ability to self-tune based on performance rather than requiring expert manual intervention or labeled datasets. Future work aims to integrate DMHE into Simultaneous Localization and Mapping (SLAM) systems to further enhance autonomous flight in unknown, dynamic environments.

Summary updated

Abstract

Estimating and reacting to external disturbances is of fundamental importance for robust control of quadrotors. Existing estimators typically require significant tuning or training with a large amount of data, including the ground truth, to achieve satisfactory performance. This paper proposes a data-efficient differentiable moving horizon estimation (DMHE) algorithm that can automatically tune the MHE parameters online and also adapt to different scenarios. We achieve this by deriving the analytical gradient of the estimated trajectory from MHE with respect to the tuning parameters, enabling end-to-end learning for auto-tuning. Most interestingly, we show that the gradient can be calculated efficiently from a Kalman filter in a recursive form. Moreover, we develop a model-based policy gradient algorithm to learn the parameters directly from the trajectory tracking errors without the need for the ground truth. The proposed DMHE can be further embedded as a layer with other neural networks for joint optimization. Finally, we demonstrate the effectiveness of the proposed method via both simulation and experiments on quadrotors, where challenging scenarios such as sudden payload change and flying in downwash are examined.
BibTeX
@misc{wang2021differentiable,
  author = {Bingheng Wang and Zhengtian Ma and Shupeng Lai and Lin Zhao and Tong Heng Lee},
  title = {{Differentiable Moving Horizon Estimation for Robust Flight Control}},
  year = {2021},
  eprint = {2108.03212},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2108.03212},
}
Figure