Adaptive Control of SE(3) Hamiltonian Dynamics With Learned Disturbance Features

Thai P. Duong, Nikolay Atanasov

TL;DR

This paper introduces a novel adaptive control framework for robots operating in complex environments with unknown disturbances. By learning disturbance patterns offline using a Hamiltonian-based neural ODE network and then compensating for them online using geometric tracking errors, the system achieves significantly improved trajectory tracking performance compared to traditional methods.

What Problem Does This Paper Address?

Reliable robot autonomy in dynamic environments hinges on accurate system models. However, real-world conditions often introduce unpredictable disturbances that deviate from these models. While adaptive control can compensate for such uncertainties, it typically requires a known model of these disturbances. This paper tackles the challenge of designing adaptive controllers for rigid-body systems (like ground, aerial, and underwater vehicles) that move on the SE(3) manifold (representing position and orientation), where the disturbance characteristics are unknown and must be learned. The core problem is to develop a control strategy that can effectively learn these unknown disturbance features from data and then use this learned knowledge to adapt and maintain accurate trajectory tracking online.

Methods

The proposed approach consists of two main stages:

  1. Offline Disturbance Feature Learning:
    • Hamiltonian-based Neural ODE Network: The authors leverage a neural ordinary differential equation (ODE) network that is structured to respect the Hamiltonian dynamics of the system. This network is trained on state-control trajectory data collected under various disturbance conditions. The goal is to learn a general representation of disturbance features, denoted by $W(x)$, and specific weights $a^*$ for each disturbance realization.
  2. Problem Formulation: A specific optimization problem (Problem 1) is defined to learn the disturbance features and weights by minimizing the difference between predicted and actual system states, with regularization terms to prevent overfitting.
  3. Training: Gradient descent is used to update the parameters of the neural ODE network and the disturbance weights, backpropagating the loss through the ODE solver using adjoint states.
  4. Online Adaptive Control Stage:
    • Geometric Adaptive Control: A passivity-based tracking controller is designed for SE(3) Hamiltonian dynamics. This controller incorporates an energy-shaping term, a damping-injection term, and crucially, a disturbance compensation term.
  5. Adaptation Law: An adaptation law is developed to estimate the unknown disturbance weights online. This law utilizes geometric tracking errors (position, orientation, and velocity errors on the SE(3) manifold) to update the disturbance estimates.
  6. Controller Structure: The overall control input $u$ is a combination of these three terms: $u = u_{ES} + u_{DI} + u_{DC}$. The disturbance compensation term $u_{DC}$ directly uses the online estimate of the disturbance.
  7. Stability Guarantee: The paper provides a theoretical guarantee (Theorem 1) that under certain conditions, the tracking errors and disturbance estimation errors converge to zero or are uniformly bounded.

Datasets

  • Training Data: The paper uses simulated state-control trajectory data. For the disturbance learning stage, this data is collected from systems experiencing different, unknown disturbance realizations. The paper mentions that such data can be obtained from odometry algorithms, motion capture systems, or by manually controlling the robot under various disturbance conditions.
  • Evaluation Data:
    • Pendulum: Simulated data for a fully-actuated pendulum with a friction disturbance.
  • Quadrotor: Simulated data for an under-actuated quadrotor using the PyBullet physics engine, incorporating disturbances like wind, rotor failures, and near-ground effects.

Key Insights and Ideas

  • Learning Disturbance Features on SE(3): A significant contribution is the ability to learn disturbance features directly on the SE(3) manifold, which is crucial for robots with rotational dynamics. This moves beyond simpler Euclidean space assumptions.
  • Hamiltonian-based Neural ODEs for Dynamics Learning: Using neural ODEs that respect the underlying physical structure (Hamiltonian dynamics) is a powerful way to learn system dynamics and disturbances, leading to more robust and interpretable models.
  • Geometric Tracking Errors for Adaptation: Employing geometric errors (rather than just Cartesian errors) for the adaptation law is key to achieving stable and effective online compensation for disturbances on manifolds.
  • Bridging Offline Learning and Online Adaptation: The paper elegantly combines offline learning of general disturbance patterns with online adaptation to specific disturbance realizations, creating a flexible and powerful adaptive control system.
  • Demonstrated Performance Improvement: The results clearly show that learning disturbance features significantly outperforms methods that rely on generic disturbance observers without prior knowledge of the disturbance structure.

Key Results

  • Pendulum: The adaptive controller achieved significantly lower tracking errors (angle and velocity) and disturbance estimation errors compared to a disturbance observer (DOB) and a system without adaptation. The learned disturbance features allowed the controller to quickly adapt to changes in the ground-truth disturbance.
  • Quadrotor: For both diamond-shaped and spiral trajectories, the adaptive controller with learned disturbance features resulted in tracking errors that were approximately 5 times lower than those without adaptation. The controller successfully recovered from sudden changes in disturbances (e.g., rotor failures) that caused uncontrolled behavior in the non-adaptive system.

Limitations and Future Work

  • Limitations:
    • The current approach relies on offline learning, which requires collecting a diverse dataset of disturbance realizations. The performance might degrade if the online disturbances are drastically different from those encountered during offline training.
  • The paper assumes that the generalized mass matrix $M(q)$, potential energy $V(q)$, and input gain $g(q)$ are known.
  • The stability analysis is provided for specific conditions and domains.
  • Future Work:
    • Deploying the geometric adaptive controller on real UGV and UAV robot systems.
  • Exploring online learning of disturbance features or meta-learning approaches to adapt more rapidly to entirely new disturbance types.
  • Investigating the impact of different neural network architectures and ODE solvers on performance.
  • Extending the framework to more complex robotic systems and dynamics.

Summary updated

Abstract

Adaptive control is a critical component of reliable robot autonomy in rapidly changing operational conditions. Adaptive control designs benefit from a disturbance model, which is often unavailable in practice. This motivates the use of machine learning techniques to learn disturbance features from training data offline, which can subsequently be employed to compensate the disturbances online. This letter develops geometric adaptive control with a learned disturbance model for rigid-body systems, such as ground, aerial, and underwater vehicles, that satisfy Hamilton’s equations of motion over the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$SE{(}3{)}$ </tex-math></inline-formula> manifold. Our design consists of an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">offline disturbance model identification stage</i> , using a Hamiltonian-based neural ordinary differential equation (ODE) network trained from state-control trajectory data, and an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">online adaptive control stage</i> , estimating and compensating the disturbances based on geometric tracking errors. We demonstrate our adaptive geometric controller in trajectory tracking simulations of fully-actuated pendulum and under-actuated quadrotor systems.
BibTeX
@inproceedings{duong2022adaptive,
  author = {Thai P. Duong and Nikolay Atanasov},
  title = {{Adaptive Control of SE(3) Hamiltonian Dynamics With Learned Disturbance Features}},
  booktitle = {IEEE Control. Syst. Lett},
  year = {2022},
  url = {https://openreview.net/forum?id=9NkEsHis0oL},
}
Figure