Accounts of using the Tustin-Net architecture on a rotary inverted pendulum

Stijn van Esch, Fabio Bonassi, Thomas B. SchönarXiv 2024 · DOI · arXiv

Introduction

Modeling the complex dynamics of a rotary inverted pendulum (RIP) is a fundamental challenge in control theory. Traditionally, researchers choose between two extremes: first-principles modeling based on Euler-Lagrange equations, which requires deep domain expertise, or black-box neural networks, which often lack physical consistency. This paper explores the Tustin-Net, a physics-informed architecture designed to bridge this gap by embedding physical constraints directly into the learning process.

The Modeling Challenge

Mechanical systems often harbor "hidden" complexities that are difficult to capture through standard physical derivations. For instance, in the Quanser Qube Servo 2 system, the authors discovered that a standard physical model performed poorly until they accounted for the nonlinear elasticity of an external encoder cable.

Schematic of the rotary inverted pendulum system, highlighting the physical parameters and the spring effect k1 of the encoder cable.

While first-principles models require such meticulous manual adjustments, standard neural networks like LSTMs may fail to respect the basic relationship between position and velocity, leading to physically impossible predictions over long time horizons.

The Tustin-Net Architecture

The Tustin-Net addresses these issues by hard-coding the integration step into the network's structure. It separates the state into position and velocity components:

  1. Position Dynamics: These are fixed using the Tustin (trapezoidal) method, ensuring that the next position is mathematically consistent with the predicted velocities.
  2. Velocity Dynamics: A feedforward neural network, $f(z; \Theta)$, predicts the increments in velocity based on current states and control inputs.
The Tustin-Net architecture, showing the integration of the neural network f(z; Theta) with trapezoidal integration steps.

Training Strategy: Algorithm 1

A key contribution of this work is a novel transfer learning strategy designed to handle the "unbalanced" nature of dynamic datasets. Since the pendulum spends significant time near equilibrium, standard training can lead to overfitting on static data. The proposed Algorithm 1 involves:

  • Pre-training: Training the network exclusively on "transient" data where the pendulum is in motion.
  • Freezing: Locking the hidden layer weights to preserve the learned physics.
  • Fine-tuning: Training only the output layer on the full dataset to refine the model's performance across all states.

Experimental Results

The Tustin-Net, when trained using Algorithm 1, demonstrated superior performance compared to both standard training methods and naive first-principles models. Measured in Root Mean Squared Error (RMSE), the results show a clear advantage for the physics-informed approach:

Model Free-fall RMSE
Euler-Lagrange (Expert) ~1.42
Tustin-Net (Standard) ~5.31
Tustin-Net (Algorithm 1) ~0.75

The results indicate that the physics-informed architecture, combined with a strategic training approach, can outperform expert-derived models without requiring explicit knowledge of specific hardware quirks like cable elasticity.

Conclusion

The Tustin-Net offers a robust middle ground for system identification. By enforcing physical consistency through trapezoidal integration and utilizing a transient-focused training strategy, it achieves high accuracy on real-world hardware. While first-principles models remain superior for extrapolation into unseen operational regions, Tustin-Nets provide a powerful tool for modeling complex systems where manual derivation is impractical.

Summary updated

Abstract

In this report we investigate the use of the Tustin neural network architecture (Tustin-Net) for the identification of a physical rotary inverse pendulum. This physics-based architecture is of particular interest as it builds on the known relationship between velocities and positions. We here aim at discussing the advantages, limitations and performance of Tustin-Nets compared to first-principles grey-box models on a real physical apparatus, showing how, with a standard training procedure, the former can hardly achieve the same accuracy as the latter. To address this limitation, we present a training strategy based on transfer learning that yields Tustin-Nets that are competitive with the first-principles model, without requiring extensive knowledge of the setup as the latter.
BibTeX
@misc{esch2024accounts,
  author = {Stijn van Esch and Fabio Bonassi and Thomas B. Schön},
  title = {{Accounts of using the Tustin-Net architecture on a rotary inverted pendulum}},
  year = {2024},
  eprint = {2408.12266},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.12266},
}
Figure