Relaxation and asymptotic expansion of controlled stiff differential equations

Michaël Herty, Hicham Kouhkouh

TL;DR

This paper provides a rigorous mathematical framework for optimizing "stiff" systems—dynamical models where some components change much faster than others. By applying asymptotic expansions to the Hamilton-Jacobi-Bellman (HJB) equation, the authors demonstrate how to simplify complex, high-frequency control problems into manageable "reduced" models that remain accurate even as the system's stiffness increases.

What Problem Does This Paper Address?

In many physical systems—ranging from traffic flow and granular gases to shallow water waves—the underlying dynamics are "stiff." This means the system contains processes operating on vastly different timescales. Mathematically, this is represented by a small parameter $\epsilon$ (the stiffness parameter). When $\epsilon$ is very small, standard numerical methods for optimal control become unstable or computationally ruinous.

While researchers have extensively studied how to simulate these systems, controlling them optimally is a much steeper challenge. Most existing approaches rely on Pontryagin’s Maximum Principle, which focuses on individual trajectories. However, this paper tackles the problem through the lens of the Hamilton-Jacobi-Bellman (HJB) equation. The HJB approach is powerful because it provides a "value function"—a map of the optimal cost from any given state—but it is notoriously difficult to solve for stiff, high-dimensional systems. The authors seek to bridge this gap by finding a way to approximate the value function using simpler, non-stiff equations.

Methods

The authors employ a combination of singular perturbation theory and multi-scale analysis:

  1. Singular Perturbation Recasting: They first transform the stiff system into a "slow-fast" structure. The "slow" variables represent the macroscopic behavior we care about, while the "fast" variables capture the high-frequency oscillations or relaxations.
  2. HJB Asymptotic Expansion: Instead of solving the stiff HJB equation directly, they propose an ansatz (an educated guess) for the value function: $V^\epsilon = V_0 + \epsilon V_1 + \epsilon^2 V_2 + \dots$.
  3. The "Cell Problem": To find the correction terms ($V_1, V_2$), they solve what is known as a "cell problem." This is a localized HJB equation that describes how the fast dynamics behave when the slow variables are "frozen."
  4. Cascaded Multi-Scale Approach: For systems with more than two timescales (e.g., macro, meso, and micro scales), the authors introduce a cascaded method. They solve a sequence of HJB equations, passing the results from the fastest scale up to the slowest.

Datasets

Not discussed in the paper. As a theoretical mathematics paper, it focuses on proofs and analytical derivations rather than empirical datasets. The "evaluation" is performed through rigorous mathematical verification and application to well-known physical models.

Key Insights and Ideas

  • Robust Feedback Design: The primary "aha moment" is that the asymptotic expansion allows for the design of feedback controls that are naturally robust to stiffness. By knowing $V_1$, a controller can account for first-order "fast" effects without needing to simulate them in real-time.
  • Convergence to Reduced Dynamics: The authors prove that as stiffness goes to infinity ($\epsilon \to 0$), the optimal control of the complex system converges to the optimal control of a much simpler "reduced" system.
  • Universality of Relaxation: The paper shows that this framework isn't just for a niche set of ODEs; it applies to any system derived from the semi-discretization of kinetic or hyperbolic PDEs, such as the Jin-Xin relaxation scheme.

Key Results

  • Theorem 3.1 (First-Order Convergence): The authors rigorously prove that the value function $V^\epsilon$ converges locally uniformly to $V_0$, the solution of the reduced HJB equation.
  • Theorem 4.1 (Three-Scale Expansion): They successfully extend the theory to three scales, providing a systematic way to derive second-order approximations ($V_2$).
  • Application Examples: The paper demonstrates the utility of the theory by applying it to several classic models:
  • Jin-Xin Relaxation: A standard model for numerical hyperbolic conservation laws.
  • Goldstein-Taylor Model: Used in particle physics and transport theory.
  • Traffic Flow: Specifically, second-order models that account for driver anticipation.
  • Granular Gases: Modeling the cooling and dynamics of particles with inelastic collisions.

Limitations and Future Work

  • Stability Requirements: The theory relies on a "stability condition" (Assumption a.6), requiring the fast dynamics to have negative eigenvalues. If the fast system is unstable or purely oscillatory, the expansion may fail.
  • Full-Rank Constraints: The control matrices must have full rank to ensure controllability, which might not hold in under-actuated systems.
  • Computational Complexity of Corrections: While $V_0$ is easy to compute, finding $V_1$ and $V_2$ still requires solving auxiliary PDEs, which can be challenging in very high dimensions.
  • Future Directions: The authors suggest extending this work from semi-discretized ODEs to fully continuous PDE control problems, which would further generalize the results for fluid dynamics and wave propagation.

Summary updated

Abstract

The control of relaxation-type systems of ordinary differential equations is investigated using the Hamilton–Jacobi–Bellman equation. Firstly, we recast the model as a singularly perturbed dynamics which we embed in a family of controlled systems. Then we study this dynamics together with the value function of the associated optimal control problem. We provide an asymptotic expansion in the relaxation parameter of the value function. We also show that its solution converges toward the solution of a Hamilton–Jacobi–Bellman equation for a reduced control problem. Such systems are motivated by semi-discretisation of kinetic and hyperbolic partial differential equations. Several examples are presented including Jin–Xin relaxation.
BibTeX
@article{herty2024relaxation,
  author = {Michaël Herty and Hicham Kouhkouh},
  title = {{Relaxation and asymptotic expansion of controlled stiff differential equations}},
  journal = {International Journal of Control},
  year = {2024},
  doi = {10.1080/00207179.2024.2346732},
}

Loading related content...

Figure