Turnpike in Lipschitz-nonlinear optimal control

Carlos Esteve, Borjan Geshkovski, Dario Pighin, Enrique ZuazuaarXiv 2020 · arXiv

Introduction

The turnpike property is a fundamental phenomenon in optimal control where the optimal solution for a long-horizon task spends the majority of its duration near a static steady state. This behavior is analogous to a long-distance journey where the most efficient route involves quickly reaching a highway (the turnpike), staying on it for the bulk of the trip, and exiting only near the destination.

Historically, proving this property for nonlinear systems required two restrictive conditions: "smallness" (initial data and targets must be close to equilibrium) and high-order smoothness ($C^2$ dynamics). These requirements pose significant hurdles for modern machine learning, where architectures like Residual Networks (ResNets) and Neural ODEs often use non-smooth activation functions (e.g., ReLU) and process large-scale data. This paper addresses this gap by establishing a global turnpike result for systems with globally Lipschitz nonlinearities.

Methodology: Variational Strategy and Bootstrap Argument

The authors depart from traditional methods that analyze the optimality system derived from the Pontryagin Maximum Principle. Instead, they employ a purely variational strategy based on:

  1. Quasi-turnpike Controls: The construction of a suboptimal control that explicitly steers the system to the target steady state, maintains it there, and then steers it to the final destination.
  2. Bootstrap Argument: Since the optimal control must outperform any suboptimal construction, the authors use bounds from the "quasi-turnpike" case to bootstrap an exponential decay estimate for the true optimal trajectory.
  3. Controllability Estimates: The proof relies on the system being controllable with a cost proportional to the distance from the target.

This flexible methodology applies to both finite-dimensional ODEs and infinite-dimensional PDEs, such as semilinear wave and heat equations.

Key Theoretical Insights

The research demonstrates that the turnpike property is more robust than previously thought, requiring only global Lipschitz continuity rather than $C^2$ smoothness. By avoiding linearization techniques, the authors prove that the property holds regardless of the initial state's distance from the target, provided sufficient controllability.

A central result is the "double-arc" estimate for the optimal state $y_T(t)$, which characterizes the "entry" and "exit" phases of turnpike behavior. Furthermore, for "driftless" systems common in machine learning, the authors prove that the optimal control $u(t)$ itself decays exponentially toward zero.

State stabilization and control decay

Numerical Validation: Neural ODEs and ResNets

The practical implications for deep learning are explored through a binary classification task. Using a synthetic dataset of 2400 points in $\mathbb{R}^3$, the authors model a continuous-time ResNet (Neural ODE) with a tanh nonlinearity.

Evolution of the binary classification task

The experiments confirm that trajectories quickly stabilize toward steady states, and the network weights (controls) effectively vanish after a certain depth. This suggests that in deep architectures, adding layers beyond a certain point leads to diminishing returns as the network simply stabilizes around a steady state. The implementation is available at github.com/borjanG/dynamical.systems.

Limitations and Future Directions

While providing a significant advancement, the paper identifies several avenues for future work:

  • Controlled Steady States: Extending the proof to scenarios where a constant non-zero control is required to maintain the steady state.
  • Controllability Assumptions: Refining results for underactuated nonlinear systems where current controllability assumptions might not hold.
  • Non-Lipschitz Nonlinearities: Addressing systems with superlinear growth, which present challenges for obtaining uniform trajectory bounds.
  • State Constraints: Incorporating hard constraints on states or controls, which are vital for robotics and safety-critical applications.

Summary updated

BibTeX
@misc{esteve2020turnpike,
  author = {Carlos Esteve and Borjan Geshkovski and Dario Pighin and Enrique Zuazua},
  title = {{Turnpike in Lipschitz-nonlinear optimal control}},
  year = {2020},
  eprint = {2011.11091},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2011.11091},
}
Figure