Online convex optimization for robust control of constrained dynamical systems

Marko Nonhoff, Emiliano Dall'Anese, Matthias Albrecht Müller

Introduction

In modern control applications like autonomous driving or smart grids, systems must navigate environments where the "cost" of an action is often unknown and time-varying. While Online Convex Optimization (OCO) is a powerful tool for handling such uncertainty, it traditionally struggles with two critical aspects of physical systems: complex system dynamics and hard safety constraints. This paper introduces a novel framework that integrates OCO with Robust Model Predictive Control (MPC) to ensure that physical boundaries are never violated, even amidst disturbances and noisy sensor data.

The Challenge of Unknown Costs and Hard Constraints

Traditional control assumes a known cost function (e.g., minimizing fuel consumption). However, if an obstacle suddenly appears or energy prices fluctuate, the cost function changes unpredictably. Standard OCO algorithms typically allow for "soft" constraint violations, which is unacceptable in robotics or power systems where hitting a wall or overloading a circuit can be catastrophic. The authors address this by controlling a linear system facing unknown costs and external disturbances while guaranteeing strict adherence to safe operating limits.

Methodology: Robust OCO for Control

The proposed approach, Algorithm 1: Robust OCO for Control, merges MPC-style constraint tightening with the OCO loop. The process involves four key stages at each time interval:

  1. $\mu$-Step Prediction: Predicting the system's state several steps ahead based on historical inputs.
  2. Online Gradient Descent (OGD): Using the gradient of the previous cost function to estimate the current optimal steady state.
  3. Constraint Tightening: Shrinking the allowed state and input space to account for future uncertainty from disturbances and noise.
  4. Steering & Scaling: Calculating an input sequence to move toward the estimated optimal state. If a sequence risks violating a tightened constraint, a scaling factor ($\beta_t$) is applied to ensure safety.
Geometric representation of the Robust OCO algorithm, showing the interaction between gradient descent, constraint sets, and scaling.

Numerical Case Study: Autonomous Vehicle Tracking

The framework was evaluated through a simulation of an autonomous vehicle using nonlinear kinematics. The scenario required the car to maintain speed on a two-lane road, detect a slower vehicle ahead, and eventually overtake it. Random measurement noise was added to velocity and position sensors, and external disturbances were modeled to test the system's robustness.

Key Results and Insights

The authors provide a rigorous proof that the "dynamic regret"—the performance gap between their controller and a theoretical benchmark with perfect foresight—is bounded by the variation of the cost functions and the magnitude of disturbances. This links the learning performance of OCO with the stability requirements of control theory.

Velocity tracking performance showing the controller managing transitions between cruising at 120 km/h, slowing for a lead vehicle, and accelerating to overtake.

In simulations, the controller successfully managed complex maneuvers: cruising at 120 km/h, slowing to 70 km/h to maintain a 55m safety buffer, and then accelerating to 124 km/h to overtake. Crucially, the system respected all road boundaries and actuator limits despite the noisy sensor data. By utilizing Online Gradient Descent instead of solving large-scale optimization problems at every step, the controller remains computationally efficient for real-time use.

Limitations and Future Work

While the simulation demonstrated success with a nonlinear vehicle model, the formal theoretical proofs are currently limited to Linear Time-Invariant (LTI) systems. Future research aims to generalize these results to nonlinear or time-varying linear systems and explore more aggressive performance benchmarks, such as comparing against the optimal sequence of states in hindsight rather than just the optimal steady state.

Summary updated

Abstract

This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study of a tracking control problem of an autonomous vehicle.
BibTeX
@article{nonhoff2026online,
  author = {Marko Nonhoff and Emiliano Dall'Anese and Matthias Albrecht Müller},
  title = {{Online convex optimization for robust control of constrained dynamical systems}},
  journal = {IEEE Trans. Automatic Control},
  year = {2026},
  doi = {10.1109/tac.2025.3641731},
}
Figure