Online convex optimization for robust control of constrained dynamical systems

Marko Nonhoff, Emiliano Dall'Anese, Matthias A. Müller

This paper, recently published in IEEE Transactions and Journals, tackles a fundamental challenge in modern robotics and automation: how do we control a system to achieve a goal that is constantly changing and unknown in advance, all while ensuring it never breaks safety rules in a noisy environment?

TL;DR

The authors present a novel control algorithm that merges Online Convex Optimization (OCO) with Robust Model Predictive Control (MPC) to handle time-varying goals and exogenous disturbances. Their framework is the first of its kind to provide rigorous guarantees for both robust constraint satisfaction and bounded "dynamic regret"—meaning the system stays safe and performs nearly as well as if it knew the future.

What Problem Does This Paper Address?

In classical control, we usually design a system to reach a fixed target or follow a pre-defined path. However, real-world applications like autonomous driving or smart grids are far more chaotic. The "best" thing to do might change every second (time-varying cost functions), and the system is constantly buffeted by wind, road friction, or sensor errors (disturbances and measurement noise).

While the field of Online Convex Optimization (OCO) provides great tools for "learning" how to act when goals are unknown, it often struggles with hard safety constraints—like "don't drive off the road." Existing OCO-based controllers either ignore noise or can only guarantee safety in a perfect, noise-free world. This paper fills that gap by creating a controller that is both "smart" (it learns the changing goal) and "safe" (it respects physical limits even when sensors are noisy).

Methods

The authors propose Algorithm 1: Robust OCO for control. The core innovation lies in how it balances performance with safety:

  1. Constraint Tightening: Borrowing from robust MPC, the algorithm doesn't just aim for the edge of the safety zone. It calculates a "buffer" based on the worst-case noise and disturbances, effectively shrinking the allowed operating space to ensure that even a sudden gust of wind won't push the system out of bounds.
  2. $\mu$-step Ahead Prediction: The controller predicts the system's state several steps into the future to anticipate how current actions will affect future safety.
  3. Online Gradient Descent (OGD): To handle the unknown, changing goals, the algorithm uses OGD to iteratively update its estimate of the "optimal" state based on the costs it observed in the previous step.
  4. Safe Scaling ($\beta_t$): If the OGD step suggests a move that might be risky, a scaling factor $\beta_t$ is used to "dial back" the action, ensuring the resulting trajectory stays within the tightened safe zone.
Schematic illustration of the algorithm showing how it predicts future states and adjusts its path to stay within tightened constraints.

Datasets

The authors evaluated their approach using a numerical case study of an autonomous vehicle.

  • Scenario: A car traveling at 120 km/h on a two-lane road encounters a slower vehicle (70 km/h). The car must decide whether to slow down or overtake while staying within lane boundaries and respecting acceleration limits.
  • Setup: The system was subjected to uniform measurement noise for position and velocity, and exogenous disturbances representing model mismatches.

Key Insights and Ideas

  • Safety-Performance Decoupling: By using constraint tightening, the authors decouple the "safety" problem (handled by the buffer) from the "optimization" problem (handled by OGD). This allows the controller to be aggressive in pursuing goals without risking a crash.
  • Linear Regret Bounds: The authors mathematically prove that the "dynamic regret"—the gap between their controller and an omniscient one that knows the future—grows only linearly with the variation of the cost functions and the magnitude of the noise. This is a "gold standard" result in OCO.
  • Handling Measurement Noise: Unlike many previous works that only consider disturbances to the system's movement, this framework explicitly accounts for sensor inaccuracies ($v_t$), making it much more applicable to real-world hardware.

Key Results

The simulation results demonstrate that the controller is highly effective:

  • Robust Safety: In the overtaking scenario, the car successfully stays within lane boundaries and maintains a safety distance of ~55m from the slower vehicle, despite significant sensor noise (see Figures 3 and 5).
  • Effective Tracking: The car successfully tracks a velocity reference that sits right on the edge of its physical constraints, reaching approximately 124 km/h when the limit is 130 km/h due to the safety buffers (Figure 4).
  • Optimization vs. Explicit Solutions: The authors compared a version that optimizes the path ($g_t$) versus a simpler explicit version. The optimization-based version was smoother and more decisive during the overtaking maneuver (Figure 6).

Lateral position tracking showing the car successfully changing lanes to overtake while staying within road bounds. Velocity tracking showing the car slowing down for the obstacle and then accelerating to overtake, respecting the red constraint line.

Limitations and Future Work

While powerful, the current framework has a few boundaries:

  • Linear Systems: The theoretical proofs are currently limited to Linear Time-Invariant (LTI) systems. While they applied it to a linearized car model, extending this to fully nonlinear systems is a key next step.
  • Steady-State Benchmark: The performance is measured against the "optimal steady state." Comparing it against the absolute optimal trajectory in hindsight is a much harder problem that remains open.
  • Computational Load: While OCO is generally low-complexity, the addition of robust constraint tightening and multi-step predictions adds overhead that needs to be managed for high-speed applications.

Summary updated

Abstract

This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study on a tracking control problem of an autonomous vehicle.
BibTeX
@article{nonhoff2026online,
  author = {Marko Nonhoff and Emiliano Dall'Anese and Matthias A. Müller},
  title = {{Online convex optimization for robust control of constrained dynamical systems}},
  journal = {IEEE Trans. Automatic Control},
  year = {2026},
  doi = {10.1109/tac.2025.3641731},
}
Figure