Online Control under Nonstochastic Disturbances

97 concepts72 papers1 proposal

Classical control theory relies on a comforting assumption: the world's randomness is polite. In frameworks like the Linear Quadratic Regulator (LQR) or Kalman filtering, environmental disturbances are typically modeled as independent, identically distributed (i.i.d.) Gaussian noise with a zero mean. Under these assumptions, engineers can derive elegant, closed-form optimal controllers.

However, real-world systems rarely experience such well-behaved noise. A drone navigating urban canyons encounters erratic wind gusts; an autonomous vehicle faces non-stationary sensor drift or strategic, adversarial interference. When disturbances are structured, correlated, or actively malicious, traditional stochastic control models degrade significantly.

To bridge this gap, a modern paradigm has emerged: nonstochastic control. By reframing control through the lens of Online Convex Optimization (OCO), this framework provides mathematical guarantees for systems facing arbitrary, worst-case disturbances without relying on probabilistic assumptions.

The Core Challenge: The Non-Convexity of State Feedback

To understand why nonstochastic control requires a fundamentally different approach, consider a standard linear dynamical system:

$$x_{t+1} = Ax_t + Bu_t + w_t$$

where $x_t$ is the state, $u_t$ is the control input, and $w_t$ is an arbitrary, potentially adversarial disturbance. The controller's goal is to minimize a sequence of convex cost functions $c_t(x_t, u_t)$ over time.

In classical control, we often look for a static state-feedback policy, setting $u_t = K x_t$. If we try to learn the optimal matrix $K$ online as costs and disturbances unfold, we immediately run into a mathematical wall. Because the state $x_t$ depends recursively on previous control actions, the cost $c_t(x_t, u_t)$ becomes highly non-convex with respect to the policy parameters $K$. In online learning, non-convex landscapes generally preclude sublinear regret guarantees, making it difficult to ensure the controller improves over time.

The Solution: Disturbance-Response Parameterization

The breakthrough that enabled nonstochastic control was a change of perspective in how we parameterize policies. Instead of choosing the control input based on the current state, what if we choose it based on the history of past disturbances?

As shown by Naman Agarwal et al. in Online Control with Adversarial Disturbances, we can stabilize the system using a nominal, fixed feedback matrix $K$, and then inject an adaptive component based on a finite window of past disturbances $w_t$:

$$u_t = K x_t + \sum_{i=1}^{H} M_i w_{t-i}$$

Here, the parameters we optimize online are the matrices $\mathcal{M} = \{M_1, M_2, \dots, M_H\}$.

Why does this change everything? Because the transfer mechanism from the past disturbances $w_{t-i}$ to the current state $x_t$ is linear. Consequently, the state $x_t$ and the control input $u_t$ become linear functions of the policy parameters $\mathcal{M}$. When you pass a linear function into a convex cost function $c_t$, the resulting objective remains strictly convex with respect to $\mathcal{M}$.

By shifting from state-feedback to disturbance-response parameterization, the authors successfully reduced a complex, stateful control problem into a standard Online Convex Optimization problem with memory.

Measuring Success via Regret Minimization

In an environment where disturbances can be chosen by an adversary, aiming for an absolute "optimal" strategy is impossible because the adversary can always adapt to defeat it. Instead, nonstochastic control evaluates performance using regret.

Regret measures the difference between the cumulative cost incurred by our online controller and the cost that would have been incurred by the best fixed policy chosen in hindsight:

$$\text{Regret} = \sum_{t=1}^T c_t(x_t, u_t) - \min_{\mathcal{M}} \sum_{t=1}^T c_t(x_t(\mathcal{M}), u_t(\mathcal{M}))$$

A controller is successful if its regret is sublinear (e.g., $\mathcal{O}(\sqrt{T})$), meaning that on average, the controller performs just as well as the best fixed policy as time goes on.

Elad Hazan et al. demonstrated in The Nonstochastic Control Problem that sublinear regret is achievable even when the underlying system dynamics are initially unknown. Further tightening these guarantees, Dylan J Foster et al. introduced the Riccatitron algorithm in Logarithmic Regret for Adversarial Online Control, proving that for strongly convex cost functions, one can achieve a logarithmic regret bound of $\mathcal{O}(\log T)$, matching the performance bounds typically reserved for purely stochastic environments.

Maintaining Safety in Adversarial Environments

In real-world deployments, minimizing costs is secondary to maintaining safety. If a drone encounters an unexpected, non-stochastic wind gust, it must adapt its policy without violating physical boundary constraints.

To handle this, researchers have integrated safety projections directly into the online learning loop. For instance, Hongyu Zhou et al. proposed Safe-OGD in Safe Non-Stochastic Control of Linear Dynamical Systems, which pairs online gradient descent with dynamic safety projections to guarantee zero safety violations under arbitrary noise.

However, projecting onto safety boundaries can be risky if a severe disturbance occurs exactly at the boundary. To create a defensive buffer, Nanfei Jiang et al. developed the OGD-BZC algorithm in Online Nonstochastic Control with Convex Safety Constraints. This approach systematically "shrinks" the allowed safety region during optimization:

[ True Safety Boundary ]
       |
       v  [ Buffer Zone ]
       +------------------+
       |                  |
       |  Reduced Safe    |  <-- Controller operates here
       |  Region          |
       |                  |
       +------------------+

By restricting the controller to this slightly smaller, tightened region, the system maintains a safety margin capable of absorbing sudden, worst-case adversarial shocks without breaching the true physical boundaries.

Scaling Up: Spectral Control

While disturbance-response parameterization solves the convexity problem, it introduces a computational bottleneck. To capture long-range dependencies in highly sensitive systems, the history window $H$ must be large, which causes the number of parameters in $\mathcal{M}$ to explode.

To overcome this computational hurdle, Anand Paresh Brahmbhatt et al. introduced Online Spectral Control (OSC) in A New Approach to Controlling Linear Dynamical Systems. Instead of maintaining a raw history of past disturbances, OSC applies spectral filtering techniques based on Hankel matrices to compress the history into a compact, mathematically optimal representation. This provides an exponential runtime speedup over previous methods, allowing nonstochastic control techniques to scale effectively to complex, high-dimensional, and partially observed robotic systems.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 81 papers
  2. Version 1 · 2026-10-04 · 105 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_adversarial_disturbance_modeling_6dcfe00c,
  title = {adversarial disturbance modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adversarial-disturbance-modeling/rdr_6dcfe00cbfd040239197d50b630dc003},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_adversarial_disturbance_modeling_6dcfe00c_v2,
  title = {adversarial disturbance modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adversarial-disturbance-modeling/rdr_6dcfe00cbfd040239197d50b630dc003/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake