Nonstochastic Control for Adversarial Environments

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2105 papers117 conceptsfrozen 2026-10-04

Classical control theory is built on a "polite" view of the world. In the standard Linear Quadratic Regulator (LQR) or Gaussian (LQG) frameworks, uncertainty is modeled as white noise—random, independent, and zero-mean. If you nudge a system, the environment nudges back, but only in a predictable, statistical sense.

However, as Elad Hazan, Sham M. Kakade, and Karan Singh argue in The Nonstochastic Control Problem, real-world disturbances are rarely so well-behaved. Wind gusts, sensor glitches, or fluctuating computational workloads are often correlated, non-stationary, or even adversarial. When the noise is "mean"—designed or patterned to exploit a controller's weaknesses—traditional stochastic methods can fail catastrophically.

The emerging field of nonstochastic control replaces the "average-case" Gaussian assumption with a "worst-case" regret minimization framework. It treats control not as a static optimization problem, but as an online learning game against an adversary.

From Expected Cost to Regret

In stochastic control, the goal is to minimize the expected cost over time. In nonstochastic control, we use a more ambitious metric: regret.

Regret is the difference between the cumulative cost incurred by our controller and the cost of the best fixed controller we could have chosen in hindsight, given the actual sequence of disturbances that occurred. As Max Simchowitz, Karan Singh, and Elad Hazan explain in Improper Learning for Non-Stochastic Control, achieving "sublinear regret" means that as time goes on, the average performance of our learner converges to the performance of the best possible fixed policy, even if the environment was actively trying to trip us up.

The Mechanism: Disturbance Response Policies

The breakthrough that makes nonstochastic control computationally feasible is the realization that we can reparameterize the control problem into a convex one.

In a linear system, the current state is essentially a sum of all past inputs and all past disturbances. By parameterizing the controller to choose actions based on a window of past disturbances—a "disturbance response" policy—the cost function becomes convex with respect to the controller's parameters. This allows researchers to apply the powerful machinery of Online Convex Optimization (OCO).

A primary example is the Gradient Perturbation Controller (GPC). Instead of following a fixed rule, GPC observes the disturbance at each step and performs a gradient descent update on its policy parameters. Naman Agarwal, Anirudha Majumdar, and Elad Hazan demonstrated in A Regret Minimization Approach to Iterative Learning Control that this approach allows robots to adapt to unmodeled dynamics like wind or friction during execution, rather than just between trials.

Best of Both Worlds

A common critique of robust or adversarial methods is that they are "too conservative"—they might perform poorly in normal, random environments because they are too busy preparing for a monster that never arrives.

Recent work has shown this is a false dichotomy. Max Simchowitz in Making Non-Stochastic Control (Almost) as Easy as Stochastic and Gautam Goel et al. in Best of Both Worlds in Online Control proved that modern online controllers can achieve the best of both worlds: they match the optimal performance of LQR when noise is stochastic, while maintaining sublinear regret when noise becomes adversarial.

Scaling to the Real World

The theory is rapidly expanding to handle the complexities of physical systems:

By shifting the focus from "average noise" to "any noise," nonstochastic control provides a rigorous foundation for autonomous systems that must operate in the unpredictable, often uncooperative, real world.

Papers in this version

Showing 60 of 105 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 81 papers
  2. Version 1 · 2026-10-04 · 105 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_adversarial_disturbance_modeling_6dcfe00c,
  title = {adversarial disturbance modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adversarial-disturbance-modeling/rdr_6dcfe00cbfd040239197d50b630dc003},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_nonstochastic_control_for_adversarial_en_6dcfe00c_v1,
  title = {Nonstochastic Control for Adversarial Environments},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/nonstochastic-control-for-adversarial-environments/rdr_6dcfe00cbfd040239197d50b630dc003/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}