Classical control theory is built on a "polite" view of the world. In the standard Linear Quadratic Regulator (LQR) or Gaussian (LQG) frameworks, uncertainty is modeled as white noise—random, independent, and zero-mean. If you nudge a system, the environment nudges back, but only in a predictable, statistical sense.
However, as Elad Hazan, Sham M. Kakade, and Karan Singh argue in The Nonstochastic Control Problem, real-world disturbances are rarely so well-behaved. Wind gusts, sensor glitches, or fluctuating computational workloads are often correlated, non-stationary, or even adversarial. When the noise is "mean"—designed or patterned to exploit a controller's weaknesses—traditional stochastic methods can fail catastrophically.
The emerging field of nonstochastic control replaces the "average-case" Gaussian assumption with a "worst-case" regret minimization framework. It treats control not as a static optimization problem, but as an online learning game against an adversary.
From Expected Cost to Regret
In stochastic control, the goal is to minimize the expected cost over time. In nonstochastic control, we use a more ambitious metric: regret.
Regret is the difference between the cumulative cost incurred by our controller and the cost of the best fixed controller we could have chosen in hindsight, given the actual sequence of disturbances that occurred. As Max Simchowitz, Karan Singh, and Elad Hazan explain in Improper Learning for Non-Stochastic Control, achieving "sublinear regret" means that as time goes on, the average performance of our learner converges to the performance of the best possible fixed policy, even if the environment was actively trying to trip us up.
The Mechanism: Disturbance Response Policies
The breakthrough that makes nonstochastic control computationally feasible is the realization that we can reparameterize the control problem into a convex one.
In a linear system, the current state is essentially a sum of all past inputs and all past disturbances. By parameterizing the controller to choose actions based on a window of past disturbances—a "disturbance response" policy—the cost function becomes convex with respect to the controller's parameters. This allows researchers to apply the powerful machinery of Online Convex Optimization (OCO).
A primary example is the Gradient Perturbation Controller (GPC). Instead of following a fixed rule, GPC observes the disturbance at each step and performs a gradient descent update on its policy parameters. Naman Agarwal, Anirudha Majumdar, and Elad Hazan demonstrated in A Regret Minimization Approach to Iterative Learning Control that this approach allows robots to adapt to unmodeled dynamics like wind or friction during execution, rather than just between trials.
Best of Both Worlds
A common critique of robust or adversarial methods is that they are "too conservative"—they might perform poorly in normal, random environments because they are too busy preparing for a monster that never arrives.
Recent work has shown this is a false dichotomy. Max Simchowitz in Making Non-Stochastic Control (Almost) as Easy as Stochastic and Gautam Goel et al. in Best of Both Worlds in Online Control proved that modern online controllers can achieve the best of both worlds: they match the optimal performance of LQR when noise is stochastic, while maintaining sublinear regret when noise becomes adversarial.
Scaling to the Real World
The theory is rapidly expanding to handle the complexities of physical systems:
- Continuous Time: Real robots don't move in discrete "steps." Jingwei Li and Baoxiang Wang have extended nonstochastic regret bounds to continuous-time systems in Online Control with Adversarial Disturbance for Continuous-time Linear Systems, bridging the gap between OCO and classical differential equations.
- Black-Box Systems: What if you don't know the physics of the system (the $A$ and $B$ matrices)? Elad Hazan and Xinyi Chen showed in Black-Box Control for Linear Dynamical Systems that it is possible to learn the system and control it simultaneously with sublinear regret, though it requires an initial "startup cost" to probe the system's dimensions.
- Efficiency at the Edge: For systems near instability, traditional gradient methods can be slow. Anand Brahmbhatt introduced Online Spectral Control (OSC) in A New Approach to Controlling Linear Dynamical Systems, which uses spectral filtering to achieve exponential runtime improvements, making these robust methods practical for high-frequency applications like drone flight.
By shifting the focus from "average noise" to "any noise," nonstochastic control provides a rigorous foundation for autonomous systems that must operate in the unpredictable, often uncooperative, real world.