TL;DR
This paper extends the "regret-optimal" control framework to the more realistic (and difficult) setting of measurement feedback, where the controller cannot directly see the system's state or the disturbances hitting it. The authors prove that while some performance bounds are impossible to achieve in unstable systems, they can derive new controllers that achieve optimal regret by cleverly reducing the problem to a standard $H_\infty$ control problem in a synthetic system.
What Problem Does This Paper Address?
In control theory, we usually design controllers using $H_2$ (optimal for Gaussian noise) or $H_\infty$ (optimal for worst-case noise) frameworks. However, these require making strong assumptions about how disturbances are generated. Regret-optimal control offers a "universal" alternative: it designs a causal controller (one that only knows the past) to compete against a hypothetical noncausal controller that knows the entire sequence of disturbances in advance.
Until now, most regret-optimal research assumed "full information"—meaning the controller knows the exact state of the system at every step. This paper tackles measurement-feedback control, where the controller only receives noisy, indirect observations (like a GPS signal with interference). The challenge here is a "double whammy": the controller must simultaneously estimate the hidden state and choose actions, all while competing against an omniscient benchmark that isn't fooled by measurement noise.
Methods
The authors use a combination of classical control theory and modern online learning concepts:
- Reduction to $H_\infty$: The core innovation is showing that minimizing regret in a physical system is mathematically equivalent to minimizing the $H_\infty$ norm (worst-case gain) of a specially constructed synthetic system.
- Youla Parameterization: They use this technique to search over the space of all possible stable causal controllers.
- Complexity Measures: They evaluate regret against different "complexity" measures of the disturbance:
- Energy: The total magnitude of the disturbances.
- Pathlength: How much the disturbance changes over time (its "jitteriness").
- DAREs: The final controllers are computed by solving Discrete Algebraic Riccati Equations, which are standard tools in control engineering, making the results practically implementable.
Datasets
The paper does not use real-world datasets. Instead, it evaluates the proposed algorithms using numerical simulations on a Double Integrator System. This is a classic benchmark in control theory that models the basic kinematics of a moving object (position and velocity).
Key Insights and Ideas
- The Impossibility of Perfection: The authors prove a "negative result": in unstable systems, it is impossible to have a bounded "competitive ratio" (the ratio of your cost to the benchmark's cost). This is because the benchmark knows the truth, while the online controller can be easily misled by initial measurement noise.
- Information Disparity: The main difficulty in measurement feedback is the "disparity" between the online and offline controllers. The offline benchmark sees the actual driving disturbance, whereas the online controller only sees a noisy version. This makes the filtering and control tasks "strongly coupled," unlike in simpler $H_2$ settings where they can be solved separately (the Separation Principle).
- Synthetic System Mapping: By mapping the regret problem to a synthetic $H_\infty$ problem, the authors can leverage decades of existing control theory to solve a modern machine-learning-inspired objective.
Key Results
- Energy-Optimal Controller: They derived a controller (Theorem 4) that achieves the best possible regret relative to the combined energy of the driving disturbance and measurement noise.
- Pathlength-Optimal Controller: They derived a second controller (Theorem 5) optimized for cases where the driving disturbance changes slowly (low pathlength) but measurement noise is still present.
- Performance Gains: In simulations:
- The Energy-optimal controller performed nearly as well as $H_2$ in random noise but was much more robust to adversarial spikes.
- The Pathlength-optimal controller significantly outperformed all other causal controllers (including $H_2$ and $H_\infty$) when the disturbance followed a "random walk," closely tracking the performance of the omniscient noncausal benchmark.
Limitations and Future Work
- Linear Systems Only: The current work is strictly limited to Linear Time-Invariant (LTI) systems. Extending this to non-linear systems remains a major open challenge.
- Quadratic Costs: The regret is defined based on quadratic cost functions. While standard, this might not capture the nuances of systems with hard constraints (e.g., "never hit this wall").
- Computational Complexity: While the solution involves DAREs, the synthetic systems created are larger than the original system (e.g., doubling the state dimension), which could increase the computational burden for very high-dimensional systems.
Future work likely involves applying these regret-optimal filters to "System Level Synthesis" (SLS) to handle safety constraints or exploring time-varying systems.
Summary updated