TL;DR
This paper introduces a data-driven control algorithm that combines Online Convex Optimization (OCO) with behavioral systems theory to manage unknown linear systems. By using a single prerecorded data sequence instead of a mathematical model, the controller can navigate time-varying energy costs and noisy sensor data, achieving "sublinear regret"—meaning its performance asymptotically approaches that of an ideal controller that knows the future.
What Problem Does This Paper Address?
In traditional control engineering, you usually need a mathematical model (like $x_{t+1} = Ax_t + Bu_t$) to design a controller. However, building these models is often expensive or impossible. Furthermore, real-world objectives often change: energy prices fluctuate, or production goals shift.
Existing "Online Convex Optimization" (OCO) methods can handle these shifting goals, but they typically assume you already have a perfect model of the system. This paper bridges that gap. It tackles the challenge of controlling a completely unknown linear system using only noisy output feedback while trying to minimize a time-varying "economic" cost function (like minimizing electricity bills as rates change).
Methods
The authors propose a Data-Driven Output Feedback algorithm (Algorithm 1). Its key components include:
- The Fundamental Lemma: Instead of using system matrices ($A, B, C, D$), the authors use a result from behavioral systems theory. They use a "Hankel matrix" built from a single, "persistently exciting" prerecorded trajectory of the system to represent all possible future behaviors.
- Steady-State Manifold Characterization: They derive a new way to define the system's equilibrium points (steady states) using only raw data.
- Online Gradient Descent (OGD): At each time step, the algorithm performs a gradient step to move toward the optimal steady state for the current cost function, then projects that result onto the data-driven manifold.
- Noise Estimation: A clever internal mechanism compares predicted outputs with actual noisy measurements to "learn" and cancel out measurement noise over time.
Datasets
The paper does not use a standard public dataset. Instead, it evaluates the algorithm using a detailed simulation of a 5-zone HVAC (Heating, Ventilation, and Air Conditioning) system.
- The Setup: A linear thermal dynamics model where only three of the five zones are measured.
- The Challenge: The system is subjected to unknown process noise (extra heat sources), sensor noise, and a simulated "failing sensor" where measurement error spikes for several hours.
- Economic Factors: The cost function is driven by real-world-style fluctuating energy prices over a 24-hour cycle.
Key Insights and Ideas
- Noise is a Constant Burden, Not a Dealbreaker: The most significant theoretical finding is that measurement noise only adds a constant term to the "regret" upper bound. It doesn't stop the algorithm from being efficient in the long run.
- Asymptotic Noise Recovery: The algorithm is mathematically proven to "unmask" the true measurement noise over time, allowing it to control the actual system state rather than just reacting to sensor errors.
- Economic Flexibility: Unlike many controllers that only care about "tracking" a setpoint, this approach handles "economic" costs, where the goal is to find the cheapest way to operate within a range of acceptable states.
- Shorter Horizons = Aggressive Control: The authors found that a shorter "prediction horizon" ($\mu$) makes the controller more aggressive, which actually improved performance in their thermal control example.
Key Results
- Sublinear Regret: The authors prove that the "regret" (the difference between the algorithm's performance and the theoretical best possible performance) grows slower than time ($T$). This means that on average, the error goes to zero as $T$ increases.
- Robustness to Failure: In the HVAC simulation, the controller successfully tracked the optimal temperature even when a sensor "failed" (adding a $2^\circ C$ error). The algorithm identified the error and corrected its estimates within a few hours.
- Superiority over Baselines: When compared to existing data-driven methods (like those in [28]), this OCO-based approach showed significantly less overshoot and fewer oscillations when the target temperature or energy prices changed suddenly.
Limitations and Future Work
- Linear Systems Only: The current theoretical guarantees are strictly for Linear Time-Invariant (LTI) systems. While many real systems can be approximated as linear, highly nonlinear systems (like high-performance aircraft) would require further extensions.
- Initial Data Requirements: The algorithm requires one "clean" (noise-free) prerecorded trajectory to start. In some industrial settings, getting a perfectly noise-free initial run might be difficult.
- Constraints: While the authors mention that OCO can handle state and input constraints, this specific paper focuses on the unconstrained case to simplify the regret analysis. The authors identify adding hard constraints and handling nonlinearities as the next frontiers for this research.
Summary updated