Online Stochastic Optimization for Unknown Linear Systems: Data-Driven Synthesis and Controller Analysis

Gianluca Bianchin, Miguel Vaquero, Jorge Cortés, Emiliano Dall'AnesearXiv 2021 · arXiv

Abstract

This paper presents a novel model-free framework for regulating unknown, noisy linear systems toward optimal operating points defined by stochastic optimization problems. By utilizing Behavioral System Theory, the authors derive a data-driven estimate of the system's steady-state transfer function from a single trajectory of historical data. This estimate informs a stochastic gradient-descent controller that achieves convergence to a "stable optimizer" despite non-negligible system dynamics and stochastic disturbances. The approach is validated through numerical simulations and a real-world ride-sharing case study in Manhattan.

Introduction

In complex infrastructures such as power grids and transportation networks, the goal is often to steer a system toward an optimal state—for instance, minimizing energy loss or maximizing throughput. Traditional control strategies typically rely on a two-stage process: system identification followed by controller design. However, obtaining precise mathematical models is often difficult, and systems are frequently subject to unknown stochastic noise.

Existing online optimization techniques often assume known dynamics or neglect them entirely by assuming they are sufficiently fast. This research addresses the gap where system dynamics are unknown, significant, and stochastic. The proposed framework enables system regulation using only historical input-output data and real-time feedback, bypassing the need for explicit state-space modeling.

Methodology

The proposed framework consists of two primary components: data-driven transfer function computation and a stochastic gradient-descent controller.

Data-Driven Transfer Function Computation

Drawing on the Fundamental Lemma of Behavioral System Theory, the authors demonstrate that the steady-state transfer function $G$ can be reconstructed directly from a "persistently exciting" trajectory of historical data. This approach avoids the estimation of internal state matrices ($A, B, C, D$), focusing instead on the input-output mapping at equilibrium. A key insight is that transient data contains sufficient information to characterize steady-state behavior, eliminating the need for the system to reach equilibrium during the training phase.

Stochastic Gradient-Descent Controller

The controller is designed based on Online Stochastic Gradient Descent (SGD). It utilizes the data-driven estimate of $G$ alongside real-time noisy measurements to update control inputs. A critical challenge in this setting is the "decision-dependent distribution" effect: control inputs shift the system's state, which in turn alters the distribution of observed costs. The authors analyze the "Stable Optimizer"—the point where the input is optimal for the distribution it induces—and establish explicit convergence bounds.

Results and Analysis

The framework was evaluated through Monte Carlo simulations and a practical application in urban mobility.

Transfer Function Accuracy

Numerical experiments confirmed that the data-driven characterization of the transfer function remains accurate even as the system's state size increases. As shown in the figure below, the error between the estimated and true transfer functions, as well as the error in equilibrium output predictions, remains controlled across a wide range of state dimensions.

Numerical validation of the data-driven transfer function characterization.

Manhattan Ride-Sharing Case Study

The authors applied the controller to a ride-service scheduling problem using New York City TLC data from March 2019. The objective was to dynamically adjust pricing across 18 Manhattan regions to maximize profit and fleet utilization. The online optimization controller significantly outperformed fixed-pricing strategies, demonstrating the ability to adapt to time-varying demand and vehicle rebalancing needs in real-time.

Conclusion

This work provides a robust, data-driven path for optimizing unknown linear systems under stochastic noise. By focusing on the transfer function rather than full system identification, the method reduces complexity and enhances robustness. Future research will explore extensions to nonlinear dynamics, multi-agent distributed settings, and the "dual control" problem of maintaining exploration (persistence of excitation) while simultaneously optimizing performance.

Summary updated

Abstract

This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this paper we first show that the steady-state transfer function of a linear system can be computed directly from control experiments, bypassing explicit model identification. Then, we leverage the estimated transfer function to design a controller -- which is inspired by stochastic gradient descent methods -- that regulates the system to the solution of the prescribed optimization problem. A distinguishing feature of our methods is that they do not require any knowledge of the system dynamics, disturbance terms, or their distributions. Our technical analysis combines concepts and tools from behavioral system theory, stochastic optimization with decision-dependent distributions, and stability analysis. We illustrate the applicability of the framework on a case study for mobility-on-demand ride service scheduling in Manhattan, NY.
BibTeX
@misc{bianchin2021online,
  author = {Gianluca Bianchin and Miguel Vaquero and Jorge Cortés and Emiliano Dall'Anese},
  title = {{Online Stochastic Optimization for Unknown Linear Systems: Data-Driven Synthesis and Controller Analysis}},
  year = {2021},
  eprint = {2108.13040},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2108.13040},
}
Figure