Abstract
This paper introduces DUST (Data-driven Uncertainty quantification and density STeering), a novel framework for the control of unknown stochastic systems using noisy historical data. DUST integrates covariance steering with data-driven control by leveraging the subspace geometry of raw data to derive tight uncertainty bounds. This approach allows for the simultaneous estimation of noise and the design of robust controllers, ensuring that the system reaches a target probability distribution with high probability, even in the absence of an explicit mathematical model.
Introduction
Traditional control methodologies typically rely on established mathematical models, such as state-space matrices ($A$ and $B$). However, practical applications often only provide access to "raw data"—noisy logs of past inputs and outputs. While data-driven control has gained traction, existing methods frequently assume noise-free data or focus solely on steering the mean state.
DUST addresses the more complex challenge of Stochastic Optimal Control: steering the entire probability density (the "cloud" of possible states) of an unknown system. By explicitly quantifying noise in offline data, DUST constructs safety buffers essential for reliable control in uncertain environments. It ensures that even when the underlying model is a mystery, the system can be guided to a target distribution with high confidence.
Methods
The DUST framework operates through a structured multi-step pipeline designed to move from raw data to optimal control trajectories.
- Data Collection: The process begins with a single trajectory of offline data consisting of inputs and noisy state measurements.
- Noise Estimation: Historical noise realizations are estimated using Maximum Likelihood Estimation (MLE) or Neural Networks (NN). MLE is prioritized for its ability to provide analytical error bounds and confidence intervals.
- Uncertainty Quantification: This core innovation utilizes the Robust Fundamental Lemma and subspace decomposition to create "Tight" uncertainty sets. Unlike generic uncertainty balls, these sets account for the fact that noise estimation errors are constrained by the subspace of the collected data.
- Robust Control Synthesis:
- Mean Steering (DD-MS): Implemented via System Level Synthesis (SLS) to optimize closed-loop responses, ensuring the average path remains within bounds despite model inaccuracies.
- Covariance Steering (DD-CS): Formulated as a Semi-Definite Program (SDP) that uses derived uncertainty bounds to ensure the state's "spread" (covariance) meets the target at the final time step.
Key Insights and Results
A primary finding of this work is that noise estimation error in data-driven settings is highly structured. By accounting for the reduced degrees of freedom in the data's subspace, the "Tight" uncertainty bounds proposed are significantly less conservative than previous methods.
Key Results include: * Tightness: Empirical evaluations on a 2D double-integrator model showed that these bounds almost perfectly match true error quantiles (e.g., at $\delta=0.1$, the bound was 0.326 vs. the true value of 0.326). In contrast, "Loose" bounds from prior literature were up to 14 times larger for long data horizons. * Reliability: In high-noise scenarios, the DUST robust controller maintained 100% feasibility in reaching the target distribution. In contrast, standard data-driven controllers that neglect noise estimation errors often failed completely, dropping to 0% feasibility. * Separation Principle: The framework successfully decouples the steering of the mean from the steering of the covariance, allowing for specialized robust optimization for each component.
Conclusion and Future Work
DUST provides a robust bridge between data-driven estimation and stochastic control. While currently focused on Linear Time-Invariant (LTI) systems with Gaussian noise, future research directions include:
- Nonlinearity: Extending the framework to nonlinear systems, potentially via Koopman operators.
- Non-Gaussian Noise: Utilizing Distributionally Robust Control (e.g., Wasserstein ambiguity sets) to handle unknown noise distributions.
- End-to-End Design: Moving toward a purely end-to-end data-to-control pipeline that remains robust without intermediate noise estimation steps.
Summary updated