In classical control theory, the path to managing a system follows a rigid two-step pipeline: first, perform System Identification to derive a mathematical model (like the $A$ and $B$ matrices in a state-space equation), and second, design a controller based on that model.
However, for complex systems like power grids, chaotic networks, or biological processes, building an accurate model is often the hardest part of the problem. A growing body of research is bypassing this "model-first" bottleneck, developing methods that map raw experimental data directly to optimal control actions while maintaining the rigorous stability guarantees of classical theory.
The Model Bottleneck
To understand why this shift is happening, consider the Linear Quadratic Regulator (LQR), a cornerstone of control. As Steven J. Bradtke noted in Reinforcement Learning Applied to Linear Quadratic Regulation, finding the optimal LQR controller typically requires precise knowledge of the system's physics. If your model is even slightly off, the resulting controller might not just be sub-optimal—it might be unstable.
In many modern applications, such as the "linear-threshold networks" studied by Xuan Wang and Jorge Cortés in Data-Driven Stabilization of Unknown Linear-Threshold Network Dynamics, the connectivity matrices are unknown and practically impossible to estimate. This creates a "system identification bottleneck" where the effort required to model the system exceeds the effort required to control it.
From Matrices to Trajectories
The core intuition behind the "direct" data-driven approach is that a sufficiently rich history of a system's behavior contains all the information necessary to predict its future.
Instead of using data to estimate the parameters of a differential equation, researchers use data to form a "data-driven representation" of the system. This is often rooted in Willems’ Fundamental Lemma, which states that for a linear system, any trajectory can be expressed as a linear combination of previously collected trajectories, provided the initial data was "persistently exciting."
This insight led to the development of Data-Enabled Predictive Control (DeePC). As explained by Jeremy Coulson, John Lygeros, and Florian Dorfler in Regularized and Distributionally Robust Data-Enabled Predictive Control, DeePC replaces the traditional model in Model Predictive Control (MPC) with a Hankel matrix of raw input-output data. By doing so, the controller "predicts" the future by looking at how the system reacted to similar inputs in the past, entirely bypassing the need for an explicit $A$ or $B$ matrix.
Bridging Reinforcement Learning and Control
While Reinforcement Learning (RL) is inherently model-free, it has historically lacked the safety and stability guarantees required for physical engineering. Recent work has focused on merging the adaptive power of RL with the formal rigor of control theory.
For example, Hesameddin Mohammadi showed in Learning the model-free linear quadratic regulator via random search that model-free random search can solve the LQR problem with high efficiency, providing the first rigorous sample complexity bounds for this approach. Similarly, Minghao Han and colleagues developed H∞ Model-free Reinforcement Learning with Robust Stability Guarantee, which integrates Lyapunov stability analysis into a deep RL pipeline. This ensures that even as the agent learns through trial and error, it stays within "safe" operational bounds.
Handling Noise and Uncertainty
A major challenge for data-driven methods is noise. If your "model" is just a collection of noisy data points, how do you prevent the controller from over-fitting to that noise?
The solution often involves Distributionally Robust Optimization (DRO). Jeremy Coulson and Henk van Waarde have demonstrated that adding specific regularization terms to data-driven controllers is mathematically equivalent to optimizing against the worst-case noise distribution. This provides a bridge between the heuristic "tuning" of machine learning and the formal "robustness" of control theory, as seen in their work on the Robust Fundamental Lemma for Data-driven Control.
Scaling to Large-Scale Infrastructure
The ultimate goal of this research is to manage systems that are too large or too dynamic for any single model to capture. In Dissipativity-Based Data-Driven Decentralized Control of Interconnected Systems, Taiki Nakano and John Lygeros propose a decentralized framework. Instead of modeling a whole power grid, they use data to learn the "dissipative properties" (how energy flows and dissipates) of local components.
By ensuring each local controller respects these dissipative bounds, they can guarantee the stability of the entire global network without ever needing a global map. This "property-based" control, rather than "model-based" control, represents a significant shift toward more resilient and scalable autonomous systems.