To build an accurate mathematical model of a dynamic system—whether it is a drone, a chemical refinery, or a semiconductor manufacturing line—you typically need to observe how its outputs change in response to different inputs. In an ideal world, you would inject random, independent probing signals into the system and measure the results.
But in the real world, many systems are inherently unstable or too dangerous to run without supervision. As Boroujeni et al. point out, you cannot simply "let them run" to collect data; an active feedback controller must remain engaged at all times to keep the system from crashing.
This necessity introduces a fundamental statistical paradox. When data is collected under active feedback control, standard system identification methods break down, yielding highly biased and dangerously inaccurate models. Overcoming this "closed-loop identification bias" requires specialized techniques designed to untangle the true dynamics of a system from the influence of its controller.
The Mechanism of Bias: How Feedback Corrupts Data
To understand why standard statistical estimators fail in a closed-loop environment, we have to look at the direction of data causality.
In an open-loop setting, the input signal $u_t$ is chosen independently of the system's state. It influences the output $y_t$, which is also corrupted by some unmeasurable process or measurement noise $e_t$. Because $u_t$ and $e_t$ are statistically independent, standard regression techniques (like Ordinary Least Squares) can easily isolate the exact causal effect of $u_t$ on $y_t$.
Open-Loop: Input (u) ---------> System ---------> Output (y)
^
|
Noise (e)
In a closed-loop setting, the feedback controller creates a bidirectional highway. If a random noise spike $e_t$ alters the output $y_t$, the controller immediately detects this deviation and adjusts the next input $u_{t+1}$ to counteract it.
Closed-Loop: Input (u) --------> System ---------> Output (y)
^ |
| v
+----------- Controller <------------+
^
|
Noise (e)
Because the controller reacts to the noise, the input signal becomes inherently correlated with the system noise. As Javanmard et al. note, this feedback loop completely violates the independent and identically distributed (i.i.d.) assumptions that underpin traditional statistical estimators.
When an identification algorithm encounters this closed-loop data, it cannot distinguish whether a change in the output was caused by the input signal or by the noise. As a result, the algorithm suffers from systematic bias.
In the worst-case scenario, if the noise is highly influential, the identification algorithm will completely fail to recover the plant's dynamics. Instead, as demonstrated by González et al., the algorithm may erroneously converge to the negative inverse of the controller ($-K^{-1}$). Because the mathematical relationship enforced by the controller ($u = -Ky$) is often much cleaner than the noisy physics of the actual system, the estimator mistakenly identifies the controller's behavior rather than the system's true transfer function.
The Pitfall for Data-Driven Control
This feedback-induced correlation does not just break traditional physics-based modeling; it also cripples modern data-driven control frameworks.
Data-Driven Predictive Control (DDPC) and subspace identification methods are designed to predict future system trajectories directly from historical data columns without needing an explicit parametric model. However, Moffat et al. demonstrate that subspace-based DDPC methods are highly susceptible to a specific "Subspace Bias" when trained on closed-loop data. Subspace methods look at historical matrices to find geometric relationships; when system innovations (noise) are correlated with past inputs, the algorithm misinterprets this non-causal correlation as a true causal relationship.
The practical consequence of this misinterpretation is severe performance degradation. Wang et al. show that standard DDPC methods become entirely ineffective when applied to closed-loop operational data, leading to highly suboptimal control and poor reference tracking. Similarly, in high-precision manufacturing contexts like semiconductor lithography, Li et al. observe that ignoring these hidden correlations yields parameters that cannot accurately distinguish between external process drifting and actual control effects.
Mathematical Keys to Breaking the Loop
To safely identify a system while it remains under control, researchers have developed several distinct mathematical strategies to decouple the feedback loop.
1. Instrumental Variables (IV)
The intuition behind the Instrumental Variable approach is to find a "proxy" signal—an instrument—that is highly correlated with the actual input $u_t$, but completely uncorrelated with the system noise $e_t$.
As Schoukens et al. explain, the ideal instrument in a closed-loop system is the original external reference signal (the setpoint). Because the reference signal is injected from outside the system, it is mathematically impossible for the internal system noise to influence it. By filtering the input data through the lens of this external reference, the feedback-induced correlation is stripped away, leaving an unbiased estimate of the system's true dynamics. Wang et al. successfully leveraged this instrumental variable approach to redesign DDPC, allowing predictive controllers to learn safely from active operational data.
2. High-Order Modeling and Null-Space Fitting
Another way to bypass the bias is to absorb the feedback dynamics into a larger, temporary model. Traditional Prediction Error Methods (PEM) try to estimate the system parameters and a simple noise model simultaneously, which often fails if the noise dynamics are complex and non-convex (Galrinho et al.).
To fix this, Galrinho et al. introduced Weighted Null-Space Fitting (WNSF). This framework first fits a high-order linear model to the closed-loop data. Because this initial step has a high degree of freedom, it effectively captures the joint behavior of the system and the controller without getting trapped in local minima. The algorithm then uses a weighted least-squares step to strip away the controller's influence and extract the underlying low-order plant parameters cleanly.
3. Predictor Forms and Output Linearization
Instead of correcting the data after the fact, some methods mathematically transform the closed-loop problem so that it mirrors an open-loop structure.
In partially observable settings where states are hidden, Lale et al. utilize the Kalman filter's "predictor form." By reformulating the system equations to predict future observations based on past actions and past prediction errors (rather than raw states), they isolate the feedback dependencies and achieve optimal, logarithmic regret bounds during adaptive control.
For highly complex, nonlinear systems, Boroujeni et al. propose a method that linearizes the system's output around a known external excitation signal. This mathematical transformation converts a biased closed-loop identification problem into a statistically consistent open-loop problem, ensuring that the trainable components remain entirely independent of measurement noise.
Real-World Impact and Theoretical Limits
When these de-biasing mechanisms are properly implemented, they allow engineers to achieve remarkable modeling accuracy in high-stakes industrial environments. For example, Ping Wu applied a closed-loop subspace identification method (PBSID_OPT) to a Blast Furnace gas top pressure recovery turbine. Even though the safety controllers could not be deactivated, the specialized closed-loop identification decoupled the noise correlations, achieving over 95% modeling accuracy and allowing a predictive controller to stabilize furnace pressure within a razor-thin $\pm$1 kPa margin.
Despite these practical successes, the theoretical boundaries of closed-loop identification remain an active area of research. González et al. recently provided a rigorous mathematical reality check, proving that widely used refined instrumental variable methods can still exhibit subtle biases when operating under continuous-time feedback loops, though they regain full statistical consistency if the controller is implemented in discrete-time. Furthermore, extending these statistical guarantees to multi-input systems using coordinate descent algorithms remains a significant open question (González et al.), proving that as long as controllers react to noise, the mathematics of untangling the loop will remain central to data-driven engineering.
Key Foundational Work
An overview of subspace identification (2006) — contributed 3 concept elements to this direction.
Identification and Control of Blast Furnace Gas Top Pressure Recovery Turbine Unit (2012) — contributed 1 concept elements to this direction.
Efficient Reinforcement Learning for High Dimensional Linear Quadratic Systems (2012) — contributed 1 concept elements to this direction.