Learning Plants the Controller Already Stabilizes: Projected Closed-Loop Identification in Dual-Youla Coordinates

When a machine can only be run under a feedback controller (an unstable drone, a chemical reactor, a precision servo), the data used to learn its model is shaped by that controller. The controller reacts to noise, so its commands become correlated with the noise. A naive fit then mixes up "the plant responds to the input" with "the controller responds to the output," and can converge to the negative inverse of the controller instead of the plant. Separately, online learners in adaptive control keep their parameter estimates inside a safe region by projection: after each update, the estimate is snapped back to the nearest point of an allowed set. This keeps estimates bounded. It does nothing about the bias, and it is cheap only when the set is a box or a ball. A stability-certified set would normally require solving a matrix inequality at every step.

This note proposes a small change of coordinates that brings the two ideas together. Every plant that a given controller stabilizes can be written as a fixed nominal plant plus a stable correction $R$. This is the dual Youla–Kučera parameterization, standard control theory that we reuse rather than claim. In these coordinates three things become true at once. Identification of $R$ is an open-loop problem driven only by the external reference, so the feedback-induced bias disappears when the controller is known. "The current controller stabilizes this model, with margin" becomes a convex norm bound on $R$ that is cheap to project onto. And the negative-inverse-of-controller trap lies outside that set by construction. We propose an online, recursive estimator of $R$ with a norm-ball projection whose radius is read off the closed-loop data. We will test whether it gives consistent, always-stabilizable estimates, including on open-loop unstable plants and when the controller is only estimated.

Core Claim

Let the running controller be $K = U V^{-1}$ and a nominal plant it stabilizes be $G_0 = N D^{-1}$, both written as ratios of stable transfer functions (coprime factorizations). Every plant stabilized by $K$ can then be written as

$$G(R) = (N + V R)(D - U R)^{-1}, \qquad R \text{ stable},$$

up to sign convention. The signals $\alpha_t$ and $\beta_t$ are obtained by filtering the reference $r$, input $u$ and output $y$ through these known factors. They satisfy $\beta_t = R\,\alpha_t + v_t$, where $\alpha_t$ depends only on the reference and $v_t$ is filtered noise uncorrelated with it. Fitting $R$ is therefore an ordinary open-loop regression.

We parameterize $R$ as a finite impulse response with coefficients $\theta \in \mathbb{R}^d$. The update is a recursive instrumental-variable or least-squares step followed by

$$\theta_{t+1} = \Pi_{\Theta_\gamma}(\theta_{t+1/2}), \qquad \Theta_\gamma = \{\theta : \|\theta\|_1 \le \gamma\}.$$

The $\ell_1$ norm of the impulse response upper-bounds the peak frequency gain $\|R\|_\infty$, and projection onto an $\ell_1$ ball costs $O(d\log d)$. By a small-gain argument, every iterate is then a plant the current controller stabilizes, with a sensitivity margin set by $\gamma$. The plant $-K^{-1}$ makes $1 + GK = 0$, so it cannot be reached by any finite $R$ and lies strictly outside $\Theta_\gamma$.

The claim has three empirically separable parts. First, with the controller known, the projected estimator is consistent at the usual $1/\sqrt{T}$ rate, since this is a linear regression on exogenous regressors. Second, 100% of intermediate estimates remain stabilizable, including for open-loop unstable plants. Third, when the controller is only estimated, the residual bias stays bounded by a function of $\gamma$ and the controller error, instead of drifting toward $-K^{-1}$.

Relation to Prior Work

Three papers supply the debiasing methods we compete with. All three are batch methods and assume the controller is known exactly. Consistency analysis of refined instrumental variable methods for continuous-time system identification in closed-loop proves that closed-loop refined instrumental variables are consistent under sample-and-hold digital feedback. It also gives the negative-inverse-of-controller result for direct estimators that motivates our feasible set. Data-Driven Predictive Control Using Closed-Loop Data: An Instrumental Variable Approach builds instruments from the reference and from coprime factors of the controller, inside a predictive controller. We use the same factor algebra, but as coordinates for a constraint set rather than as an instrument. Parametric Identification Using Weighted Null-Space Fitting reaches prediction-error efficiency from closed-loop data without local minima. Its high-order intermediate fit assumes stable plant and noise models, whereas our $R$ is stable even when $G$ is not.

The closest work in spirit is Neural Identification of Feedback-Stabilized Nonlinear Systems. It also rewrites a feedback-stabilized unstable plant as an interconnection of the known controller with a stable learned operator, built from recurrent equilibrium networks that are contractive by construction. It is batch, nonlinear, needs the exact controller, and degrades at low signal-to-noise ratio. We trade nonlinearity for an online linear estimator with a convex, data-sized constraint and a test of controller mismatch.

Two papers motivate the gap from the projection side. In Adaptive Economic Model Predictive Control for linear systems with performance guarantees, a projected least-mean-squares update keeps estimates in a compact admissible set. It assumes an open-loop stable plant and does not address feedback-correlated noise. Composite Adaptive Control for Time-varying Systems with Dual Adaptation pairs projection with leakage and proves bounded errors. It is explicit that a residual error proportional to the disturbance remains. Both show that projection in raw parameter coordinates bounds the estimate without removing the bias. That is the specific failure the coordinate change is meant to fix.

Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems constrains and partly competes with our approach. Its predictor-form regression whitens the noise and is consistent under feedback without instruments. It is our strongest unconstrained baseline, discussed below. Iterative Youla-Kucera Loop Shaping For Precision Motion Control shows that Youla-type parameters stay well behaved where direct controller tuning destabilizes. That is evidence the coordinates are numerically benign, though it works on the controller side, not on identification.

Research Program

The first component is the estimator. We will build recursive least squares and recursive instrumental variables in dual-Youla coordinates, with the reference and its delays as instruments, followed by $\ell_1$-ball projection. We will also build a frequency-gridded variant that projects onto a peak-gain bound at a few hundred frequencies, so we can measure how conservative the $\ell_1$ surrogate is.

The second component is a data-driven radius. Because the reference is external, the transfer from $r$ to $u$ is an open-loop map and can be estimated consistently. The deviation of this measured closed loop from the nominal one bounds how large $R$ can be, which gives $\gamma$ from data plus a safety factor. We will report results with an oracle $\gamma$ as well, to separate the radius question from the coordinate question.

The third component is an unknown controller. When the controller is deterministic, $r - u = K y$ holds without noise, so $K$ can be fit from logged signals. Saturation, anti-windup and unmodeled delay break this exactly, and that is where the risk lies. We will analyze how controller error $\Delta K$ leaks noise into $\alpha_t$. We aim to prove that the projected estimator's asymptotic bias is bounded by a quantity that grows with $\|\Delta K\|$ and is capped by $\gamma$, whereas the unprojected estimator's bias has no such cap.

The fourth component is use inside an adaptive loop. We feed each projected estimate into a certainty-equivalent predictive controller of the kind used by Degner et al. When the controller is redesigned, the coordinates are re-centered, mapping the current $G(R)$ to new factors. The open question is whether this re-centering preserves convergence when the controller changes slowly.

Baselines

The do-nothing reference point is direct recursive least squares on the plant's input–output regressor, with no instruments and no projection. It is what a practitioner gets from a standard toolbox, and it should show the negative-inverse drift. Next is projected least-mean-squares onto a physically plausible box in raw plant coordinates, following Adaptive Economic Model Predictive Control for linear systems with performance guarantees. It is the existing method closest to ours on the projection side. Beating it means lower asymptotic parameter error at equal excitation power, not just boundedness.

The strongest debiasing competitors are batch. Closed-loop refined instrumental variables from González et al. and weighted null-space fitting from Galrinho et al. will be rerun on sliding windows at the same update interval as our estimator. Any window estimate the controller does not stabilize will be rejected afterwards. On predictive control, the transient predictor of The Bias of Subspace-based Data-Driven Predictive Control and instrumental-variable predictive control from Wang et al. are the reference controllers. Predictor-form regularized least squares from Lale et al., run recursively, is the strongest unconstrained online competitor. On nonlinear benchmarks we compare to internal-controller identification from Boroujeni et al.. Its reported scalar-benchmark mean squared error of 0.0034 and point-mass-robot $R^2$ of 0.9378 (versus 0.8368 for direct identification at low excitation $\sigma = 10$) are the only numerical figures in our file. For the double integrator, furnace, building and continuous-time benchmarks, no reported result found on Lacuna for numerical baseline values, so we will regenerate all baselines ourselves.

Fairness rules are as follows. All methods see identical simulated trajectories, reference signals and noise seeds. Model orders are chosen by the same validation split. Each method gets the same hyperparameter search budget, about 50 configurations. Windowed batch methods get compute comparable to the full recursive run.

Three ablations isolate the key ingredient. The first removes projection but keeps dual-Youla coordinates. The second applies the same projection in raw plant coordinates instead of $R$. The third replaces the data-derived $\gamma$ with an oracle. Our method counts as winning on one condition: it matches the batch instrumental-variable error asymptotically and also keeps every intermediate estimate stabilizable while those baselines do not, or it degrades less under controller mismatch.

Why not just…?

Why not just run projected least-mean-squares with a tighter box, as in adaptive MPC? A box in raw plant coordinates is not aligned with stability or with the bias direction. Degner et al. and the composite adaptive control paper both obtain bounded but biased estimates. If the true plant is near the box face nearest to $-K^{-1}$, a tighter box only converts drift into boundary trapping. The ablation that projects in raw coordinates tests exactly this. If it matches our method, the coordinate change is not doing the work and we will say so.

Why not just rerun batch instrumental variables on a sliding window and discard unstable results? This is the most serious threat. With a known controller it may well match our asymptotic accuracy. Its weaknesses are transient. Early or low-excitation windows can produce models the controller does not stabilize, and discarding them leaves the adaptive controller with stale models. Each refit also costs a full batch solve. If windowed refits reach equal accuracy, an equal rejection-free rate, and comparable compute, the contribution shrinks to a convenient parameterization. The headline comparison is designed to show whether that happens.

Why not just use predictor-form least squares, which is already consistent under feedback? Lale et al. show that this whitening removes the feedback correlation for linear systems. However, it needs a long, truncated history, and it gives no guarantee that each intermediate model is stabilized by the current controller. Its high-order fit also becomes poorly conditioned for open-loop unstable plants. We will run it with and without our projection appended. If predictor-form regression plus a cheap post-hoc stability check equals our method on unstable plants, that combination is the better recommendation.

Why not just inject more dither? Larger probing signals do reduce the relative bias. They also cost tracking accuracy and wear. The low-excitation gap reported by Boroujeni et al. (0.8368 versus 0.9378 in $R^2$) shows that the regime that matters is weak excitation. We will sweep the dither amplitude and report the excitation each method needs to reach a fixed parameter error.

Proposed Experiments

The primary testbed is the double integrator under proportional–derivative feedback from The Bias of Subspace-based Data-Driven Predictive Control. It is open-loop unstable, the bias of naive predictors is documented, and the transient predictor gives a strong control baseline. We will measure relative parameter error against sample count from $10^2$ to $10^5$, the fraction of intermediate estimates the controller stabilizes, closed-loop tracking cost when estimates drive a certainty-equivalent predictive controller, and per-step compute.

The continuous-time second-order system of González et al., under sample-and-hold feedback, checks consistency against an estimator with proven guarantees. The two-input, two-output tubular furnace of Wang et al. tests the multivariable factorization. The 2R2C building model of Degner et al. tests the adaptive loop with controller redesign over simulated weeks.

Controller mismatch is the decisive stress test. On each benchmark we add input saturation, an unmodeled one-to-three-sample delay, and a 10–30% gain error to the true controller, while the estimators use either the nominal or a regression-fit controller. The prediction is specific. Unprojected recursive instrumental variables and internal-controller identification should show bias growing with mismatch. Our projected estimator should show bias that plateaus at a level set by $\gamma$, with zero destabilizing estimates. If the plateau does not appear, the mismatch-robustness claim fails.

Finally, we run the linear method on the scalar unstable and point-mass-robot benchmarks against the reported internal-controller-identification figures. We expect to lose on nonlinear accuracy. This experiment marks the boundary of the linear method and does not count toward the win condition. Whether a projected $R$ fit stays competitive at low excitation will tell us whether a nonlinear extension of the constraint is worth building.