Certified Stability Margins as a Convex Replacement for Noise Injection in Learning Controllers for Unstable Plants

When a controller for an unstable system is tuned only to make the output follow a reference, the optimizer often finds a trick. It places a zero of the controller exactly on the plant's unstable pole, so the two cancel. From the outside, the loop looks perfect. Inside, a hidden signal grows exponentially and is invisible until something nudges it. Challenges in Model Agnostic Controller Learning for Unstable Systems documents this failure. It also shows that the standard fix, injecting random noise into the control signal during training, avoids the cancellation but costs about 25% in tracking error on a first-order unstable plant. Injecting the noise into the reference instead does not prevent the cancellation at all.

We propose to replace noise injection with a convex constraint that has a direct meaning: a lower bound on the controller's stability margin. The margin is the largest mismatch between the model and the true plant, measured in the gap metric, that the loop can tolerate and stay stable. The gap metric is a distance between plants that remains meaningful when the plants are unstable. A cancelling controller has zero margin, so any positive bound excludes it. The margin is the inverse of the worst-case gain of four closed-loop maps together. Those maps are affine in the variables of the input-output parameterization (IOP), a convex way to write every internally stabilizing controller as a set of closed-loop maps. The resulting design problem is therefore a semidefinite program. If the bound exceeds a data-driven estimate of how far the identified model sits from the true plant, the learned controller provably stabilizes the true plant. The trade-off between performance and stability then becomes one explicit, certified knob, where noise injection offers only a heuristic whose location and strength must be guessed.

Core Claim

Let $P$ be the plant and $K$ the controller, in a positive-feedback convention. Write $\|\cdot\|_\infty$ for the worst-case gain over frequency. The margin is

$$ b_{P,K} = \left\| \begin{bmatrix} I \\ K \end{bmatrix} (I - PK)^{-1} \begin{bmatrix} I & P \end{bmatrix} \right\|_\infty^{-1}, $$

and it is zero exactly when the loop is internally unstable. The classical robust-stability result states that $K$ stabilizes every plant $P'$ whose gap distance from $P$ is less than $b_{P,K}$. That result is exact rather than an outer approximation.

In the IOP of On the Parameterization of Stabilizing Controllers using Closed-loop Responses, the four blocks inside the norm are, up to a fixed affine rearrangement, the decision variables $(Y, U, W, Z)$. Write $\hat P$ for the model identified from data, $J$ for a convex tracking cost (the $H_2$ norm of the reference-to-error map), and $\delta$ for the required margin. We solve

$$ \min_{Y,U,W,Z} \; J(Y,U,W) \quad \text{s.t.} \quad \text{IOP affine constraints for } \hat P, \quad \left\| M(Y,U,W,Z) \right\|_\infty \le 1/\delta, $$

where $M$ is the four-block map. With finite-impulse-response variables, the norm bound becomes a linear matrix inequality through the Kalman–Yakubovich–Popov lemma.

The claim has three parts. First, any $\delta > 0$ rules out the cancellation mirage. Second, as $\delta \to 0$, the optimal cost approaches the best cost achievable by internally stabilizing controllers. This is the Youla-optimal value that Challenges in Model Agnostic Controller Learning for Unstable Systems identifies as the ideal target, not the "super-optimal" cancelling value. Third, choosing $\delta$ above a high-probability estimate of the gap between $\hat P$ and the true plant gives a stability certificate for the true plant. The certificate needs no prestabilizing controller and does not require the plant to be minimum phase.

What is new is not the margin, which is classical, but where it is used. We place it inside a data-driven, factorization-free convex parameterization for open-loop unstable plants. It serves as the bridge between how accurately the model was estimated and whether the learned controller is internally stable.

Relation to Prior Work

Two papers motivate the gap.

Challenges in Model Agnostic Controller Learning for Unstable Systems establishes the cancellation failure and the costs of noise injection and prestabilization. It explicitly names learning stabilizing controllers from finite time-domain data via the gap metric as open. We take that suggestion literally, but in a convex parameterization rather than a model-agnostic learner.

On the Parameterization of Stabilizing Controllers using Closed-loop Responses supplies the method (IOP) and also constrains it. It proves IOP is robust to constraint errors only for open-loop stable plants, via a small-gain argument, and warns that unstable plants must first be prestabilized. Our margin constraint is designed to replace that small-gain argument in the unstable case. One analytic target is to show that a residual in the IOP equality constraints acts as a gap-bounded plant perturbation, so the same margin absorbs both identification error and numerical error.

Three data-driven convex methods address the neighbouring problem but not this one.

Data-Driven System Level Synthesis replaces the model with Hankel data matrices and robustifies against norm-bounded noise. Data-Driven Distributionally Robust System Level Synthesis adds Wasserstein disturbance ambiguity and reduces the design to a linear program. Learning Stabilizing Controllers for Unstable Linear Quadratic Regulators from a Single Trajectory stabilizes all plants in an ellipsoidal credible set.

All three assume full-state feedback, and the first two work over finite horizons. Pole-zero cancellations between controller and plant are therefore outside their scope. Our method works in output feedback over an infinite horizon. It measures model uncertainty in the gap metric, which does not need the true plant to be stable.

Regularization for Covariance Parameterization of Direct Data-Driven LQR Control makes the performance-stability trade-off explicit through a regularizer weight $\lambda$, but it notes that no performance bounds for $\lambda$ are known. Our $\delta$ plays the same role with a stated meaning: it is the tolerated plant mismatch.

Input-Output Data-Driven Representation: Non-Minimality and Stability shows that the unconstrained pseudoinverse keeps the hidden ("latent") poles of non-minimal data models stable, but that this guarantee breaks once constraints are added. This is a warning for us. We therefore identify an explicit model $\hat P$ and certify the loop in the gap metric, rather than relying on the pseudoinverse structure.

The distributionally robust control-design papers supply a template: turn a min-max over an uncertainty set into a single convex program. Data-driven distributionally robust LQR with multiplicative noise and Distributionally Robust Density Control with Wasserstein Ambiguity Sets are examples. Their known weakness is conservatism from outer approximations. Our inner maximization is over a gap ball, and its robust-stability condition is necessary and sufficient, so the synthesis step adds no approximation. The remaining conservatism is concentrated in the estimate of $\delta$.

Research Program

The work has three components.

The first is estimating the gap radius from data. From input-output data of the unstable plant, collected over short excitation windows, we fit a finite-order model $\hat P$ by least squares. We then compute a high-probability upper bound $\hat\delta$ on the gap between $\hat P$ and the true plant in two ways. The empirical route bootstraps over data resamples and computes the gap between $\hat P$ and each refit model. The analytic route, available for finite-order models, converts least-squares confidence ellipsoids on the coefficients into a bound on the normalized coprime-factor error, which bounds the gap. This step is the main risk. If $\hat\delta$ is loose, the margin constraint becomes infeasible or very conservative.

The second is synthesis. We solve the semidefinite program above with finite-impulse-response IOP variables of length $N$ and sweep $\delta$ to trace the Pareto curve of tracking cost against certified margin. Controllers are realized from the IOP variables following the stable-realization recipe of On the Parameterization of Stabilizing Controllers using Closed-loop Responses, and their true-plant loops are checked for internal stability by computing closed-loop poles.

The third is analysis, with three target statements. The first is continuity: the optimal cost converges to the internally stabilizing optimum as $\delta \to 0$ and $N \to \infty$. The second is residual absorption: equality-constraint errors of size $\epsilon$ reduce the certified margin by at most a quantity proportional to $\epsilon$. The third is a finite-sample stability guarantee: the true loop is stable with probability at least $1-\beta$ whenever $\delta \ge \hat\delta_\beta$, where $\hat\delta_\beta$ is the gap bound computed at confidence level $1-\beta$.

Baselines

The naive reference point is direct minimization of the $l_2$ tracking error over controllers of the same order, with no stability constraint. Challenges in Model Agnostic Controller Learning for Unstable Systems reports that this produces an unstable cancellation on the first-order benchmark. We expect to reproduce that result. It calibrates the "mirage" cost that no stable controller can reach.

The most directly competing heuristic is noise injection into the control channel. It is the method our constraint is meant to replace. The reported reference is an approximately 25% increase in tracking error when noise is injected into the control action (Challenges in Model Agnostic Controller Learning for Unstable Systems). Because that number corresponds to one noise level, we will sweep noise variance across the same grid size as our $\delta$ sweep. Both methods get the same data and the same controller order, and the comparison is between Pareto curves. We beat it if, at every tracking-cost level, our method keeps the true loop internally stable across trials and noise injection does not, or if we reach the same stability rate at clearly lower tracking cost.

Two-step prestabilization from the same paper is run whenever a stabilizing baseline is available. The paper reports "near-optimal performance" for minimum-phase plants but no numeric gap, so our comparison there is qualitative. The decisive setting is non-minimum-phase unstable plants, where a baseline is hard to find.

The strongest model-based competitor is certainty-equivalence IOP. It uses the same identified $\hat P$ and the same program with the margin constraint removed ($\delta = 0$). This baseline is also the ablation that isolates our key ingredient, because it enforces internal stability on the model but not on the true plant.

A second ablation replaces the $H_\infty$ bound on the four-block map with an $H_2$ penalty on the same map. This is the convex analogue of noise injection: a soft average-case penalty rather than a worst-case certificate. It tests whether the certificate, and not merely the extra regularization, is what matters.

For a state-feedback sanity check on the marginally unstable Laplacian system, the strongest reported numbers come from Regularization for Covariance Parameterization of Direct Data-Driven LQR Control. At noise $\sigma = 0.7$, it reports a 99% stability success rate with median optimality gap 0.189, against 88% and 0.2697 for certainty equivalence. At $\sigma = 1.0$, it reports 97% against 78%. We run our method with outputs equal to states under the same 1,000-trial protocol. Matching 99% success with no worse median gap would show that the method loses nothing in the easy setting.

Classical Glover–McFarlane loop shaping, which maximizes the same margin without a performance objective, is included on the identified model. No reported result was found on Lacuna for it on these benchmarks.

Why not just…?

Why not just tune the noise injection level more carefully? Smaller noise might shrink the 25% penalty. The result in Challenges in Model Agnostic Controller Learning for Unstable Systems that reference-channel noise fails outright shows the method depends on where noise enters, and nothing tells the user when the noise is too small to block the cancellation. The variance sweep settles the question empirically. If some noise level matches our Pareto curve and stability rate, the honest conclusion is that our contribution reduces to the certificate and the choice of $\delta$, not better performance.

Why not just identify a model and design the internally stabilizing optimal controller for it? This is certainty-equivalence IOP, and it is the most serious threat, especially with plentiful data. Prior evidence shows certainty equivalence failing at moderate sample sizes: a nearly 17% instability rate with 500 samples in Data-driven distributionally robust LQR with multiplicative noise, and consistent failures of naive data-driven synthesis in Data-Driven System Level Synthesis. The mechanism is that the nominal optimum tends to sit near cancellation, so its margin is small. We sweep data length. We expect the two methods to converge at large data and to separate sharply at small data. If they never separate, the margin constraint is unnecessary in practice.

Why not just use distributionally robust System Level Synthesis? Data-Driven Distributionally Robust System Level Synthesis handles disturbance ambiguity and model mismatch well, but it needs the full state and a finite horizon. It therefore cannot express the output-feedback cancellation that is the failure mode here. On state-feedback instances we run it as an additional baseline. On output-feedback instances it does not apply.

Why not just use classical loop shaping, which already maximizes this margin? Loop shaping maximizes margin alone. It does not trade the margin against a tracking objective, and it gives no data-driven rule for how much margin is needed. Our method is the constrained version with $\delta$ set from data. If loop shaping lands on our Pareto curve, the novelty narrows to the estimation of $\delta$.

Proposed Experiments

The first testbed is the scalar unstable plant of Challenges in Model Agnostic Controller Learning for Unstable Systems, under the same tracking setup. We report the tracking $l_2$ error and the internal stability of the true loop under random initial states. We extend this plant with a right-half-plane zero to create a non-minimum-phase unstable case where prestabilization is not readily available.

The second testbed is the 3-state Laplacian system of Regularization for Covariance Parameterization of Direct Data-Driven LQR Control. It is run twice: once with full state for comparability, and once observed through one or two outputs to create the output-feedback case. The protocol uses 1,000 trials at $\sigma \in \{0.7, 1.0\}$ and reports the success rate and median optimality gap. The submarine longitudinal model from Input-Output Data-Driven Representation: Non-Minimality and Stability serves as a higher-order output-feedback case.

The decisive checks are three.

The first check is certificate validity. The fraction of trials in which the true loop is unstable while $\delta \ge \hat\delta_\beta$ should not exceed $\beta$. A violation would falsify the estimation step.

The second check is Pareto dominance. On the output-feedback benchmarks, our tracking-versus-stability-rate curve should dominate both the noise-injection sweep and the $H_2$-penalty ablation.

The third check is small-data separation. Against certainty-equivalence IOP, the stability rate should differ substantially at short data lengths and converge at long ones.

If the first check fails because $\hat\delta$ is too optimistic, or the method is infeasible because $\hat\delta$ is too loose, the synthesis idea survives but the program becomes a study of gap-radius estimation. We would report that outcome as such.