Consider learning a model of an unknown linear system while controlling it, when the motors or valves can only push so hard and the disturbances are Gaussian, so occasionally arbitrarily large. A sharp theoretical result says this works only if the plant is stable or marginally stable on its own, before any control is applied. If any mode of the plant grows on its own, a large enough random shock eventually outruns the bounded actuator. The open-loop stability of the true plant is therefore a precondition of the whole problem. Yet the online estimators used in adaptive controllers routinely produce intermediate models that are unstable, and the controller then plans against a plant it could not possibly stabilize.
We propose to build the known feasibility boundary directly into the estimator. After each online least-squares update, the estimated state matrix is projected onto a convex set of matrices that contract in a fixed Lyapunov-weighted norm. A Lyapunov-weighted norm is an energy-like measure of the state, under which a stable system never grows. In that norm, the projection is a closed-form singular-value clip. It is non-expansive, meaning it never moves the estimate farther from the truth, provided the true plant lies in the set. It also guarantees that every model the controller ever sees is one that bounded control can hold in mean square. The question we test is whether this cheap, certified projection beats both the generic convex parameter sets used today and the obvious hack of clipping eigenvalues. We measure that on transient state variance, cost, and actuator chattering.
Core Claim
The setting is a discrete-time plant $x_{t+1}=Ax_t+Bu_t+w_t$ with hard input bound $\|u_t\|\le R$ and Gaussian $w_t$. Bounded control can keep $\sup_t \mathbb{E}\|x_t\|^2$ finite when $A$ is marginally stable with non-defective unit eigenvalues. This was shown by the decomposition-and-push-back construction of Attaining Mean Square Boundedness of a Marginally Stable Stochastic Linear System With a Bounded Control Input, and by simple saturated feedback under reachability in Moment stability of stochastic processes with applications to control systems. It is impossible when the spectral radius exceeds one, per On Mean-Square Boundedness of Stochastic Linear Systems With Quantized Observations and Stabilizing Stochastic Predictive Control under Bernoulli Dropouts. These papers motivate the proposal: they tell us exactly which models are admissible.
The obstacle is geometric. The set of stable matrices is non-convex for $n\ge 2$, so it cannot serve directly as the convex set that makes projected updates well-behaved. Our observation is that for any fixed positive definite $P$, the set
$$\mathcal{C}(P,\rho)=\{A:\ A^\top P A \preceq \rho^2 P\}$$
is convex. Here $\rho\le 1$ is a contraction level. Every $A$ in this set is stable with Lyapunov certificate $P$, and the union of these sets over all $P$ recovers the admissible plants. Write $\tilde A = P^{1/2}AP^{-1/2}$. Membership in $\mathcal{C}(P,\rho)$ is then just $\sigma_{\max}(\tilde A)\le\rho$, where $\sigma_{\max}$ is the largest singular value. Euclidean projection in the weighted metric $\|X\|_P=\|P^{1/2}XP^{-1/2}\|_F$ therefore consists of three steps: transform the estimate, clip its singular values at $\rho$, and transform back.
This is cheaper than an eigendecomposition and is a true projection. It never increases the $\|\cdot\|_P$ distance to $A^*$ whenever $A^*\in\mathcal{C}(P,\rho)$. For marginally stable plants with $\rho=1$, the set contains rotations and other unit-circle dynamics expressed in $P$-coordinates. This sidesteps the complaint in the file that plain spectral-norm balls exclude non-normal oscillatory systems: those systems are excluded only for $P=I$.
What is new is the content of the projection set, not the act of projecting. Adaptive Economic Model Predictive Control for linear systems with performance guarantees supplies the template: projected least-mean-squares adaptation inside economic model predictive control, with the true plant assumed open-loop stable. However, its projection set is described only as a compact convex set of physically plausible models. Nothing indicates that this set encodes the stability assumption the proofs rely on, and the paper uses soft rather than hard input constraints.
Research Program
The first component is the algorithm. We use a preconditioned least-mean-squares step on $(A,B)$, with the gradient taken in the $P$-weighted geometry so the standard projected-descent analysis applies. This is followed by the singular-value clip on $A$ and a box projection on $B$ to keep the plant reachable. The controller is the bounded-input policy that is certainty-equivalent to the current estimate, meaning it treats the estimate as if it were the true plant. We use two such controllers: the saturated feedback of Moment stability of stochastic processes with applications to control systems, and the economic predictive controller of Degner et al. with the soft input penalty replaced by a hard saturation.
The second component is the choice of $P$, which is the crux and the main risk. We study three regimes in order of difficulty. In the first, $P$ comes from physics, such as stored thermal energy in the building model. In the second, $P$ is solved once by semidefinite programming so that it certifies every matrix in an initial least-squares confidence ellipsoid. In the third, $P$ is adapted slowly on a second timescale, and each change is confined to a small neighborhood of the previous $P$. This is the same device by which Breaking the Deadly Triad with a Target Network confines a slow target network relative to the fast weights to keep the iteration contractive.
The third component is analysis. In regimes one and two, we aim to show that projected estimation error decays at the usual least-mean-squares rate while the closed-loop state stays mean-square bounded. The proof would combine non-expansiveness with the Foster–Lyapunov drift arguments of the stability literature. The file flags that those arguments use fourth moments of the noise while projected-gradient analyses use second moments, so we will state the result under the fourth-moment condition and identify where it enters. We will not claim to close Gap 3 in general.
We deliberately exclude strictly unstable plants. The impossibility theorems say no estimator can rescue them under bounded inputs and Gaussian noise.
Baselines
The naive reference is unprojected recursive least squares feeding the same certainty-equivalent controller. It shows how often, and for how long, raw estimates leave the admissible region, and what that costs in state variance.
The strongest existing method is projected least-mean-squares adaptive economic predictive control from Adaptive Economic Model Predictive Control for linear systems with performance guarantees. It is the only work in the file combining online parameter projection with an open-loop stability assumption. We run it with its convex parameter box, on identical noise seeds, the same data stream, the same step-size grid, and the same horizon. Inputs will be hard-saturated at the same $R$ for all methods. We also report its original soft-constraint variant separately, so the comparison does not hinge on our modification.
The file reports only qualitative results for this baseline on the 2R2C building model: much smaller temperature oscillations and lower accumulated cost than a non-adaptive controller over about 20 days. No reported number was found on Lacuna for its cost or variance, so we will reproduce it ourselves. Beating it means lower cumulative cost and a lower peak in the Monte Carlo estimate of $\mathbb{E}\|x_t\|^2$ during the first adaptation phase, with no worse asymptotic cost.
Two ablations isolate the key ingredient. Spectral-norm clipping with $P=I$ tests whether the Lyapunov weighting matters beyond simply bounding the matrix. Fixed-$P$ versus slowly adapted $P$ tests whether adaptation of the certificate is needed at all.
Why not just…?
Why not just clip the eigenvalues of the estimate into the unit disk? This is the real threat. It is simple and enforces stability directly. However, it is not a projection. For non-normal or nearly defective estimates it can move the estimate far from the truth, and it requires an eigendecomposition that is numerically fragile. That fragility is the problem Stable Spectral Learning Based on Schur Decomposition addresses, where eigenvalue-based recovery error was 0.120 against 0.039 for a Schur-based method at $N=2000$. We include eigenvalue clipping as a baseline on deliberately non-normal plants. If it matches our method on cost and variance, we will report that the certificate buys only theory, not performance.
Why not just add a penalty or barrier on the eigenvalues? Soft penalties do not bound iterates. In Breaking the Deadly Triad with a Target Network, regularized off-policy methods diverged to errors of $10^5$ on Baird's counterexample, while projection gave stability even with zero regularization. We include an eigenvalue log-barrier baseline, which tests whether that lesson transfers to system identification.
Why not just wait, since least squares converges to the stable truth anyway? This is honest: the benefit is transient. Our evaluation therefore reports early-phase metrics separately, together with the fraction of steps on which the raw estimate is unstable. If that fraction is negligible on a benchmark, projection is unnecessary there and we will say so.
Why not just use a Euclidean norm ball? A norm ball large enough to contain non-normal stable plants contains unstable ones. The $P=I$ ablation and the AE-MPC box quantify this cost directly.
Proposed Experiments
The first testbeds are the stochastic stability benchmarks with Gaussian noise and bounded $R$, now with unknown $(A,B)$. These are the 4D system of a rotating block plus a Schur-stable block from Ramponi et al., the 3D marginally stable system from Stabilizing Stochastic Predictive Control under Bernoulli Dropouts, and the 2D rotation with radial quantization from On Mean-Square Boundedness of Stochastic Linear Systems With Quantized Observations. For the latter we use 8 bins, the setting reported as nearly matching infinite precision. We add a randomized family of non-normal marginally stable 4D plants, where $P\neq I$ should matter most.
The application testbed is the 2R2C building model of Degner et al. We also include the 4th-order linear system of Towards non-stochastic targeted exploration with Gaussian rather than energy-bounded noise. That paper's parameter errors, 0.0389 for stochastic and 0.0257 for designed exploration, come from an open-loop identification setting. They serve only as a scale reference for identification accuracy, not as a closed-loop baseline.
Metrics are computed over at least 200 seeds. They are the Monte Carlo running peak and tail of $\mathbb{E}\|x_t\|^2$, cumulative cost, $\|\hat A_t-A^*\|_P$, the fraction of steps with $\rho(\hat A_t)>1$, saturation switching rate as a measure of chattering, and per-step computation.
Two checks decide the proposal. The first is whether the Lyapunov-metric projection lowers transient variance and cost against both the AE-MPC box and eigenvalue clipping on the non-normal family. The second is whether, for marginally stable plants whose truth sits on the boundary of $\mathcal{C}(P,1)$, projection induces switching between $\pm R$ that inflates variance. Gap 4 of the file flags this chattering risk. If it does, we will compare against the smooth projection of Adaptive Spacecraft Attitude Tracking Control with Actuator Uncertainties applied at the same boundary.
References
- Attaining Mean Square Boundedness of a Marginally Stable Stochastic Linear System With a Bounded Control Input
- Moment stability of stochastic processes with applications to control systems
- On mean-square boundedness of stochastic linear systems with quantized observations
- Stabilizing Stochastic Predictive Control Under Bernoulli Dropouts
- Adaptive Economic Model Predictive Control for linear systems with performance guarantees
- Breaking the Deadly Triad with a Target Network
- Stable Spectral Learning Based on Schur Decomposition
- Towards targeted exploration for non-stochastic disturbances
- Adaptive Spacecraft Attitude Tracking Control with Actuator Uncertainties