Controllers that are optimal over an infinite horizon, for example ones robust to uncertain noise statistics or ones that minimize regret against a clairvoyant controller, usually cannot be built. Their transfer functions are non-rational, so running them would need unbounded memory. Current practice has two steps. First compute the ideal controller in the frequency domain, then fit a low-order rational approximation to it. The fitting step ignores hard limits on states and actuators, and the whole design is done offline for one assumed noise distribution. We propose to remove both steps. The controller is written from the start as a sum of a few fixed decaying modes, called simple poles, and the learnable parameters are the residues, the coefficient matrices that weight each mode. For any plant these residues live in a convex set that contains every stable closed-loop behavior satisfying hard state and input bounds over the infinite horizon. The residues are then tuned online from observed disturbances by gradient steps, and after each step the residues are projected back onto that set. Every iterate is therefore a finite-order controller that can be implemented, and every iterate is certified safe. Because projection never lengthens a step, the controller also changes slowly enough to keep switching between iterates stable.
In two sentences: fix a small set of poles, and describe a whole family of stable, constraint-satisfying, finite-order controllers as a convex set of residue matrices. Then learn inside that set online with projected stochastic gradient, so the controller adapts to the actual disturbances and never leaves the set where rationality and safety are guaranteed.
Core Claim
System level synthesis designs a controller by optimizing the closed-loop responses $\Phi_x, \Phi_u$, the maps from disturbance to state and to input. Doing so turns stabilization into the affine constraint $(zI-A)\Phi_x - B\Phi_u = I$, where $A, B$ are the plant matrices and $z$ is the shift operator. Approximation by Simple Poles - Part II supplies the method we reuse. It restricts each response to $\Phi(z) = \sum_{i=1}^M R_i/(z-p_i) + D$ over a fixed pole set $\{p_i\}$ inside the unit disc. It shows the synthesis stays convex and always feasible for stabilizable plants, and that suboptimality shrinks linearly with the distance from the chosen poles to the optimal ones. That paper explicitly leaves hard constraints and online adaptation open. Our first claim is that hard infinite-horizon constraints are convex in the residues.
For a disturbance with $\|w_t\|_\infty \le \bar w$, the peak state satisfies $|x_t|_j \le \bar w \, \|e_j^\top \Phi_x\|_{\ell_1}$, where $e_j$ selects the $j$-th state and $\|\cdot\|_{\ell_1}$ is the sum of absolute impulse-response entries. The impulse response of each pole term is $R_i p_i^{k-1}$, so this norm is bounded by
$$ \|e_j^\top \Phi_x\|_{\ell_1} \le \|e_j^\top D_x\|_1 + \sum_{k=1}^{K}\|e_j^\top \textstyle\sum_i R_i p_i^{k-1}\|_1 + \sum_i \frac{|p_i|^K}{1-|p_i|}\|e_j^\top R_i\|_1 . $$
Here $K$ is a truncation length. The right side is convex in the residues, and the tail term is an exact geometric bound, not an FIR cutoff. Increasing $K$ tightens the bound without adding decision variables. The same construction applies to input bounds through $\Phi_u$. The achievability equation is linear in the residues, so the feasible set $\mathcal{S}$ is a polytope intersected with an affine subspace.
The second claim concerns learning. The disturbance statistics are unknown and may drift, so the residues $\theta = (R_i, D)$ are updated as
$$ \theta_{t+1} = \Pi_{\mathcal{S}}\big(\theta_t - \eta_t \hat g_t\big), $$
where $\hat g_t$ is a stochastic gradient of the observed quadratic cost, formed from a sliding window of reconstructed disturbances, and $\eta_t$ is a step size. The projection $\Pi_{\mathcal{S}}$ enforces achievability and constraints exactly. Because projection is non-expansive, as used in Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization and in projected TD learning (TD(lambda) Converges with Probability 1), the controller moves by at most $\eta_t\|\hat g_t\|$ per step. This bounded variation is the ingredient needed to extend safety from each frozen controller to the time-varying closed loop.
Relation to Prior Work
Three lines of work motivate the gap. The first computes infinite-horizon optimal controllers and then fits rational approximations after the fact. The Distributionally Robust Infinite-Horizon LQR shows a second-order fit often lands within 1% of optimum but warns that too low an order can inflate cost badly. Optimal Infinite-Horizon Mixed H2/H∞ Control proves the optimal controller is non-rational and fits a 4th-order approximation on REA4. Infinite-Horizon Distributionally Robust Regret-Optimal Control solves regret-optimal control by Frank-Wolfe and then applies H∞ model reduction. All three are offline and unconstrained. We make the rational structure the search space instead of a later fitting step, and we add hard constraints.
The second line enforces hard constraints through closed-loop responses, but only with FIR truncation. Safely Learning to Control the Constrained LQR does this for safe exploration with batch least squares. Distributionally Robust Infinite-horizon Control and Data-Driven Distributionally Robust SLS do it under Wasserstein ambiguity. A System Level Approach to Regret Optimal Control and Safe Control with Minimal Regret do it for regret, over finite horizons. Their FIR responses force the state to exactly zero after $L$ steps, which Approximation by Simple Poles shows produces poor damping and actuator spikes and requires very large $L$ when a system has both fast and slow modes. We keep their constraint logic and replace the FIR basis.
The third line is adaptive control with projection. Adaptive Economic MPC projects least-mean-square plant estimates onto a plausible set. That work constrains us. Its guarantees need open-loop stable plants and soft constraints, and it projects plant estimates rather than controller parameters. We project the controller parameters themselves onto a set that already certifies stabilization, so unstable plants such as the double integrator are covered. An Input-Output Parametrization of Stabilizing Controllers uses Ritz bases in a similar spirit for output feedback. We treat it as the route to an output-feedback extension, not as a competitor, because it has neither hard constraints nor online updates.
Research Program
The first stage is the offline object. We will derive the residue-space constraint set $\mathcal{S}$ for state and input box constraints under bounded disturbances. We will quantify how conservative the geometric tail bound is as a function of $K$ and of pole radius. We will also show that Fisher et al.'s feasibility guarantee survives the constraints whenever the constrained problem is feasible for some stable response with poles near the chosen set; their linear suboptimality-in-pole-distance argument should carry over once the constraints are added.
The second stage is the online algorithm. The affine achievability constraint has a projection that can be precomputed in closed form. The inequality part is handled by a warm-started splitting method, Dykstra or ADMM, with a fixed small iteration budget, and we will measure how often the budget is enough. Disturbances are reconstructed from the state and the known model. The gradient comes from the frozen-controller cost over a window.
The third stage is the theory that matters most. We will prove that if every $\theta_t \in \mathcal{S}$ satisfies constraints with margin $\epsilon$, and per-step variation is bounded by $\eta\,G$ where $G$ bounds the gradient, then the time-varying closed loop stays stable and satisfies constraints. We need a stated relation between $\eta$, $\epsilon$, and the slowest pole radius. The margin shrinks as poles approach the unit circle, which is a real tension with modeling slow modes, and we expect this to be the main theoretical cost.
A final extension, attempted only if the stages above succeed, replaces the known plant with a projected least-mean-square estimate. It enlarges $\mathcal{S}$'s tightening with a small-gain robustness term as in Safely Learning to Control the Constrained LQR.
Baselines
The naive reference is the unconstrained H2-optimal (LQR) controller. It shows how often constraints are violated when they are ignored, and its cost is the floor under Gaussian noise. Safe Control with Minimal Regret reports H2 about 10% better than its safe regret controller under pure Gaussian noise. It also reports that classical H2 and H∞ controllers incur 40% to over 100% extra cost under structured disturbances. These figures set expectations for both regimes.
The strongest existing constrained method is FIR-based robust SLS with hard constraints, as in Distributionally Robust Infinite-horizon Control and Safely Learning to Control the Constrained LQR. It will be run on identical disturbance samples and constraint sets, with $L$ swept until either cost saturates or solve time exceeds a budget matched to ours. No reported result was found on Lacuna for DRInC's aggregate cost on the double integrator; the file reports only zero safety violations across 1,000 test distributions. We therefore rerun it ourselves.
For the unconstrained infinite-horizon line, the baseline is the two-step pipeline of Infinite-Horizon Distributionally Robust Regret-Optimal Control followed by H∞ reduction to the same order as our controller. The reported anchors are the AC17 numbers from Optimal Infinite-Horizon Mixed H2/H∞ Control: H2 cost 54.28 with H∞ norm 81.3 for pure H2, and H2 cost 57.92 at γ = 60. Our unconstrained limit ($\mathcal{S}$ without inequalities) should reproduce these values, which checks the implementation.
Two ablations isolate the ingredients. The first is offline constrained simple-pole SLS solved once for the nominal Gaussian disturbance, which removes online projection. The second is online projected FIR-SLS with the same update rule, which removes the rational basis. Beating the method means, at an equal controller order and an equal per-step compute budget, zero constraint violations with lower average cost than FIR-SLS on multiscale plants, and lower cost than the offline simple-pole design under non-Gaussian or drifting disturbances.
Why not just…?
Why not just use FIR-SLS with a long horizon? On plants with a single time scale it may be competitive, and we will report that honestly. Approximation by Simple Poles shows that FIR responses force deadbeat behavior and scale badly when fast and slow modes coexist. The multiscale plant and REA4 experiments measure cost, peak input, and solve time against $L$ to show where the crossover occurs.
Why not just fit a low-order model to the unconstrained optimal controller? Post-hoc reduction as in The Distributionally Robust Infinite-Horizon LQR certifies nothing about time-domain peaks. We will count constraint violations of the reduced controller under the same disturbances. If it happens to be safe on some benchmark, the comparison reduces to cost, and the method's value there rests only on adaptation.
Why not just re-estimate the disturbance statistics from a sliding window and re-solve the constrained simple-pole program? This is the real threat. The projection is a program of similar size, so compute alone does not separate the two approaches. The differences are two. Re-solving can jump arbitrarily between controllers, which voids the switching-stability argument unless jumps are rate-limited, and rate-limiting a re-solve amounts to our projected step. Re-solving also needs an explicit spectral model of the disturbance, whereas the gradient uses the cost directly. We include periodic certainty-equivalent re-solving as a baseline at matched compute. If it matches our cost and never loses stability, the contribution shrinks to the offline constrained parameterization together with the switching guarantee, and we will say so.
Why not just use adaptive MPC with projected parameter estimates? Adaptive Economic MPC relies on soft constraints and open-loop stable plants. We will run it on the open-loop unstable double integrator and report violations and failures of recursive feasibility. On the stable 2R2C thermal model it serves as a fair comparison.
Proposed Experiments
The plants are the double integrator from Safely Learning to Control the Constrained LQR and DRInC, the mass-spring-damper from A System Level Approach to Regret Optimal Control with its bound $x_1 \le 25$, and discretized COMPleib AC15, AC17, and REA4 with added box constraints. We also include a constructed two-time-scale plant. Disturbances follow the protocol of Safe Control with Minimal Regret: Gaussian, sinusoidal, sawtooth, and step profiles. We add a drifting regime in which the profile switches mid-run. Metrics are average cost, cost gap to the clairvoyant controller over long horizons, constraint violations over 1,000 realizations, peak input, controller order, and wall-clock time per update.
Three checks are decisive. First, the offline constrained simple-pole design must match or beat FIR-SLS in cost at a much lower order on REA4 and the two-time-scale plant, with zero violations. Second, under switching disturbances, the online projected controller must reduce cost relative to the offline design while every iterate remains safe, and the observed per-step variation must stay within the bound the theory requires. Third, against periodic re-solving at matched compute, the projected update must show either lower cost or strictly fewer transient violations. If neither appears, the online component is not justified.
References
- Approximation by Simple Poles - Part II: System Level Synthesis Beyond Finite Impulse Response
- Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
- TD(lambda) Converges with Probability 1
- The Distributionally Robust Infinite-Horizon LQR
- Optimal Infinite-Horizon Mixed H2/ H∞ Control
- Infinite-Horizon Distributionally Robust Regret-Optimal Control
- Safely Learning to Control the Constrained Linear Quadratic Regulator
- Distributionally Robust Infinite-Horizon Control: From a Pool of Samples to the Design of Dependable Controllers
- Data-Driven Distributionally Robust System Level Synthesis
- A System Level Approach to Regret Optimal Control
- Safe Control with Minimal Regret
- Adaptive Economic Model Predictive Control for linear systems with performance guarantees
- An Input-Output Parametrization of Stabilizing Controllers: Amidst Youla and System Level Synthesis