Stability-Certified Sensitivity Projection for Fast Online Feedback Optimization

Online feedback optimization (OFO) runs an optimization algorithm directly as a controller on a live physical system, such as a power grid, a building, or an oil field. Instead of solving a planning problem on a computer model, the controller nudges the plant's inputs a little at every sampling step and uses measured outputs as feedback. It drives the plant toward the best operating point even when the model is wrong. The controller needs one model ingredient: the sensitivity matrix, which says how much each measured output moves when each input is changed. When this matrix is learned online, for example by recursive least squares, the controller adapts to changing conditions. It can also fail badly. If the data stop being informative, say during a quiet stretch with nothing exciting the system, or right after a line outage, the estimate can drift. Without warning, the controller is then optimizing with a sensitivity that points the wrong way.

We propose to keep the learned sensitivity inside a set that is certified to be safe for the closed loop, and to project the estimate back onto that set at every step. The set is not a generic box of physically plausible values. It is the set of sensitivity estimates for which the OFO loop at its current, aggressive gain provably keeps contracting, given what the data say the true sensitivity could be. In two sentences: we turn a closed-loop stability condition into a convex constraint on the learned model, and project onto it online. As a result, every intermediate controller the system ever runs is stabilizing, even without probing noise and without slowing the controller down.

Core Claim

The three directions in the brief meet at one specific point. Parameter projection is the classical trick of clipping iterates onto a convex set that contains the truth, so they cannot drift away from it. Examples include projected TD learning in TD(lambda) Converges with Probability 1 and the projected least-mean-squares filter in Adaptive Economic Model Predictive Control for linear systems with performance guarantees. Sequential strong stability means that every controller produced during learning stabilizes the plant, not just the final one. Our claim is that in data-driven OFO, the right set to project the sensitivity estimate onto is a convex inner approximation of "sensitivities under which this OFO loop is certified to converge for every plant consistent with the data." Projecting onto that set gives sequential strong stability of the coupled plant, estimator and controller. Two consequences follow:

  1. Fast gains in the spirit of A Stability Condition for Online Feedback Optimization without Timescale Separation become usable with a learned model.
  2. Stability no longer depends on persistent probing noise. Excitation then only affects how quickly the controller reaches optimality.

Concretely, the controller updates inputs $u_k$ by

$$u_{k+1} = \Pi_{\mathcal U}\big[u_k - \alpha\,(\nabla_u f(u_k) + \hat H_k^\top \nabla_y g(y_k))\big],$$

where:

  • $\Pi_{\mathcal U}$ is projection onto the input limits;
  • $\alpha$ is the controller gain;
  • $f$ and $g$ are the costs on inputs and on measured outputs $y_k$;
  • $\hat H_k$ is the current sensitivity estimate.

Least squares supplies a confidence set $\mathcal E_k$ that contains the true sensitivity $H$ with high probability. We define the certified set

$$\mathcal C_k = \{\hat H : M(\hat H, H; \alpha) \succeq 0 \ \text{ for all } H \in \mathcal E_k\}.$$

Here $M$ is the matrix inequality from a discrete-time version of the composite Lyapunov argument (described below). For each fixed $H$, $M$ is affine in $\hat H$. So $\mathcal C_k$ is an intersection of convex sets, and the S-lemma turns it into a single linear matrix inequality over an ellipsoid. The estimator then uses $\hat H_k = \Pi_{\mathcal C_k}(\hat H_k^{\text{RLS}})$.

The link to classical projection is direct. Once the confidence radius is small enough that the true $H$ itself lies in $\mathcal C_k$, the projection is non-expansive toward the truth. It therefore never makes the estimate worse, and it only removes the destabilizing part of a drifted estimate.

Research Program

Certificate. The analytical core extends the max-type Lyapunov function $V=\max\{\xi V_x, V_u\}$ of A Stability Condition for Online Feedback Optimization without Timescale Separation in two ways. That paper treats the exact-model case for input-constrained OFO. We need, first, a version that holds in discrete time and, second, one that holds with a mismatched sensitivity. The paper explicitly leaves discrete time open, and it does not consider learned models. The mismatch enters the dominance condition through an alignment term between $\hat H$ and $H$, and through a norm bound on $\hat H$. Writing this as $M(\hat H, H;\alpha)\succeq 0$ is the step that turns a stability theorem into a projectable set. This step is genuinely uncertain: the resulting condition may be too conservative at high gains.

Sequential stability guarantee. The target theorem is uniform boundedness of the plant state and exponential convergence to a neighborhood of the optimum whenever $\mathcal C_k$ is nonempty. The neighborhood should shrink with the confidence radius.

Fallback when the set is empty. If $\mathcal C_k$ is empty, which can happen early in learning or after a topology change, the rule is to hold the last certified estimate and reduce $\alpha$ to the largest gain for which that estimate is still certified. Restoring certification is then a one-dimensional search over the gain, not over trial rollouts. This differs from the spectral-radius backtracking of Structured Policy Iteration for Linear Quadratic Regulator. That method needs exact model evaluations at each candidate step. Ours needs only the confidence set already maintained by the estimator.

Probing noise. We will prove that stability holds with zero probing noise: the set only shrinks when data are informative, and is otherwise just conservative. Excitation enters only the optimality bound. This is the contrast with Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation and with Adaptive Control of Unknown Linear Switched Systems via Policy Gradient Methods. Both inject probing noise continuously, and the latter attributes its conservative state bounds to that noise.

Computation. The projection is a small semidefinite program in the entries of $\hat H$. We will measure its solve time against the sampling period.

Scope. We restrict to open-loop stable plants, where OFO is used in practice. We treat abrupt changes, such as line outages, as the switching regime. We do not claim results for open-loop unstable plants.

Baselines

Naive reference: frozen sensitivity. OFO with a fixed nominal sensitivity $H_0$, at the same gain, is the "do nothing" reference. Its known failure mode is drift. In Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation, the static-model controller diverged under a 20% admittance-matrix error while the adaptive one remained stable. We will run it with the same $H_0$ initializations as every other method.

Strongest existing method: unprojected RLS-OFO. This is the recursive-least-squares OFO controller of Picallo et al., with probing noise. It is the only method in the file that learns sensitivities inside an OFO loop on a grid benchmark. The file gives no numerical tracking-error or violation count for it: no reported result found on Lacuna for RLS-OFO tracking error or voltage-violation counts on IEEE 123-bus, beyond the qualitative "drastically reduced the number of voltage violations." We will reimplement it on the same feeder, load and generation traces. Its forgetting factor and noise amplitude will get the same tuning budget, a fixed grid search, as our method's confidence level. Beating it means three things:

  • no increase in time-averaged suboptimality relative to the AC optimal power flow optimum in excitation-rich runs;
  • strictly fewer voltage violations and bounded states in excitation-starved and post-outage runs, where it is expected to drift;
  • all of this with probing noise switched off for our method.

Ablation: box projection. This isolates the key ingredient. It is the same RLS-OFO with projection onto a box of physically plausible sensitivities, mirroring the projected filter of Adaptive Economic Model Predictive Control for linear systems with performance guarantees. If box projection matches the certified set on every scenario, the certificate is not earning its keep.

Slow-gain reference. OFO at a timescale-separated gain, with the same RLS estimator, shows what speed the certificate buys. On the synthetic plant of Bianchi and Dörfler, the earlier bound was $\alpha<10.1$ and their condition allowed $\alpha=100$. We will test whether our certified set preserves operation near $\alpha=100$ with a learned model.

Why not just…?

Why not just use a small gain so the plant is always quasi-static? This is safe and simple, and with plain RLS it may well be stable. Its cost is speed. A Stability Condition for Online Feedback Optimization without Timescale Separation reports that low gains "resulted in significant lag" after disturbances, while $\alpha=100$ tracked almost instantly. The slow-gain baseline will quantify that lag on fast solar and wind traces. Small gains also do not prevent drift of the estimate itself; they only slow its consequences.

Why not just project onto a box of physically plausible sensitivities? This is the most serious threat. It is exactly what Adaptive Economic Model Predictive Control for linear systems with performance guarantees does for its model parameters. A box bounds the estimate's size but not its direction. A drifted estimate can stay inside the box and still be misaligned with the true sensitivity enough to push outputs the wrong way at high gain. The box ablation, run in the excitation-starved and outage scenarios, settles this. If the box never produces a certificate failure that the certified set prevents, we will report that the cheaper method suffices. The contribution then reduces to the stability proof for box-projected OFO.

Why not just monitor the loop and revert to the nominal model when something looks wrong? A safety-filter style fallback detects instability only after outputs misbehave, which is too late for hard voltage limits. It also needs a detection threshold that trades false alarms against delay. Backtracking on a stability measure, as in Structured Policy Iteration for Linear Quadratic Regulator, avoids this but requires evaluating candidate controllers, which is impossible on one live trajectory. We will include a reactive-reversion variant in the outage scenario and count the violations it incurs before triggering.

Proposed Experiments

IEEE 123-bus distribution feeder. This is the main testbed, using the setup of Adaptive Real-Time Grid Operation via Online Feedback Optimization with Sensitivity Estimation: ECO household loads at 1-second resolution, and solar and wind traces from HelioClim-3 and MERRA-2. Initial admittance errors range from 0% to 40%, which includes the reported 20% case. We report four quantities:

  • time-averaged gap to the instantaneous AC optimal power flow optimum;
  • count and integral of voltage violations;
  • maximum output excursion;
  • the fraction of steps at which each method's current estimate fails the certificate, evaluated after the fact against the true sensitivity.

The decisive experiment has three phases. First, a two-hour low-variability period with probing noise off. Second, a scripted line outage that changes the true sensitivity abruptly. Third, a return to volatile generation. The prediction is that unprojected RLS accumulates certificate failures in the quiet phase and violations after the outage. Certified projection should show zero certificate failures and bounded states throughout, at worst with a temporary gain reduction from the fallback rule.

Synthetic plant from Bianchi and Dörfler. On their LTI plant, we sweep $\alpha$ from 10 to 100 with a learned sensitivity. This tests whether the certified set stays nonempty at high gain or collapses, which is the main risk of conservatism.

Blocaux subtransmission area. Using only continuous setpoints from Subtransmission grid control via online feedback optimization, we test scale and timing. That paper reports 88.9% of available wind injected and a step solve under 40 milliseconds with a 10-second sampling period. Our projection must fit in the same budget without lowering wind injection relative to an RLS controller that is given the true sensitivity.

Gas lift. We check transfer to the 5-well gas-lift case of Sensitivity of Online Feedback Optimization to time-varying parameters. There, a predicted 14% loss versus an actual 6.1% loss under parameter mismatch gives a reference scale for how much a misestimated sensitivity costs.

Failure criterion. The proposal fails if any of three things happens: the certified set is empty most of the time at useful gains; box projection matches it in every scenario; or the projection cannot run within the sampling period.