Describing Functions for Projected Feedback Optimizers

Many learning and control loops keep their updates inside safe bounds by projection: after each gradient step, any value that leaves the allowed set is clipped back onto it. When such a projected optimizer runs online against a physical plant, the clipping and the step accumulation together form a nonlinear feedback element. Current analysis proves these loops stable in idealized continuous time. It cannot say whether, under realistic actuator lag, sampling or delay, the loop will settle or lock into a sustained oscillation against the bounds, and if it oscillates, at what frequency and amplitude. We propose to answer that question with harmonic linearization. This is the classical engineering method that replaces a nonlinearity with an amplitude-dependent gain obtained from the first Fourier harmonic of its response to a sine wave.

The idea in two sentences: a projected gradient update on a box is exactly an integrator whose state is clamped at the bounds. If we compute the describing function of this "clamped integrator" (its amplitude- and frequency-dependent equivalent gain), we can predict, using standard frequency-domain tools, the controller gain at which a projected optimizer starts to oscillate against its constraints, along with the frequency and size of the resulting cycle. That prediction becomes a cheap design rule for choosing aggressive but safe gains. It replaces both conservative time-domain bounds and brute-force simulation sweeps.

Core Claim

Two lines of work meet here without having touched each other. On one side, A Stability Condition for Online Feedback Optimization without Timescale Separation shows that projected online feedback optimization stays exponentially stable for any controller gain $\alpha>0$ under a parameter-dominance condition. Online feedback optimization means running gradient descent on a cost while it drives a live plant. That paper raises usable gains from the singular-perturbation bound $\alpha<10.1$ to $\alpha=100$, and it notes that at high gain the controller approaches bang-bang behaviour. Its analysis is entirely in the time domain and assumes an ideal continuous-time loop. On the other side, Large-Signal Impedance Modeling and Stability Analysis of a Grid-Connected Inverter Considering the Influence of a Limiter in Different Control Links uses describing functions of hard limiters to predict sustained inverter oscillations accurately, at "around 54%V1 at 565 Hz" and "64%V1 at 535 Hz", verified by FFT. Those limiters are static clamps on signals inside a fixed controller. Nothing is being learned or optimized.

Our claim is that the projection inside an online optimizer is not a static limiter. Projected gradient steps on an interval $[a,b]$ have the form

$$u_{k+1} = \Pi_{[a,b]}\big(u_k - \alpha\, g_k\big),$$

where $u_k$ is the input, $g_k$ the measured gradient and $\alpha$ the gain. This is an integrator with an anti-windup clamp, a dynamic nonlinearity with memory. Its describing function $N(A,\omega)$ is therefore complex and depends on both the amplitude $A$ and the frequency $\omega$ of the gradient signal, much like the backlash and hysteresis cases in the classical theory. A sustained oscillation of the closed loop with linear plant-plus-sensing dynamics $G(j\omega)$ is predicted where the harmonic balance condition holds:

$$G(j\omega)\,N(A,\omega) = -1.$$

Solving this condition gives three predictions with no long simulation: the onset gain, the cycle frequency, and the amplitude at the bound. It also tells us which unmodeled effects matter, namely delay, actuator lag and sampling. This is not in conflict with Bianchi and Dörfler. Their guarantee covers the ideal model, and a correct describing-function analysis must predict no cycle there. We test that consistency explicitly. The new content concerns what happens once their assumptions break in the ways any physical deployment breaks them.

Research Program

The first piece of work is analytical. We derive the describing function of the clamped integrator, in both continuous time (tangent-cone projected flow) and discrete time (the update above). We treat the sinusoidal gradient input as possibly offset, because optimizers operate near an active bound rather than symmetrically about it. That offset case requires a dual-input describing function covering the bias and the fundamental. We extend the result from boxes to Euclidean balls by working with the radial component, which covers the ball projections used in Breaking the Deadly Triad with a Target Network. The ball case is a secondary target; we do not claim results for stochastic reinforcement-learning iterates.

The second piece turns the analysis into a design rule. For a given plant model with lag or delay, we compute the largest gain $\alpha$ whose Nyquist locus $G(j\omega)$ avoids the critical locus $-1/N(A,\omega)$ for all reachable amplitudes, with a stated margin. This is the "describing-function gain bound." It should sit between the conservative timescale-separation bound and the onset of oscillation.

The third piece is a validity study. The method's known weakness is its low-pass assumption: the plant must filter out the higher harmonics generated by the clamp. The research file flags this as the main failure mode, for example in Frequency Domain Analysis of Nonlinear Series Elastic Actuator via Describing Function, which reports a describing-function model agreeing with Simscape "typically within $\pm 1$ Hz". Projection injects sharp corners, so the assumption could fail exactly where it is needed. We therefore also implement a few-harmonic balance (three to five harmonics) as a correction. We map, as a function of plant roll-off and gain, where the one-harmonic prediction suffices and where it breaks.

Baselines

The naive reference point is small-signal linearization. In the interior of the constraint set, the projected update is treated as a plain integrator with gain $\alpha$, and loop stability is read off the linear Nyquist plot. It predicts a single critical gain and no amplitude. Near an active bound it gives either the interior gain or zero, neither of which is right. We run it on identical plant models. Beating it means correctly predicting cycles that it calls either stable or unbounded. The inverter paper's statement that small-signal impedance "cannot accurately describe the characteristics of the limiter" motivates the expectation that it fails.

The strongest existing analytical competitor is the time-domain certificate of A Stability Condition for Online Feedback Optimization without Timescale Separation, together with the older singular-perturbation bound it improves on. The reported numbers are a prior bound of $\alpha<10.1$ and stable operation at $\alpha=100$ on their LTI plant. We apply both certificates to the same plants, with and without the added lag and delay. Beating them means two things. In the ideal case the describing-function bound should agree with them. In the lagged case it should give a gain bound that is valid, meaning confirmed oscillation-free in simulation, and larger than any bound they still certify. It should also supply the cycle amplitude, which they cannot. No reported result was found on Lacuna for oscillation amplitude or frequency of projected feedback optimization under delay, so the ground truth comes from our own simulations.

The strongest practical competitor is brute-force time-domain simulation over a grid of gains and disturbance amplitudes. This is the method that answers the question "honestly." It serves as ground truth and as a cost baseline. Both methods get the same plant models. The simulation sweep is run at the resolution needed to locate onset gain within 2%, and we report wall-clock time for both. Describing functions must match the sweep's onset gain and cycle frequency within roughly 10%, which is comparable to the inverter and actuator agreement reported in the file, at a small fraction of its cost.

The ablation that isolates our key ingredient replaces the clamped-integrator describing function with the static saturation describing function used in the inverter limiter models. If the static version predicts onset and frequency equally well, the memory of the projected update does not matter and our contribution collapses to a reuse of existing tools.

Why not just…?

Why not just simulate every candidate gain? For one plant and one disturbance profile, simulation is cheaper to set up and fully accurate, and it is a real threat for small problems. It does not yield a margin, an explanation of which dynamics drive the cycle, or a bound that transfers when delay or lag changes. Each new configuration needs a new sweep, and long-horizon sweeps of stiff oscillatory loops are what harmonic balance was developed to avoid, as Parallel Preconditioned Hierarchical Harmonic Balance for Analog and RF Circuit Simulation argues for circuits. Our cost-and-accuracy comparison against the sweep settles this. If describing functions are not both accurate and much faster, we concede the point.

Why not just lower the gain or add a filter until oscillations disappear? That is the remedy, but it does not tell you how far to lower the gain. The singular-perturbation bound is the conservative version of this advice, and Bianchi and Dörfler show that it leaves an order of magnitude of usable gain on the table ($\alpha<10.1$ versus $\alpha=100$). Our design-rule experiment measures whether the describing-function bound recovers most of that headroom under lag, while staying oscillation-free.

Why not replace hard projection with a smooth penalty, so there are no corners to cause chatter? Smooth penalties trade the cycle for constraint violation. The file's evidence is that hard parameter bounds are what keep estimates inside the stable model set in Adaptive Economic Model Predictive Control for linear systems with performance guarantees, and in safety-limited inputs such as infusion bounds in Real-time Projected Gradient-based Nonlinear Model Predictive Control with an Application to Anesthesia Control. We include a barrier-penalized optimizer at matched tuning effort as a secondary comparison. We report both oscillation amplitude and peak constraint violation, because a penalty that removes cycles but breaks safety limits is not a fix.

Proposed Experiments

The main testbeds are the two settings from Bianchi and Dörfler: their LTI plant and their constrained nonlinear plant under periodic step disturbances. We first reproduce them as published and confirm that the describing-function analysis predicts no sustained cycle there. A spurious prediction in this setting would falsify the method outright. We then add, separately, a first-order actuator lag, a pure delay of one to five samples, and zero-order-hold sampling. For each, we sweep $\alpha$ from 1 to 200. Metrics are onset-gain error, cycle-frequency error, error in amplitude at the bound, compute time against the simulation sweep, and settling time achieved when $\alpha$ is set by each method's bound.

A transfer testbed uses the 2R2C building thermal model from the adaptive economic MPC paper. The plant has real physical lags and diurnal sinusoidal disturbances, which suit harmonic analysis. We drive it with projected feedback optimization of heating power, with the power bounds as the projection set. We also drive it with the paper's projected LMS estimator, to test the ball and parameter-projection extension. That paper reports qualitative results only ("much smaller temperature oscillations"), so its controller serves as the reference and our quantitative ground truth is simulation.

The decisive checks are four. Onset gain and frequency must match simulation within about 10% across lag and delay settings. The clamped-integrator describing function must clearly beat the static-saturation ablation on these errors. The resulting gain bound must be oscillation-free while exceeding every time-domain bound that remains valid under lag. Finally, we must identify, from the few-harmonic comparison, the plant roll-off below which one-harmonic predictions should not be trusted. Failure on the second check would mean the idea reduces to existing limiter analysis. Failure on the first would mean projection corners violate the low-pass assumption too badly for the method to be useful.