Learning Fault-Recovery Corrections for Grid-Forming Inverters by Interactive Imitation in Complex-Frequency Coordinates

Grids run mostly on solar and wind connect to the network through inverters, power electronics whose behavior is set entirely by software. In such grids, and especially on resistive distribution lines, frequency and voltage cannot be controlled separately. Pushing active power to correct frequency moves voltages, and pushing reactive power to correct voltage moves phase angles. A recent family of controllers handles this by steering a single "complex frequency," the rate of change of the log voltage phasor, whose real part is the relative rate of voltage change and whose imaginary part is ordinary frequency deviation. These controllers come with stability proofs, but only near normal operation. They stop applying under large faults, wide angle swings, inverter current limits, and mixed fleets that still contain spinning generators.

We propose to keep the analytical complex-frequency controller and add a small learned correction on each inverter. The correction is switched off near normal operation and is learned only for the large-disturbance region where the theory says nothing. It is trained by interactive imitation, in the style of DAgger: the learner's own post-fault trajectories in simulation are repeatedly labeled by a slow, centralized, constraint-aware model-predictive controller (MPC) that serves as the expert. The core bet is about where errors compound. After a fault, a controller that has only seen expert trajectories drifts into states it was never shown. Correcting that drift is exactly what interactive imitation is designed for, and nothing in the current grid-learning literature does it.

Core Claim

A decentralized residual policy, written in complex-frequency coordinates and trained by DAgger against a centralized nonlinear MPC expert, will keep more grids within frequency, voltage and current limits after severe disturbances than three alternatives: the analytical complex-frequency controller alone, the same controller with its gains optimized on the same scenarios, and residuals trained offline by cloning the expert or with expert-side noise injection. We make two further predictions. First, the advantage of the coupled parameterization over separate frequency-only and voltage-only corrections should grow with the resistance-to-reactance ratio of the lines. Second, the advantage of interactive over offline training should grow with disturbance severity. Both are falsifiable, and either could fail.

Where This Sits

The physical motivation comes from Häberle et al., who show that high resistance-to-reactance ratios couple active and reactive power, and from Marinescu et al., who argue that decoupled loops then produce oscillation and poor transient recovery. The method we build on is the complex-frequency control line. Seo et al. validate dispatchable virtual oscillator control on five 1-kVA inverters, with synchronization in about 150 ms. Nonlinear Stability of Complex Droop Control proves almost-global stability under conservative setpoint and connectivity conditions. Complex-Frequency Synchronization gives stability criteria that rely on separating fast and slow time scales and on linearized power flow. Dynamic Complex-Frequency Control replaces static gains with strictly positive-real transfer functions $T(s)$ to shape inertia-like responses on the IEEE 9-bus system.

These papers constrain us as much as they supply the base. Their guarantees are small-signal or rest on linearization and small-angle assumptions, and the stability of mixed converter and synchronous-machine systems is explicitly left open (He et al.). We do not replace their controller. We confine learning to the region they cannot certify.

On the learning side, the closest power-system imitation work is offline. Sondermeijer et al. regress per-inverter reactive power setpoints onto convex optimal-power-flow solutions and land within 0.15% of optimum. They work in quasi-steady state, keep active power fixed, and have no mechanism for recovering from states outside their history. Hassouna et al. clone soft labels for discrete topology switching in Grid2Op. A neighboring group of learned voltage controllers is safe by construction but reactive-power-only and trained by reinforcement learning on linearized models: Yuan et al. use input-convex networks, SAVER uses a quadratic-program projection, and Decentralized Safe RL uses Lipschitz-bounded networks. Xu et al. and real-world DRL voltage control are likewise single-channel and quasi-steady. None of these treats sub-second coupled dynamics, and none uses an expert interactively.

From imitation learning we reuse DAgger, which turns the quadratic compounding of behavioral cloning into linear compounding by labeling learner-visited states. We compare against DART, which injects noise into the expert instead. One caveat should be stated plainly. The usual objections to DAgger are unsafe rollouts and impossible resets, and both largely disappear here because training happens in simulation, where resets are free, as Swamy et al. exploit. The real cost is that every expert label is a nonlinear MPC solve. Query budget is therefore the currency in every comparison.

Research Program

Each inverter $i$ keeps the dynamic complex-frequency law and adds a gated residual:

$$\varpi_i = T_i(s)\big[(p_i^*-p_i) + j(q_i^*-q_i)\big] + g(z_i)\, r_\phi(o_i).$$

Here $\varpi_i = \dot V_i/V_i + j\dot\theta_i$ is the complex frequency the inverter imposes on its terminal voltage, and $p_i, q_i$ are its active and reactive power with setpoints $p_i^*, q_i^*$. The term $r_\phi(o_i)$ is a small complex-valued network, with parameters $\phi$, that reads only local measurements $o_i$: recent complex frequency, power deviations, current magnitude relative to its limit, and DC-link voltage. The gate $g(z_i)\in[0,1]$ is exactly zero while a local stress indicator $z_i$ stays inside a band around nominal operation. The indicator combines normalized angle-rate, voltage deviation and current-limit headroom.

Inside the gate the closed loop is the certified analytical controller, so the local stability results of He et al. still hold there. Outside it we claim no guarantee, only empirical survival. We bound the residual's output magnitude and slew rate so it cannot inject arbitrarily fast changes.

The expert is a centralized nonlinear MPC on a phasor-domain network model that includes inverter current limits and, where present, synchronous-machine swing dynamics. It sees the full state, uses a horizon of a few hundred milliseconds, and penalizes frequency and voltage band violations, current-limit saturation and control effort. Its output is converted into the per-inverter residual that would make the closed loop follow its plan. The expert is far too slow and too communication-hungry to run online, and that is the reason for distilling it.

Training follows DAgger. A disturbance (fault, line trip, load step or generator trip) is sampled. The current student is rolled out in an electromagnetic-transient simulation. States are subsampled along its trajectory where the gate is open, the expert labels them, and the residual is refit on the aggregate.

Because the expert sees everything and the student sees only local signals, some expert actions may not be reproducible from local information. We measure this directly as the irreducible training error of the residual on expert-visited states. It bounds what any decentralized imitator can achieve, and it tells us whether a failure lies in the information structure or in the imitation scheme.

Baselines

The naive reference point is the analytical controller with no residual, in two forms: static complex droop and the dynamic $T(s)$ controller of Dynamic Complex-Frequency Control, tuned as in that paper. It is the right floor because it is what a utility would deploy, and the residual must improve on it to justify any learning. We found no reported scalar result on Lacuna for the IEEE 9-bus experiments; the paper reports only the qualitative absence of frequency and voltage-rate jumps under a 25 MW step. We will therefore reproduce it and report our own numbers.

The strongest competitor is that same controller with its $T(s)$ coefficients and droop rotation optimized by gradient-free search over the identical training disturbance set. It gets a simulator budget equal to all rollouts plus all expert solves used by our method. This asks whether a better-tuned linear controller already captures the gain.

The offline imitation baselines use the same residual architecture, gate and number of expert labels. The first is behavioral cloning on expert trajectories, the Sondermeijer-style recipe moved to the dynamic setting. The second is DART, with noise variance fit as in Lee et al.. DART is the most dangerous imitation competitor. On robot grasping it lifted success from 49% under behavioral cloning to 79%, it was about three times cheaper than DAgger in computation, and it degraded supervisor reward by only 5% during training versus 80% for DAgger on Humanoid.

The reinforcement-learning baseline is soft actor-critic training the same gated residual on a reward mirroring the MPC cost. It gets environment steps matched to our total simulator time.

Two ablations isolate the key ingredients. The first replaces the complex residual with two independent real residuals, one acting only on frequency and driven by active power, the other acting only on voltage rate and driven by reactive power. The second removes the gate. The expert MPC, run online with full state, serves as an upper reference.

Our method beats a baseline if it raises the fraction of held-out severe disturbances survived without band or current-limit violation, with a 95% confidence interval excluding zero across seeds, and does not worsen frequency nadir or settling time on mild disturbances. That last condition is where the gate should make the residual inert.

Why not just…?

Why not just tune the analytical complex-frequency controller better? This is the most serious threat. If optimized $T(s)$ gains match our survival rate, the nonlinear residual buys nothing, and we would report that. Our reason to expect otherwise is structural. Linear passivity-based design cannot express state-dependent actions such as backing off reactive support as a current limit approaches, and the stability analysis it rests on assumes small angles (He et al.). The matched-budget optimized baseline settles the question directly, and splitting results by disturbance severity shows where any gap opens.

Why not just clone the expert offline, perhaps with DART noise? Offline cloning works well in quasi-steady settings (Sondermeijer et al.). Post-fault trajectories are where the student's own errors carry it away from anything the expert visited, which is the compounding-error regime analyzed by Ross et al.. DART is a real threat because it is cheaper and DAgger's safety disadvantage vanishes in simulation. DAgger has also behaved erratically on some continuous tasks, with a normalized return of 0.187 on Hopper in imitation. We therefore compare at equal expert-query budgets and report survival as a function of budget. If DART matches DAgger, we will recommend DART.

Why not just use safe reinforcement learning? The existing safe controllers (Yuan et al., SAVER) certify only reactive-power control on linearized models, so they do not transfer to coupled transients. Plain exploration also tends to fall into blackout states that end learning early (Dwivedi et al.). The soft actor-critic baseline with matched simulator time tests whether an expert is worth its solve cost.

Why not just run the MPC online? It needs full-state communication and solves at sub-cycle rates. We include it as an upper bound and report its solve time against the control period.

Proposed Experiments

We use two testbeds chosen to span coupling strength. The first is the modified IEEE 9-bus system of Dynamic Complex-Frequency Control, with one bus kept as a synchronous machine to probe the mixed-fleet regime left open by He et al.. The second is the IEEE 33-bus feeder with grid-forming inverters at its generator buses, driven by wind and solar profiles from DDRE-33, where high line resistance makes coupling strong. Disturbances include load steps (including the 25 MW case), three-phase faults patterned on the 0.6 s fault of Nonlinear Stability of Complex Droop Control, line trips, and abrupt renewable drops. They are split into training sets and harder held-out sets, with fault durations and depths beyond the training range.

We report six quantities: survival fraction, frequency nadir, peak rate of voltage change, integrated voltage-band violation, time spent at the current limit, and settling time, all against expert queries consumed.

Three checks are decisive. First, on the held-out severe set, DAgger-trained residuals must beat optimized $T(s)$ and DART at equal budget. Second, on a sweep of resistance-to-reactance ratio across the 9-bus lines, the gap between coupled and decoupled residuals should grow monotonically with the ratio. A flat gap would mean the complex-frequency parameterization is cosmetic. Third, on mild disturbances the gated residual must be statistically indistinguishable from the analytical controller. That confirms learning is confined to the uncertified region rather than quietly overriding a working design.

References

  1. Grid-Forming and Spatially Distributed Control Design of Dynamic Virtual Power Plants
  2. Dynamic Virtual Power Plant: A New Concept for Grid Integration of Renewable Energy Sources
  3. Dispatchable Virtual Oscillator Control for Decentralized Inverter-dominated Power Systems: Analysis and Experiments
  4. Nonlinear Stability of Complex Droop Control in Converter-Based Power Systems
  5. Complex-Frequency Synchronization of Converter-Based Power Systems
  6. Dynamic Complex-Frequency Control of Grid-Forming Converters
  7. Regression-based Inverter Control for Decentralized Optimal Power Flow and Voltage Regulation
  8. Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach
  9. Stability Constrained Voltage Control in Distribution Grids With Arbitrary Communication Infrastructure
  10. SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation
  11. Decentralized Safe Reinforcement Learning for Voltage Control
  12. A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control
  13. Implementing Deep Reinforcement Learning-Based Grid Voltage Control in Real-World Power Systems: Challenges and Insights
  14. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
  15. DART: Noise Injection for Robust Imitation Learning
  16. Inverse Reinforcement Learning without Reinforcement Learning
  17. imitation: Clean Imitation Learning Implementations
  18. RL for Mitigating Cascading Failures: Targeted Exploration via Sensitivity Factors
  19. A Large-Scale Dataset of Distributed Renewable Energy Scenarios on the IEEE-33 Bus Network