Evolving Initial Conditions for Feedback-Alignment Learners on Power-Grid Physics

Brain-inspired learning rules let each layer of a network update its weights using only signals available nearby, without sending exact weight copies backward. A widely studied example is feedback alignment, which sends the output error back through fixed random matrices instead of the transposed forward weights. Recent theory shows that whether these rules work is largely settled before training starts: the starting weights fix how well the forward weights can come to agree with the random feedback. That agreement is the main reason these rules lose to ordinary backpropagation. This note proposes to treat the starting weights as an inherited trait. An evolutionary search, run offline across a family of electrical grids, would tune a small set of numbers describing how weights are drawn at the start of training. Each new grid is then learned with purely local updates from that evolved start.

The test domain is a neural surrogate for alternating-current power flow, meaning a network that maps bus loads and injections to voltages. It is trained with physics-violation signals. Kirchhoff's current-law residuals and voltage-limit violations are computed in output space and broadcast to every layer as extra error channels, so the physics never has to be differentiated through the network. The claim is narrow and checkable. A low-dimensional evolved initialization, chosen for alignment rather than raw loss, should transfer to grids it has never seen. It should close a substantial part of the gap between feedback alignment and backpropagation, and do better than the best hand-designed fixes at the same compute.

Core Claim and Positioning

Three findings motivate the proposal. Robertson et al. derive a conservation law for two-layer networks. It ties weight change to the agreement between forward and feedback matrices, an "alignment potential" fixed at initialization. On a 50-class CIFAR-100 subset, standard feedback alignment collapses to 32.6% while gradient descent reaches 50.6%. Their fixes are hand-written: sign-feedback (48.9%) and adaFA, which sets the forward weights equal to the feedback matrix at the start. Girotti et al. prove convergence for deep linear networks. They also show that random starts produce "anti-regularization," where tiny noisy components are learned before dominant ones, and that structured layer scales repair it. That analysis is linear and leaves finding good starts to future work. Launay et al.'s DFA study shows that direct feedback alignment scales to Transformers and NeRF but still trails, for example 52.0 versus 34.4 perplexity on WikiText-103, and is fragile to optimizer settings.

Evolutionary initialization supplies the method but constrains the design. Work such as Chen et al.'s Lévy-flight tree-seed ELM searches raw weight values against task loss. It lifts load-forecast accuracy from 92.20% to 96.60%, but that works only because extreme learning machines are tiny, and the search ignores alignment geometry. Sun et al. instead evolve the parameters of the distribution weights are drawn from. We reuse that idea because it sidesteps the curse of dimensionality. To our knowledge, no prior work evolves initialization for a local learning rule or scores candidates by alignment.

On the grid side, Dendritic Localized Learning is the closest bio-plausible-meets-grid work. Its forecast on the Electricity dataset reaches mean squared error 0.172 versus 0.175 for backpropagation through time. It treats load as a one-dimensional time series with no network physics. gPINNs embed swing equations but rely on higher-order backpropagation.

Research Program

The searched object is an initialization genome of roughly 20 to 40 numbers. For each layer it holds a weight gain, a decay exponent for the singular-value spectrum, a mixing coefficient $\rho_\ell$, and a sign-agreement fraction $s_\ell$. The mixing coefficient interpolates between an independent random matrix and a projection of that layer's fixed feedback matrix. The sign-agreement fraction sets what share of entries copy the sign of the feedback path.

The genome defines a distribution over starting weights, not the weights themselves. One genome therefore applies to networks of any width and to direct feedback alignment, where feedback matrices map output error to differently shaped hidden layers. That is a case adaFA's simple equality cannot express.

Fitness is measured after a short burst of $K$ local-rule steps (around 200) on a training grid $g$. It combines the drop in physics-informed loss with the mean cosine between each layer's local update and the true gradient. The true gradient is computed only for scoring and is never used for learning:

$$F(\theta) = \mathbb{E}_{g}\big[\,\mathcal{L}_g(0) - \mathcal{L}_g(K)\,\big] + \lambda\,\mathbb{E}_{g,\ell}\big[\cos(\Delta W_\ell^{\text{local}}, \nabla_{W_\ell}\mathcal{L}_g)\big].$$

Here $\theta$ is the genome, $\mathcal{L}_g(k)$ is the loss after $k$ steps on grid $g$, $\Delta W_\ell^{\text{local}}$ is layer $\ell$'s update under the local rule, and $\lambda$ weights the alignment term. Search uses covariance-matrix-style evolution or Lévy-flight sampling over the genome. The total budget is counted in local-training steps.

The reading of the biology is deliberate and addresses the locality concern head on. Evolution is centralized, but it acts across many "lifetimes," meaning many grids. Each deployment learns from a fixed start with local rules only.

The surrogate's output is a vector of predicted bus voltage magnitudes and angles. Given the known admittance matrix, the current-law mismatch at each bus and the voltage-bound hinge penalty are explicit functions of that output. Their output-space gradients join the supervised error and are broadcast through the fixed feedback matrices. This is a modest change to direct feedback alignment, but it is what lets physics constraints enter a local learner without automatic differentiation through the network.

Baselines

The naive reference is direct feedback alignment with standard random initialization, using the same architecture, optimizer, and physics-augmented error. Its failure mode is the one the theory predicts, and every other number is read against it.

The strongest bio-plausible competitors are sign-feedback and adaFA from Robertson et al.. They are the best existing remedies for poor alignment at initialization. We extend adaFA to direct feedback alignment with a least-squares projection of the feedback matrices, so it has a fair version. On their own 50-class CIFAR-100 setting they report 48.9% for sign-feedback against 50.6% for gradient descent. We rerun that setting as a sanity check that our implementation reproduces it.

Backpropagation with He initialization and the same physics loss is the upper bound, not something to beat.

The key ablations are two. The first is random search over the same genome space with the same number of evaluations. The second is evolution against task loss alone, setting $\lambda = 0$, which corresponds to conventional evolutionary initialization.

Every method gets the same tuning budget, measured in local-training steps. The evolutionary cost is charged against the proposal in a separate accounting.

For AC power-flow surrogates trained with feedback alignment, no reported result was found on Lacuna, so all grid numbers here are new. Beating the field means two things on held-out grids. The evolved genome must close at least half of the gap between sign-feedback and backpropagation in voltage error and current-law residual. It must also beat random genome search at equal evaluations.

Why not just…?

Why not just use backpropagation with a physics penalty? If locality is irrelevant, this is the right answer, and we report it as the ceiling. The motivation is hardware: backpropagation's weight transport and sequential locking rule out on-device learning at substations on neuromorphic chips, as argued by the DFA scaling study. The proposal only matters if local learners can approach this ceiling.

Why not just use sign-feedback or adaFA? This is the most serious threat. Sign-feedback already sits within 2 points of gradient descent on the CIFAR-100 subset. If it matches the evolved genome on the grid surrogates and on 6- to 10-layer depth, the genome adds nothing. We will report that outcome as the result rather than reframe it. The genome contains sign-agreement and mixing as special cases, so the experiment decides directly whether anything beyond them helps.

Why not just evolve against the task loss, as prior evolutionary initializers do? The $\lambda = 0$ ablation tests exactly this. With short evaluation bursts, loss-only fitness may reward starts that fit noise fast, the anti-regularization pattern Girotti et al. describe, at the cost of the final result.

Why not just learn the feedback weights, as in difference target propagation or dendritic learning? Ernoult et al. show that learned feedback must approximate the transposed Jacobian to scale. Dendritic Localized Learning doubles weight memory. We include it as a secondary comparison at matched memory. An evolved start for fixed feedback is cheaper at deployment; whether it is enough is the question.

Why not just project outputs onto feasible sets afterward? Streitmatter et al. show ad-hoc after-the-fact limiting causes saturation and violations in inverters. We still report an after-the-fact power-flow correction as a reference for constraint satisfaction.

Proposed Experiments

Training data is generated by solving AC power flow with Newton-Raphson on perturbed load profiles for the IEEE 14-bus, EU 36-bus, IEEE 118-bus, and IEEE 123-bus systems used in the feasible-scenario diffusion work. Topology variants are made by line outages.

Evolution sees only the 14-bus and 36-bus families. Testing is on the held-out 118-bus transmission system and the 123-bus radial feeder, which differ in size and structure. Metrics are voltage-magnitude and angle error, the maximum current-law residual, the rate of voltage-bound violations, and the fraction of predictions that seed Newton-Raphson to convergence. We also track layer-wise alignment cosines over training to check whether anti-regularization appears without the evolved start.

Three checks decide the result. First, transfer: a genome evolved on small grids must beat random search and sign-feedback on the 118- and 123-bus systems. Second, mechanism: the evolved start's advantage must vanish when $\lambda = 0$ or when the alignment-related genome entries are frozen at default values. Third, generality: the same genome-search procedure, rerun on the 50-class CIFAR-100 subset, must at least match the reported 48.9% sign-feedback result.

Failing the transfer check would mean initialization is grid-specific. The proposal would then reduce to per-task tuning and would not justify its cost.