Graph neural networks pass messages between connected nodes. When a node averages all its neighbors equally, it breaks down in two cases: neighbors that belong to a different class (heterophily), and edges that are spurious or adversarial. Attention-style networks fix this by learning how much to trust each neighbor. Training those trust weights, though, normally requires backpropagation, which is a global algorithm: every weight update depends on error signals carried back from the whole network. Predictive coding is a family of learning rules in which each node tries to predict its own state from its neighbors and learns only from its own local prediction error. It already trains graph networks without backpropagation. All existing versions, however, average neighbors with fixed, content-blind weights.
We propose that the trust weight on an edge should simply be how well that neighbor currently predicts the node. If each incoming edge is treated as a competing explanation of the node's state, minimizing the predictive-coding energy over these mixing weights yields a softmax over negative per-edge prediction errors. In other words, attention falls out of local energy minimization. It needs no gate parameters and no backward pass, and it can be computed at the edge during inference. The receptive field can then be widened cheaply by adding sparse long-range edges, because edges that predict badly are switched off automatically.
Core Claim
Content-adaptive neighbor weighting and spatiotemporally local credit assignment are not in tension. A predictive-coding graph whose energy includes per-edge responsibilities learns anisotropic, heterophily-robust aggregation using only quantities available at each node and its incoming edges. On adversarially perturbed and heterophilic graphs, it should close most of the gap to attention networks trained by backpropagation, and it should beat static-aggregation predictive coding.
Concretely, let node $i$ have value $x_i$. Each neighbor $j \in \mathcal{N}(i)$ sends a prediction $\mu_{ij} = f(W x_j)$, with $W$ a learned projection and $f$ a nonlinearity. The prediction error on that edge is $e_{ij} = x_i - \mu_{ij}$. Let $r_{ij}$ be the responsibility of edge $j$ for node $i$, let $\pi_{ij}$ be a structural prior (uniform, or favoring local over long-range edges), and let $\sigma_i^2$ be a per-node error variance. The energy is
$$E = \sum_i \sum_{j \in \mathcal{N}(i)} r_{ij}\left(\frac{\|e_{ij}\|^2}{2\sigma_i^2} + \log\frac{r_{ij}}{\pi_{ij}}\right).$$
Minimizing $E$ over $r_{ij}$, with each node's responsibilities summing to one, gives
$$r_{ij} = \frac{\pi_{ij}\exp(-\|e_{ij}\|^2/2\sigma_i^2)}{\sum_{k\in\mathcal{N}(i)}\pi_{ik}\exp(-\|e_{ik}\|^2/2\sigma_i^2)}.$$
The weight update is $\Delta W \propto \sum_{ij} r_{ij}\, (e_{ij}/\sigma_i^2)\odot f'(\cdot)\, x_j^\top$. This rule uses only the presynaptic value, the postsynaptic error and the edge's own responsibility. The normalization in $r_{ij}$ sums over node $i$'s incoming edges, which is the same node-local scope as the softmax in Graph Attention Networks. Setting all $r_{ij}=\pi_{ij}$ recovers static-aggregation predictive coding. Expressivity is restored with multiple heads, each its own mixture, concatenated as in graph attention. This matters because a single mixture can only select a neighbor, not sum several.
Position in Prior Work
Two papers supply the learning machinery and motivate the gap. Learning on Arbitrary Graph Topologies via Predictive Coding shows that local energy minimization trains cyclic graphs, reaching 91.76% on MNIST. Robust Graph Representation Learning via Predictive Coding brings predictive coding to message-passing graph networks and finds better robustness and calibration. Both use static or isotropic neighbor aggregation; we make the aggregation content-dependent without leaving the local regime.
Two papers supply the receptive-field and gating ideas, but train them with backpropagation. AdaptViG gates edges on a sparse scaffold of local and logarithmic long-range connections, using an exponential decay of feature similarity. Our gate has the same exponential form, but it is driven by prediction error rather than raw feature similarity, and it is learned locally. Adaptive graph diffusion networks weights whole hop distances adaptively and escapes the low-pass behavior of averaging. We borrow its synthetic spectral-filter test and its multi-hop scaffold idea, but replace global training. Graph Attention Networks is the reference for anisotropic aggregation; its coefficients come from a learned scoring vector trained end to end, whereas ours have no scoring parameters at all.
The continuous-time local learning work constrains our ambitions. Nature-Inspired Local Propagation and Spatiotemporal Local Propagation are demonstrated only on networks of one to five neurons. We therefore stay inside discrete predictive coding, which has scaled further. We treat streaming only through warm-started inference, not Hamiltonian dynamics.
Research Program
We first build the mixture predictive-coding layer and verify on small graphs that alternating inference steps on $x$ and closed-form updates of $r$ monotonically decrease $E$. We then study the main failure mode we expect: early in training all predictions are poor. If $\sigma_i$ starts small, responsibilities can collapse onto whichever edge happens to be best, and the network locks in. We handle this with a local variance update, $\sigma_i^2$ tracking the responsibility-weighted mean squared error, initialized large. Under this scheme responsibilities start near the prior and sharpen as predictions improve, a schedule we will measure rather than assume.
Next we widen the receptive field. On image-derived graphs we add AdaptViG-style logarithmic long-range edges to a local grid, with a prior $\pi_{ij}$ that mildly favors local edges. We test whether responsibilities on long-range edges become selective, rather than either dying out or averaging everything.
Finally, we test a streaming variant. Node values are carried from frame to frame, and only one or two inference steps are run per frame. This addresses the settling cost flagged in Learning on Arbitrary Graph Topologies via Predictive Coding by amortizing it over time. It is a secondary aim and does not carry the main claim.
Baselines
The naive reference points are two networks that average neighbors with fixed weights: a GCN trained by backpropagation, and GPCN from Robust Graph Representation Learning via Predictive Coding. GPCN is the closest local-learning competitor, and on Cora under Nettack at a 25% perturbation rate it reports 54.27% accuracy against GCN's 44.16%. Beating it means exceeding GPCN on the same attacked graphs with matched layer count, hidden width and inference-step budget. Because the file quotes no exact clean-accuracy numbers for GPCN, we will rerun both under identical splits.
The strongest existing method for dynamic weighting is a graph attention network trained by backpropagation (Graph Attention Networks). On image graphs the equivalent is a scaled-down AdaptViG gate on the same scaffold, also trained by backpropagation. No reported Lacuna result was found for GAT under Nettack on Cora or on heterophilic benchmarks, so we will run it ourselves with the same hyperparameter search budget as our method. Here "beating" is not required. Success is recovering most of the gap between GPCN and the backpropagation-trained attention network, at no cost in calibration. On the synthetic 2D grid spectral-filter task, AdaptViG's companion baseline, AGDN with hopwise convolution, reports near-perfect $R^2$ (about 1.0000) across filter types while GCN and GAT fail (Adaptive graph diffusion networks). That number serves as the ceiling.
Three ablations isolate the key ingredient. Setting uniform responsibilities reduces our model to a static predictive-coding graph. Using a parameter-free feature-similarity gate, $\exp(-\|x_i-x_j\|^2)$, inside the same predictive-coding network tests whether prediction error, rather than mere similarity, is what matters. A hybrid learns GAT-style gate parameters by backpropagation while everything else is learned locally; this measures what the fully local gate gives up.
Why not just…?
Why not just plug a parameter-free similarity gate into GPCN? This is the most serious threat, since it needs no new derivation. The difference matters most under heterophily and attack. There, a neighbor can look similar in raw features yet be a poor predictor after projection, or look dissimilar yet be informative through $W$. Similarity gating cannot learn which kind of dissimilarity is useful; prediction-error gating changes as $W$ learns. If the similarity-gate ablation matches our method on the attacked and heterophilic graphs, we will report that the simpler gate suffices and drop the energy-derived responsibilities.
Why not just train a graph attention network with backpropagation and stop caring about locality? For accuracy alone on static graphs, that is the right answer, and we treat it as the ceiling. The case for locality is narrower. First, predictive coding brings robustness and calibration benefits in Robust Graph Representation Learning via Predictive Coding, and we want to know whether those benefits survive the addition of attention. Second, settings with cycles or streaming input, where Learning on Arbitrary Graph Topologies via Predictive Coding argues backpropagation is awkward, need a local rule. If our model is no more robust or better calibrated than backpropagation-trained attention, the locality argument loses most of its practical force, and we will say so.
Why not just widen the receptive field with fixed multi-hop diffusion instead of dynamic weights? Fixed diffusion averages more of the graph, which is exactly the low-pass behavior that fails on heterophilic graphs and on high-pass filters. Adaptive graph diffusion networks reports that GCN and GAT fail on the high-pass and band-rejection grid tasks while adaptive hop weighting succeeds. We run a fixed-diffusion predictive-coding variant on the same task to show whether that failure carries over to local learning.
Why not just use truncated backpropagation through time for the streaming variant? Online learning of long-range dependencies shows truncation fails beyond its window and that crude online approximations reach under 60% on sequential CIFAR. Our streaming test therefore compares against truncated backpropagation at matched memory, rather than assuming it loses.
Proposed Experiments
The first decisive check is the synthetic 2D grid spectral-filter benchmark from Adaptive graph diffusion networks. A static predictive-coding graph should fail on high-pass targets just as GCN does. If responsibilities let the local model learn high-pass and band-rejection filters, measured by $R^2$ against the roughly 1.0000 ceiling, that directly shows local learning escaping the averaging limit.
The second check uses Cora, CiteSeer and PubMed, both clean and under Nettack and Meta-attack poisoning, following Robust Graph Representation Learning via Predictive Coding. We report accuracy alongside expected and maximum calibration error, and we look at whether responsibilities on injected edges fall below those on clean edges. That last measurement is a mechanism check, independent of accuracy. We also test inductive generalization on PPI.
The third check builds graphs from CIFAR10 image patches with a local-plus-logarithmic scaffold. It compares our model against the predictive-coding graphs of Learning on Arbitrary Graph Topologies via Predictive Coding, for which no reported clean CIFAR10 classification number was found on Lacuna, and against a backpropagation-trained AdaptViG-style gate at equal parameter count. We record accuracy and the fraction of long-range edges that remain active.
The secondary streaming test uses Moving-MNIST next-frame prediction with warm-started inference. It compares against truncated backpropagation and, as a speed reference, against the reported 3× training speedup of ConvS5 over ConvLSTM in Convolutional State Space Models for Long-Range Spatiotemporal Modeling; no exact FVD or PSNR values were quoted there.
The proposal succeeds if three results hold together. Our model must clearly beat GPCN on attacked and heterophilic settings. It must also beat the similarity-gate ablation, so that the gain comes from prediction error specifically. Finally, it must learn non-low-pass grid filters that static predictive coding cannot.
References
- Graph Attention Networks
- Learning on Arbitrary Graph Topologies via Predictive Coding
- Robust Graph Representation Learning via Predictive Coding
- AdaptViG: Adaptive Vision GNN with Exponential Decay Gating
- Adaptive graph diffusion networks: compact and expressive GNNs with large receptive fields
- Nature-Inspired Local Propagation
- Spatiotemporal Local Propagation
- Online learning of long-range dependencies
- Convolutional State Space Models for Long-Range Spatiotemporal Modeling