Graph neural networks classify a node by repeatedly averaging its features with its neighbors' features. Too little averaging misses distant evidence. Too much averaging makes every node look the same, a failure called over-smoothing. How much averaging is right depends on the graph. Citation networks, where linked papers usually share a topic, reward heavy averaging. "Heterophilic" graphs, where neighbors usually differ (linked Wikipedia pages, university webpages), are hurt by it. In few-shot node classification, a model sees a new task with one to five labeled nodes per class and must adapt quickly. Here the right amount of averaging is unknown in advance and cannot be tuned on a validation set, because there are not enough labels to hold any out.
We propose to make the amount of smoothing itself the thing a meta-learner adapts per task. The backbone is an equilibrium graph layer. Instead of stacking a fixed number of averaging layers, it solves for the node representations that balance staying close to the input features against agreeing with neighbors. That is equivalent to infinitely many layers held in check by an anchor to the inputs. The per-task adaptation loop fine-tunes the parameters that set this balance, along with the classifier, for many gradient steps. Both nested computations are differentiated implicitly, through their end states rather than their paths. The result is a few-shot graph learner whose memory does not grow with propagation depth or with the number of adaptation steps. It can therefore learn to smooth heavily on a homophilic task and lightly on a heterophilic one from a handful of labels.
Core Claim
The claim is narrow and testable. When few-shot tasks are drawn from graphs that differ in homophily, adapting the smoothing operator per task beats every strong alternative we can name. The alternatives are a fixed meta-learned smoothing level, a smoothing level predicted in one forward pass from the support set, and hard truncation of the receptive field. The gain should grow as the task pool becomes more heterogeneous. Constant memory is not the scientific point but the enabler. Without it, an infinite-depth layer inside a long adaptation loop does not fit on a GPU.
One correction to an appealing but wrong framing is worth making early. It is tempting to say that implicit meta-gradients "bypass backward over-smoothing." Backward Oversmoothing shows that in deep explicit graph networks the backpropagated gradients homogenize exponentially with the number of layers, trapping training at high-loss stationary points. The number of inner adaptation steps is not the number of layers. Implicit differentiation of the adaptation loop, as in iMAML, still needs Hessian-vector products through the graph network, so it inherits whatever gradient collapse the network has.
What does change the picture is making the layer an equilibrium. Consider the equilibrium layer of Implicit Graph Neural Diffusion. Its fixed point is
$$Z^* = \left(I + \tfrac{1}{\mu}\Delta_\phi\right)^{-1}\tilde{X},$$
where $\tilde{X}$ is the transformed input, $\Delta_\phi$ is a Laplacian with learned edge and vertex weights $\phi$, and $\mu > 0$ trades input fidelity against smoothness. The backward pass solves a linear system with the transpose of this same operator. When $\mu$ exceeds the largest eigenvalue $\lambda_{\max}$ of $\Delta_\phi$, which is the condition the DIGNN paper uses to guarantee a unique equilibrium, every frequency component of the gradient is damped by at most a factor $1/(1+\lambda/\mu) \ge 1/2$. A depth-$L$ explicit stack instead damps it by a factor decaying exponentially in $L$. This bounded attenuation is the reason to build on equilibrium layers here. It is a prediction we will check directly, not an assumption.
Research Program
The model has three nested levels. The innermost level is the equilibrium propagation above, solved by fixed-point iteration. The middle level is per-task adaptation. Starting from meta-learned values $\theta$, the task parameters $\phi$ minimize support-set cross-entropy plus a proximal term $\frac{\lambda}{2}\|\phi-\theta\|^2$. The parameters $\phi$ cover the edge-metric weights of $\Delta_\phi$, the trade-off $\mu$, and a linear classifier head. The node feature encoder stays shared and frozen within a task.
We parameterize $\mu = \hat{\lambda}_{\max}(\Delta_\phi)(1 + \mathrm{softplus}(s))$, where $\hat{\lambda}_{\max}$ is estimated by a few power iterations and $s$ is a free scalar. This keeps the uniqueness condition true at every adaptation step, so the inner loop cannot drive the layer into a regime where its equilibrium is ill-defined.
The outer level updates $\theta$ from query-set loss using the implicit meta-gradient of Meta-Learning with Implicit Gradients. That paper's proximal regularization makes the inner problem well-posed and approximates the inverse-Hessian-vector product with conjugate gradient. Its outer gradient is
$$\nabla_\theta \mathcal{L}_{\text{query}} = \left(I + \tfrac{1}{\lambda}\nabla^2_\phi \mathcal{L}_{\text{support}}(\phi^*)\right)^{-1}\nabla_\phi \mathcal{L}_{\text{query}}(\phi^*).$$
Here $\phi^*$ is the adapted parameter, and the inverse is applied by conjugate gradient using only Hessian-vector products. iMAML shows this gives memory independent of adaptation steps, and Memory-Reduced Meta-Learning with Guaranteed Convergence gives convergence guarantees for memory-reduced variants. Neither paper deals with an inner model that is itself a fixed point.
The main technical piece we must build is therefore a constant-memory Hessian-vector product through an equilibrium layer. The support loss depends on $\phi$ only through $Z^*(\phi)$. Its Hessian-vector product requires differentiating the adjoint linear solve a second time. We will implement this as a forward-over-reverse product in which both the forward fixed point and its adjoint are re-solved rather than stored.
Three sources of error now compound: inexact equilibrium solves, an adaptation loop that stops short of a stationary point, and truncated conjugate gradient. We will measure the meta-gradient error against a small-graph ground truth computed by exact linear algebra, as a function of all three tolerances. We will also report the regime in which the procedure is reliable. Non-convexity in the edge-metric parameters is a known risk for implicit gradients. We will measure it by running several inner-loop seeds per task and reporting how much the adapted $\phi^*$ and the resulting meta-gradients disagree.
Following Are we measuring oversmoothing in graph neural networks correctly?, we will diagnose smoothing with effective rank of the node representations rather than Dirichlet energy. That paper found effective rank correlates with test accuracy at 0.91 on average, while Dirichlet energy stays flat as trained networks fail. The DIGNN work argued its absence of over-smoothing mainly through accuracy curves and Dirichlet-energy reasoning, so this is also an audit of whether equilibrium layers truly preserve representation dimension. We will apply the same rank measure to the backpropagated gradients to test the bounded-attenuation prediction against an explicit deep stack.
Baselines
The naive reference has no meta-learning at all. A graph network is trained on the meta-training classes. At test time, its frozen embeddings feed a support-set fine-tuned linear classifier. The Meta-GNN paper reports the corresponding number for SGC on 1-shot Cora: 61.64% accuracy.
The established graph meta-learner is Meta-GNN from Meta-GNN, which wraps first-order MAML around a two-layer SGC or GCN. It reports 65.27% 1-shot accuracy on Cora with two held-out classes. That is the concrete bar on the protocol we will reproduce, and beating it requires a statistically clear margin over 65.27% under the same class split and episode count.
G-META runs MAML with prototypical loss on local subgraphs around each node. It avoids over-smoothing by limiting the receptive field, and it reports gains of up to 16.3% over nine baselines and 3-shot learning on ogbn-arxiv. The file gives no absolute accuracy, so for G-META's absolute accuracy on ogbn-arxiv and Tree-of-Life there is no reported result found on Lacuna. We will rerun it from its released protocol. G-META is the strongest existing method on the multi-graph and large-graph settings. It is also the most direct competitor to our thesis, because it controls smoothing by truncating the graph rather than by adapting the smoothing.
Two constructed baselines isolate our key ingredient. The first is the same equilibrium backbone with a prototypical-network head and no inner loop. The second is the same backbone with implicit adaptation of the classifier head only, with the smoothing parameters frozen at their meta-learned values. If either matches the full model, per-task smoothing adaptation is not doing the work.
A third constructed baseline unrolls our full model with ordinary backpropagation for as many adaptation steps as fit in the same GPU memory. It separates the value of the implicit gradient from the value of the model class.
Every method gets the same encoder width, the same episodes and class splits, the same number of meta-training episodes, and the same random-search budget of 30 configurations over its own hyperparameters. For our method, that budget includes the proximal strength $\lambda$, which the iMAML line identifies as delicate. All methods report mean accuracy with 95% confidence intervals over 1,000 test episodes and five meta-training seeds.
Why not just…?
Why not just use a prototypical network on top of an equilibrium layer, with no adaptation loop? This is the most serious threat. Metric-based few-shot learners are often as good as optimization-based ones, and the equilibrium layer may already handle heterophily well enough: the DIGNN paper reports Chameleon rising from 68.42% to 79.89%. The difference is that a frozen backbone applies one smoothing level to every task. The experiment that settles this is the heterogeneity sweep described below. If the prototypical baseline tracks our model even when episodes mix Cora-like and Texas-like tasks, we will report that per-task smoothing adaptation is unnecessary.
Why not just crop each node's neighborhood, as G-META does? Cropping fixes the receptive field at $h$ hops for every task. Adaptive Recurrent Message Passing shows that useful signal on some graphs lies far beyond a few hops: its recurrent propagation, run up to hundreds of steps, beats standard graph networks in zero-shot transfer. We therefore expect G-META to stay competitive on homophilic tasks and lose on heterophilic ones. A reversed pattern would argue against our thesis.
Why not just predict the smoothing level from the support set in one forward pass? A small network could map support-set statistics, such as the label agreement of labeled neighbors, directly to $\mu$ and the edge weights. That would be cheaper than any adaptation loop. With one to five labels per class, such statistics are very noisy, and gradient-based adaptation uses the loss itself rather than a proxy. Still, this is an honest competitor. We will implement it at matched parameter count and include it in every table.
Why not just run first-order MAML for two steps on a shallow network? First-order MAML is cheap and avoids deep propagation by staying shallow. On Omniglot, iMAML reports 99.14% against 97.9% for first-order MAML, a measurable cost of the approximation. Shallow networks cannot express light smoothing over long range. The memory-matched unrolled baseline and Meta-GNN together will show whether this cheaper route closes the gap.
Proposed Experiments
We will first reproduce the Meta-GNN protocol on Cora and CiteSeer (two held-out classes) and on Reddit (five held-out classes, Micro-F1). On Reddit, Meta-GNN reports moving 1-shot Micro-F1 from about 9.8% to about 14.2%. Next we follow G-META's protocol on ogbn-arxiv (3-shot) and Tree-of-Life.
The decisive new setting is a mixed-homophily episode pool. We meta-train on held-out classes drawn from Cora, PubMed, Chameleon and Squirrel, and meta-test on held-out classes from CiteSeer, Texas and Cornell. To vary heterogeneity, we hold the episode count fixed and gradually raise the share of heterophilic graphs in the training pool. No few-shot results on these heterophilic graphs are reported in the file, so all methods are run fresh.
The main prediction is that the gap between our full model and the frozen-smoothing ablation is near zero on the pure-homophilic pool and grows with the heterophilic share. A further prediction is that the adapted $\mu$ separates by graph type at test time.
Three mechanism checks complete the study. First, we plot peak GPU memory against adaptation steps (5 to 200) and equilibrium iterations (8 to 128), expecting flat curves for our method and linear growth for unrolled baselines. Second, we track effective rank of representations and gradients across adaptation, comparing the equilibrium backbone against explicit 16- and 32-layer stacks; bounded backward attenuation predicts that gradient rank stays roughly constant for the former and collapses for the latter. Third, on graphs small enough for exact solves, we measure meta-gradient cosine error against tolerance, to show where constant memory begins to cost accuracy.
The project fails cleanly if the prototypical or one-pass baselines match the full model across the heterogeneity sweep. That outcome would itself be a useful negative result for the field.
References
- Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?
- Implicit Graph Neural Diffusion Based on Constrained Dirichlet Energy Minimization
- Meta-Learning with Implicit Gradients.
- Memory-Reduced Meta-Learning with Guaranteed Convergence
- Are we measuring oversmoothing in graph neural networks correctly?
- Meta-GNN: On Few-shot Node Classification in Graph Meta-learning
- Graph Meta Learning via Local Subgraphs
- Adaptive Recurrent Message Passing for Test Time Computing on Graphs