Input Feature Anchoring in Graph Neural Networks

20 concepts15 papers1 proposal

Standard Graph Neural Networks (GNNs) suffer from a fundamental limitation as they grow deeper: over-smoothing. When layers repeatedly aggregate neighborhood information, the representations of all nodes converge toward a uniform average, erasing the unique local identity of individual nodes.

To build deep architectures capable of capturing long-range dependencies, we cannot rely on pure neighborhood averaging. Instead, we need a mechanism that allows information to flow globally without washing out local distinctiveness. Input feature anchoring solves this by maintaining a persistent connection to a node's original input features throughout the propagation process, acting as a gravitational pull that preserves individual node identity.


The Intuition: Balancing Diffusion and Fidelity

Graph message passing can be mathematically and intuitively understood as a physical diffusion process, like heat spreading across a surface. Without any constraints, heat eventually distributes evenly, leading to a uniform temperature where all node-specific details are lost.

Input feature anchoring introduces a "source term" or a "fidelity constraint" to this physical system. Instead of letting the node's state drift completely based on its neighbors, the initial features act as a structural anchor.

We can categorize how researchers implement this anchoring mechanism into three main paradigms: explicit feature re-injection, optimization-driven energy penalties, and identity-preserving structural modifications.


Explicit Feature Re-injection and Residuals

The most direct way to anchor a representation is to explicitly re-introduce the initial features at every layer or iteration of the network. This ensures that no matter how deep the network becomes, the original signal is never completely diluted.

Standard GNN Layer:  [Input] -> [Layer 1] -> [Layer 2] -> [Layer 3] -> Uniform Average (Oversmoothed)

Anchored GNN Layer:  [Input] ------+------------+------------+----> Distinct Representation
                                   v            v            v
                                [Layer 1] -> [Layer 2] -> [Layer 3]

In Signed random walk diffusion for effective representation learning in signed graphs, Jaemin Yoo et al. demonstrate that re-injecting a portion of a node's initial features at every step of a random walk diffusion process prevents representations from collapsing into a generic average. This structural anchor allows the network to scale deeper while maintaining node discriminability.

Similarly, Rui Xue et al. utilize a persistent residual connection to raw input features in LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagation. This design choice allows the model to aggregate global context across large graphs while avoiding the over-smoothing effect.

In the extreme case of infinite-depth networks, Juncheng Liu et al. incorporate a persistent skip-connection from the input features directly into the equilibrium equation of EIGNN: Efficient Infinite-Depth Graph Neural Networks. Because the local feature information is continuously re-injected into the global representation, the model remains stable and robust across infinite propagation steps.


Optimization Frameworks and Energy Penalties

Why does this anchoring work so effectively? We can understand its mathematical foundation by viewing GNN propagation layers not just as heuristic stacks, but as steps solving an optimization problem.

In Interpreting and Unifying Graph Neural Networks with An Optimization Framework, Meiqi Zhu et al. show that many popular GNNs are actually minimizing a joint energy function that balances two competing forces:

  1. Graph Smoothing: Making neighboring nodes similar to each other.
  2. Feature Fitting: Keeping the current representation close to the original input features.

By explicitly adding an objective term that penalizes large deviations from the original input features, the network is mathematically prevented from over-smoothing.

Building on this optimization perspective, Yongyi Yang et al. introduce TWIRLS in Graph Neural Networks Inspired by Classical Iterative Algorithms. By anchoring the GNN layers to a principled energy function via a permanent connection to the initial input features, they create an architecture that maintains stable performance across hundreds of layers.

Similarly, Hongjoon Ahn et al. derive the HALO architecture in Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph Neural Networks by integrating an anchoring term into a relation-aware energy function, allowing the model to preserve node identities even when dealing with highly complex, heterophilous relationships.

This optimization perspective naturally extends to continuous-time systems governed by Ordinary Differential Equations (ODEs). In Continuous Graph Neural Networks, Louis-Pascal A. C. Xhonneux et al. redefine graph propagation as a continuous dynamic where the derivative of the node state includes a constant "restart distribution" term representing the initial feature encoding.

From a physical diffusion perspective, Xiao Wang et al. propose HiD-Net in A Generalized Neural Diffusion Framework on Graphs, where a mathematical fidelity term anchors the evolving node state to its initial features, preventing the system from reaching a completely uniform, non-discriminative steady state.


Identity-Preserving Structural Modifications

Rather than adding an explicit residual term or an external penalty, some architectures preserve identity structurally by reinforcing a node's connection to itself during the message-passing step.

Standard Aggregation:     Node = Average(Neighbor A, Neighbor B, Neighbor C)
Identity-Preserved:       Node = Average(Neighbor A, Neighbor B, Neighbor C) + Self-Transformation(Node)

Edward J. Smith et al. implement identity-preserving graph convolutions in GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects by incorporating the identity matrix directly into the adjacency operation. This balances local vertex information with neighborhood-averaged data, which is crucial for maintaining fine geometric details in 3D meshes.

In BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks, Hejie Cui et al. show that explicitly reinforcing a node's own features via concatenation with neighbor aggregates helps maintain distinct node identities in neuroimaging tasks.

For highly connected nodes that risk being overwhelmed by their neighborhood, Xinghao Qiao et al. introduce inferred virtual self-loops in EEGNN: Edge Enhanced Graph Neural Network with a Bayesian Nonparametric Graph Model, protecting individual features from being washed out during extensive message passing.

Furthermore, Andrea Cavallo et al. design a dedicated transformation path for the center node in GCNH: A Simple Method For Representation Learning On Heterophilous Graphs, ensuring the center node's features are not entirely lost to neighborhood averaging when neighbors are highly dissimilar.


Broadening the Scope: Memory Nodes and Semantic Anchors

The principle of anchoring manifests across diverse modalities beyond standard vector-based node embeddings.

Conclusion

Whether implemented as a continuous restart distribution, an energy penalty term, a virtual self-loop, or a text prompt, input feature anchoring changes the fundamental behavior of graph propagation. By balancing the smoothing effect of neighborhood aggregation with a persistent pull toward the original input, anchoring allows deep architectures to capture global graph topology without sacrificing the local identities that make the data meaningful.

Key Foundational Work

GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects (2019) — contributed 1 concept elements to this direction.
Continuous Graph Neural Networks (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_input_feature_anchoring_68445a69,
  title = {input feature anchoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/input-feature-anchoring/rdr_68445a69ffcd4e5d8ca0857fc89ebc11},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_input_feature_anchoring_68445a69_v1,
  title = {input feature anchoring},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/input-feature-anchoring/rdr_68445a69ffcd4e5d8ca0857fc89ebc11/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake