Standard Graph Neural Networks (GNNs) suffer from a fundamental limitation as they grow deeper: over-smoothing. When layers repeatedly aggregate neighborhood information, the representations of all nodes converge toward a uniform average, erasing the unique local identity of individual nodes.
To build deep architectures capable of capturing long-range dependencies, we cannot rely on pure neighborhood averaging. Instead, we need a mechanism that allows information to flow globally without washing out local distinctiveness. Input feature anchoring solves this by maintaining a persistent connection to a node's original input features throughout the propagation process, acting as a gravitational pull that preserves individual node identity.
The Intuition: Balancing Diffusion and Fidelity
Graph message passing can be mathematically and intuitively understood as a physical diffusion process, like heat spreading across a surface. Without any constraints, heat eventually distributes evenly, leading to a uniform temperature where all node-specific details are lost.
Input feature anchoring introduces a "source term" or a "fidelity constraint" to this physical system. Instead of letting the node's state drift completely based on its neighbors, the initial features act as a structural anchor.
We can categorize how researchers implement this anchoring mechanism into three main paradigms: explicit feature re-injection, optimization-driven energy penalties, and identity-preserving structural modifications.
Explicit Feature Re-injection and Residuals
The most direct way to anchor a representation is to explicitly re-introduce the initial features at every layer or iteration of the network. This ensures that no matter how deep the network becomes, the original signal is never completely diluted.
Standard GNN Layer: [Input] -> [Layer 1] -> [Layer 2] -> [Layer 3] -> Uniform Average (Oversmoothed)
Anchored GNN Layer: [Input] ------+------------+------------+----> Distinct Representation
v v v
[Layer 1] -> [Layer 2] -> [Layer 3]
In Signed random walk diffusion for effective representation learning in signed graphs, Jaemin Yoo et al. demonstrate that re-injecting a portion of a node's initial features at every step of a random walk diffusion process prevents representations from collapsing into a generic average. This structural anchor allows the network to scale deeper while maintaining node discriminability.
Similarly, Rui Xue et al. utilize a persistent residual connection to raw input features in LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagation. This design choice allows the model to aggregate global context across large graphs while avoiding the over-smoothing effect.
In the extreme case of infinite-depth networks, Juncheng Liu et al. incorporate a persistent skip-connection from the input features directly into the equilibrium equation of EIGNN: Efficient Infinite-Depth Graph Neural Networks. Because the local feature information is continuously re-injected into the global representation, the model remains stable and robust across infinite propagation steps.
Optimization Frameworks and Energy Penalties
Why does this anchoring work so effectively? We can understand its mathematical foundation by viewing GNN propagation layers not just as heuristic stacks, but as steps solving an optimization problem.
In Interpreting and Unifying Graph Neural Networks with An Optimization Framework, Meiqi Zhu et al. show that many popular GNNs are actually minimizing a joint energy function that balances two competing forces:
- Graph Smoothing: Making neighboring nodes similar to each other.
- Feature Fitting: Keeping the current representation close to the original input features.
By explicitly adding an objective term that penalizes large deviations from the original input features, the network is mathematically prevented from over-smoothing.
Building on this optimization perspective, Yongyi Yang et al. introduce TWIRLS in Graph Neural Networks Inspired by Classical Iterative Algorithms. By anchoring the GNN layers to a principled energy function via a permanent connection to the initial input features, they create an architecture that maintains stable performance across hundreds of layers.
Similarly, Hongjoon Ahn et al. derive the HALO architecture in Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph Neural Networks by integrating an anchoring term into a relation-aware energy function, allowing the model to preserve node identities even when dealing with highly complex, heterophilous relationships.
This optimization perspective naturally extends to continuous-time systems governed by Ordinary Differential Equations (ODEs). In Continuous Graph Neural Networks, Louis-Pascal A. C. Xhonneux et al. redefine graph propagation as a continuous dynamic where the derivative of the node state includes a constant "restart distribution" term representing the initial feature encoding.
From a physical diffusion perspective, Xiao Wang et al. propose HiD-Net in A Generalized Neural Diffusion Framework on Graphs, where a mathematical fidelity term anchors the evolving node state to its initial features, preventing the system from reaching a completely uniform, non-discriminative steady state.
Identity-Preserving Structural Modifications
Rather than adding an explicit residual term or an external penalty, some architectures preserve identity structurally by reinforcing a node's connection to itself during the message-passing step.
Standard Aggregation: Node = Average(Neighbor A, Neighbor B, Neighbor C)
Identity-Preserved: Node = Average(Neighbor A, Neighbor B, Neighbor C) + Self-Transformation(Node)
Edward J. Smith et al. implement identity-preserving graph convolutions in GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects by incorporating the identity matrix directly into the adjacency operation. This balances local vertex information with neighborhood-averaged data, which is crucial for maintaining fine geometric details in 3D meshes.
In BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks, Hejie Cui et al. show that explicitly reinforcing a node's own features via concatenation with neighbor aggregates helps maintain distinct node identities in neuroimaging tasks.
For highly connected nodes that risk being overwhelmed by their neighborhood, Xinghao Qiao et al. introduce inferred virtual self-loops in EEGNN: Edge Enhanced Graph Neural Network with a Bayesian Nonparametric Graph Model, protecting individual features from being washed out during extensive message passing.
Furthermore, Andrea Cavallo et al. design a dedicated transformation path for the center node in GCNH: A Simple Method For Representation Learning On Heterophilous Graphs, ensuring the center node's features are not entirely lost to neighborhood averaging when neighbors are highly dissimilar.
Broadening the Scope: Memory Nodes and Semantic Anchors
The principle of anchoring manifests across diverse modalities beyond standard vector-based node embeddings.
- Image Fusion: In Learning a Graph Neural Network with Cross Modality Interaction for Image Fusion, Jiawei Li 0016 et al. introduce persistent memory nodes to prevent information washout during cross-modality message passing, which is essential for maintaining color consistency and fine structural details.
- Knowledge Graphs: In entity alignment, Zequn Sun et al. propose a self-propagation trick in What Makes Entities Similar? A Similarity Flooding Perspective for Multi-sourced Knowledge Graph Embeddings that allows a node's unique features to propagate back to itself, preventing entity-specific details from being lost during graph convolutions.
- Large Language Models: Remarkably, this concept even extends to text-based graph processing. Sunwoo Kim et al. develop the Graph Lingual Network in 'Hello, World!': Making GNNs Talk with LLMs, where raw text attributes are included in the prompt at every single layer. This serves as a semantic anchor that preserves the node's core identity while an LLM integrates information from its neighborhood.
Conclusion
Whether implemented as a continuous restart distribution, an energy penalty term, a virtual self-loop, or a text prompt, input feature anchoring changes the fundamental behavior of graph propagation. By balancing the smoothing effect of neighborhood aggregation with a persistent pull toward the original input, anchoring allows deep architectures to capture global graph topology without sacrificing the local identities that make the data meaningful.
Key Foundational Work
GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects (2019) — contributed 1 concept elements to this direction.
Continuous Graph Neural Networks (2020) — contributed 1 concept elements to this direction.