Graph Neural Networks (GNNs) typically operate through message passing, where nodes iteratively aggregate information from their neighbors. While this allows nodes to learn from their context, it creates a fundamental tension: as the network grows deeper to capture long-range dependencies, node representations tend to converge toward a uniform average. This phenomenon, known as over-smoothing, causes nodes to lose their unique identity, rendering the network's output indistinguishable and ineffective.
To combat this, a growing body of research proposes feature anchoring—a mechanism that explicitly re-injects or "anchors" the hidden state of a node to its original input features at every layer.
The Over-smoothing Crisis
In a standard GCN, each layer acts as a low-pass filter. If you apply this filter too many times, you remove all high-frequency information—the specific, local details that distinguish one node from another. By the time a signal reaches the 10th or 20th layer, the representation of a "researcher" node might look identical to a "paper" node simply because they share a neighborhood.
Meiqi Zhu, Xiao Wang, and their colleagues demonstrate in Interpreting and Unifying Graph Neural Networks with An Optimization Framework that most GNNs are implicitly trying to balance two conflicting goals:
- Graph Smoothing: Making connected nodes similar.
- Feature Fitting: Staying true to the node's original attributes.
Over-smoothing happens when the smoothing term dominates. Feature anchoring restores the balance by providing a constant "ground truth" signal that prevents the representation from drifting into a generic neighborhood average.
Mechanisms of Persistence
Researchers have developed several ways to implement this anchor, ranging from architectural tweaks to formal optimization constraints.
Initial Residual Connections
One of the most direct approaches is the Initial Residual Connection, popularized by Ming Chen and others in Simple and Deep Graph Convolutional Networks. Unlike standard residual connections (which link layer $l$ to $l-1$), initial residuals link every layer back to the very first input layer ($H^{(0)}$). This ensures that even at layer 64, the model has a direct "shortcut" to the node's raw identity.
Iterative Re-injection
In the context of graph diffusion, Priyesh Vijayan and Balaraman Ravindran introduced the Higher Order Propagation Framework (HOPF). HOPF uses a Node Information Preserving (NIP) kernel that explicitly re-incorporates original features at every propagation step. Similarly, Jaemin Yoo and U Kang applied this to signed graphs in SIDNET, showing that re-injecting features acts as a structural anchor that allows the network to explore distant neighbors without "forgetting" the starting node.
Optimization and Energy Functions
A more theoretical framing treats GNN layers as steps in an optimization algorithm. In Graph Neural Networks Inspired by Classical Iterative Algorithms, Yongyi Yang and colleagues derive the TWIRLS framework. Here, the anchor is a "proximal term" in an energy function. This term penalizes the model if the hidden representations stray too far from the input features, effectively forcing the model to find a solution that is both smooth on the graph and faithful to the data.
Anchoring in Specialized Contexts
The intuition of anchoring extends beyond general node classification into specific structural problems:
- Rumor Detection: In social media graphs, every reply is a reaction to a specific source post. Tian Bian and Wenbing Huang show in Bi-GCN that concatenating the source node's features to every other node in the graph prevents the "claim" from being diluted by the "noise" of the comments.
- Multi-label Images: In ML-AGCN, Inder Pal Singh and Enjie Ghorbel use feature similarity as a "structural anchor." By reinforcing the original cosine similarities between labels, they ensure that the graph structure remains discriminative even after multiple convolutions.
Why It Works: The Fidelity Term
The success of these diverse methods points to a single intuitive principle: Fidelity. By maintaining a mathematical link to the input, the network can afford to be aggressive with its smoothing. It can look 10 or 20 hops away to find relevant context because it knows it won't lose its way back to the original node's identity. This allows GNNs to finally scale to the depths common in Computer Vision and Natural Language Processing, transforming them from shallow local aggregators into deep, global reasoning engines.