In standard Graph Neural Networks (GNNs), information propagation is inherently local. A single message-passing layer allows a node to aggregate features exclusively from its immediate, one-hop neighbors. As noted by Trang Pham et al. in Column Networks for Collective Classification, the depth of a traditional architecture directly dictates the radius of its neighborhood reach. To capture long-range dependencies spanning $K$ hops, a network must stack $K$ sequential layers.
However, this linear coupling of depth and receptive field creates a fundamental engineering bottleneck. When GNNs grow deep, they suffer from over-smoothing—where node representations converge and become indistinguishable—and over-squashing, where an exponentially growing neighborhood's structural information is compressed into a fixed-size vector. To capture global context without destroying the underlying signal, models must expand their receptive fields without relying on deep stacks of layers.
Decoupling Depth via Matrix Powers
To circumvent this bottleneck, a powerful paradigm has emerged: multi-hop neighborhood aggregation. The core intuition is to expand a layer's receptive field horizontally rather than vertically. Instead of relying on iterative, layer-by-layer propagation, these architectures manipulate the underlying graph structure or its adjacency matrix to allow direct communication across multiple hops within a single step.
As explained by Zhongtian Sun et al. in Adaptive Distance Message Passing From the Multi-Relational Edge View, this approach completely decouples the network's architectural depth from its effective receptive field. Mathematically, this is often achieved by computing higher powers of the adjacency matrix ($A^k$) or utilizing explicit shortest-path distances.
For instance, Heng Chang et al. in Power up! Robust Graph Convolutional Network via Graph Powering replace the traditional graph Laplacian with a "Variable Power Operator" that aggregates from multi-hop neighbors simultaneously. Similarly, Zhehan Zhao et al. introduce a Multi-Distance Message Passing architecture in ENAHPool that leverages powers of the adjacency matrix to let nodes communicate directly across specific shortest-path distances. By bypassing intermediate steps, the network preserves signal integrity and mitigates over-squashing.
Bypassing Intermediate Noise
Why is direct multi-hop communication superior to step-by-step message passing? The answer lies in how information degrades across a graph's topology. In many real-world graphs, immediate neighbors can introduce significant noise. As Yada Zhu demonstrates in When Heterophily Meets Heterogeneity: Challenges and a New Large-Scale Graph Benchmark, standard one-hop attention mechanisms frequently fail in heterophilic environments because immediate neighbors often belong to entirely different classes, providing misleading signals.
When forced to pass messages sequentially through these noisy intermediate nodes, the true long-range signal is diluted. To solve this, Yu Wang et al. propose the Tree Decomposed Graph Neural Network, which unrolls a node's neighborhood into distinct subgraphs for each hop. This structure connects a central node directly to its distant neighbors, preventing intermediate vertices from distorting the deep features.
By skipping the local noise, the model can extract stable patterns from the broader graph structure. This direct routing is also highly effective for capturing logical transitivity in large-scale ontologies, as shown by Julie Loesch et al. in Enhanced GAT: Expanding Receptive Field with Meta Path-Guided RDF Rules for Two-Hop Connectivity.
Attention Diffusion and Global Context
Another prominent strategy for multi-hop aggregation involves diffusing attention scores across the graph topology. Instead of computing attention strictly between adjacent nodes, attention diffusion spreads the interaction weights across an extended radius.
Guangtao Wang et al. introduce this concept with MAGNA in Multi-hop Attention Graph Neural Networks, demonstrating that diffusing attention scores allows shallow architectures to capture global structure while suppressing high-frequency noise. This is operationalized mathematically by Yonghao Liu et al. in Deep Attention Diffusion Graph Neural Networks for Text Classification, where a power series of the attention matrix is used to aggregate long-range semantic dependencies in a single step.
Taking this to the extreme, Siyuan Huang et al. develop SubTree Attention in Tailoring Self-Attention for Graph via Rooted Subtrees, utilizing a kernelized softmax to propagate keys and values along edges. This allows their model to explore neighborhoods up to 100 hops deep without performance degradation. Alternatively, Jiaxing Pi in PushNet: Efficient and Adaptive Neural Message Passing leverages Approximate Personalized PageRank (APPR) to construct node-adaptive receptive fields, allowing a single propagation step to seamlessly integrate context from distant nodes.
Theoretical Power and Structural Signatures
Beyond empirical performance, multi-hop aggregation fundamentally alters the theoretical expressiveness of graph neural networks. Standard 1-hop message-passing GNNs are bounded by the 1-Weisfeiler-Lehman (1-WL) isomorphism test, limiting their ability to distinguish certain graph structures.
In How Powerful are K-hop Message Passing Graph Neural Networks, Jiarui Feng et al. provide a rigorous proof showing that $K$-hop message passing frameworks (such as KP-GNN) significantly boost a model's ability to resolve complex graph structures and capture structural patterns completely invisible to standard 1-hop operations.
Furthermore, Man Wu et al. demonstrate in Learning Graph Neural Networks with Positive and Unlabeled Nodes that generating multiple adjacency matrices representing different path lengths provides a much richer structural signature. This expanded context proves vital in sparse graph environments where local-only methods fail due to a lack of immediate connectivity.
Key Foundational Work
Column Networks for Collective Classification (2017) — contributed 1 concept elements to this direction.
LC-RNN: A Deep Learning Model for Traffic Speed Prediction (2018) — contributed 1 concept elements to this direction.