Multi-Hop Aggregation for Graph Neural Networks

This direction is split into other directions: higher-order graph convolution, multi-hop neighborhood aggregation. Its last version stays readable and citable here.

Version 1 of 1252 papers309 conceptsfrozen 2026-10-04

Standard Graph Neural Networks (GNNs) are famously "myopic." In a single layer of message passing, a node only learns about its immediate neighbors. To understand a broader neighborhood, researchers traditionally stack multiple layers—one for each "hop" of distance. However, this deep stacking introduces two fundamental failures: over-smoothing, where node features become indistinguishable as they converge to a global mean, and over-squashing, where information from a large neighborhood is compressed into a fixed-size vector, creating a bottleneck.

Multi-hop aggregation solves this by expanding a node's receptive field within a single layer or a shallow architecture. By explicitly modeling higher-order relationships, these models capture long-range dependencies without the performance degradation associated with deep networks.

The Mechanics of Higher-Order Vision

The core intuition behind multi-hop methods is to replace or augment the standard adjacency matrix $A$ with higher-order representations. If $A$ represents 1-hop connections, then $A^k$ represents the number of paths of length $k$ between nodes.

Several researchers have proposed different ways to leverage this:

Bypassing the Depth Constraint

The primary advantage of multi-hop aggregation is efficiency in reasoning. In standard Transformers or GNNs, tracking a chain of dependencies requires a depth proportional to the chain's length.

Erwan Fagnou addressed this in ChaCAL (Chain and Causal Attention for Efficient Entity Tracking) by treating the attention matrix as a graph adjacency matrix and applying a geometric series expansion. This allows a single layer to resolve multi-hop dependencies of arbitrary length, effectively bypassing the depth constraints of standard architectures.

This "wide but shallow" approach is also vital for long-sequence modeling. Aosong Feng introduced Diffuser (Diffuser: Efficient Transformers with Multi-Hop Attention Diffusion for Long Sequences), which integrates graph diffusion into sparse attention. By allowing tokens to interact with distant neighbors through indirect paths in one layer, Diffuser achieves the expressiveness of full-attention models with significantly lower memory costs.

Capturing Global Coordination

Multi-hop aggregation is particularly effective in domains where global structure is as important as local connectivity:

  1. Skeleton-Based Action Recognition: In human movement, the relationship between a hand and a foot is crucial, even though they aren't directly connected by a bone. Wei Peng used neural architecture search to discover that higher-order joint correlations are essential for modeling global body movement (Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching).
  2. Semantic Parsing: For tasks like AMR-to-text generation, Jie Zhu showed that explicitly encoding shortest paths between all pairs of concepts allows models to capture complex structural relationships that sequence-to-sequence models ignore (Modeling Graph Structure in Transformer for Better AMR-to-Text Generation).
  3. Heterophily: When "opposites attract" in a graph (nodes of different types connect), 1-hop neighbors can be misleading. Zhining Liu demonstrated with hpGAT (hpGAT: High-Order Proximity Informed Graph Attention Network) that integrating multi-hop topological information helps preserve the discriminative power of node representations in these challenging environments.

By treating the graph not just as a set of immediate links, but as a landscape of reachable paths, multi-hop aggregation allows neural networks to see the "forest" without getting lost in the "trees" of deep, over-smoothed layers.

Papers in this version

Showing 60 of 252 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multi_hop_aggregation_for_graph_neural_n_6621a8c3,
  title = {Multi-Hop Aggregation for Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-hop-aggregation-for-graph-neural-networks/rdr_6621a8c392e44cd6b88428f861e14994},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_multi_hop_aggregation_for_graph_neural_n_6621a8c3_v1,
  title = {Multi-Hop Aggregation for Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-hop-aggregation-for-graph-neural-networks/rdr_6621a8c392e44cd6b88428f861e14994/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}