Hierarchical Diffusion Scattering Graph Neural Network

Ke Zhang, Xinyan Pu, Jiaxing Li, Jiasong Wu, Huazhong Shu, Youyong Kong

Graph Neural Networks (GNNs) have become the standard for analyzing data in non-Euclidean spaces, ranging from social networks to molecular structures. However, they face a persistent challenge: as more layers are stacked to capture deeper relationships, node representations tend to converge and become indistinguishable. This phenomenon, known as over-smoothing, limits the depth and expressive power of traditional GNN architectures.

In a study presented at IJCAI-22, researchers from Southeast University introduced the Hierarchical Diffusion Scattering Graph Neural Network (HDS-GNN). By blending traditional GNNs with the multi-scale analytical power of Diffusion Scattering Networks (DSN), the authors provide a framework that maintains distinct and detailed node features even in deep architectures.

The Over-smoothing Challenge

Most standard GNNs, such as GCN or GAT, operate primarily as low-pass filters. They learn by aggregating information from immediate neighborhoods, which naturally smooths signals across the graph. While effective for identifying general patterns, this approach has two major drawbacks:

  1. Feature Homogenization: In deep networks, every node eventually begins to resemble its neighbors, making classification tasks nearly impossible.
  2. Fixed Receptive Fields: GNNs often struggle to capture multi-scale information—intrinsic properties existing at different neighborhood distances—because they typically aggregate representations in a single-scale, layer-by-layer fashion.

Proposed Method: HDS-GNN

The core innovation of HDS-GNN is the construction of a hierarchical bridge between two distinct network types. The architecture consists of three primary modules: the Graph Diffusion Scattering Network, Inter-scale Weighting, and Hierarchical Enhancement.

The HDS-GNN architecture, showing the hierarchical bridge between the Graph Diffusion Scattering Network and the GNN backbone.
  • Diffusion Scattering Network (DSN): Unlike standard GNNs, DSNs use diffusion wavelets to perform multi-scale analysis, iteratively decomposing graph signals into different frequency bands. As the network goes deeper, it extracts increasingly finer signals.
  • Hierarchical Fusion: HDS-GNN exploits a natural symmetry: as GNN layers become smoother (lower frequency), DSN layers become sharper (extracting finer details). The model concatenates these scattering features into the GNN layers on a layer-by-layer basis.
  • Inter-scale Weighting: To handle noise and prioritize relevant neighborhood scales (e.g., 1-hop vs. 4-hop), the model utilizes a learnable weighting mechanism ($\Lambda$) that adaptively tunes the importance of different scales.
  • Node-Level Interception: While scattering networks are traditionally used for graph-level tasks, this paper "intercepts" the low-pass filtering process within the DSN to extract rich, node-level features suitable for classification.

The Frequency Complement Insight

A key insight of this work is the complementary nature of the two architectures. Standard GNNs provide global context (low frequency), while DSNs provide local texture (high frequency). Fusing them is analogous to restoring high-definition edges to a blurred photograph.

Comparison of frequency responses across layers. While GCNs act as low-pass filters that smooth signals, HDS-GNN maintains finer band-pass details in deeper layers.

The researchers demonstrated that fusing features hierarchically is superior to global fusion (simply adding features at the start or end). This suggests that the evolution of features through the network layers is critical to maintaining representation diversity.

Experimental Evaluation

The model was benchmarked on nine real-world networks, including citation networks (Cora, Citeseer), co-purchase networks (Amazon Computers), and heterophilic graphs (WebKB Texas, Cornell). HDS-GNN consistently outperformed standard backbones and other scattering-based models.

  • Performance Gains: On the Texas dataset, HDS-GAT achieved 73.6% accuracy, a 15.8% absolute improvement over standard GAT. Similarly, it outperformed GCN by 12.3% on the Cornell dataset.
  • Ablation Studies: Removing the Inter-scale Weighting or the DSN features led to significant performance drops, confirming that both multi-scale signals and adaptive weighting are essential components of the architecture.

Conclusion

HDS-GNN offers a robust solution to the over-smoothing problem by integrating multi-scale band-pass signals into the GNN hierarchy. While the current implementation uses simple concatenation for fusion, future work may explore more complex non-linear fusion methods and focus on improving the computational efficiency of calculating diffusion wavelets for larger graphs.

Summary updated

BibTeX
@inproceedings{zhang2022hierarchical,
  author = {Ke Zhang and Xinyan Pu and Jiaxing Li and Jiasong Wu and Huazhong Shu and Youyong Kong},
  title = {{Hierarchical Diffusion Scattering Graph Neural Network}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/519},
}
Figure