Node-wise Localization of Graph Neural Networks

Zemin Liu, Yuan Fang, Chenghao Liu, Steven C. H. Hoi

Abstract

Graph neural networks (GNNs) emerge as a powerful family of representation learning models on graphs. To derive node representations, they utilize a global model that recursively aggregates information from the neighboring nodes. However, different nodes reside at different parts of the graph in different local contexts, making their distributions vary across the graph. Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes. To utilize node locality without overfitting, we propose a node-wise localization of GNNs by accounting for both global and local aspects of the graph. Globally, all nodes on the graph depend on an underlying global GNN to encode the general patterns across the graph; locally, each node is localized into a unique model as a function of the global model and its local context. Finally, we conduct extensive experiments on four benchmark graphs, and consistently obtain promising performance surpassing the state-of-the-art GNNs.

Introduction

Standard Graph Neural Networks (GNNs), such as GCN or GAT, typically employ a "one-size-fits-all" global weight matrix for every node in a graph. This approach assumes that the same feature transformation should be applied uniformly across the entire graph. However, real-world graphs are rarely homogeneous. For instance, in a citation network, a node representing a "Biology" paper exists in a vastly different local environment than one representing "Computer Science."

A rigid global model can become "blurry" as it attempts to average patterns across diverse communities, potentially failing to capture the nuances of distinct local contexts. While assigning a unique weight matrix to every node would theoretically solve this, it would lead to massive overfitting and a prohibitive increase in parameters. This paper introduces the Node-wise Localized GNN (LGNN) framework, which seeks a middle ground: a model that remains globally grounded while being locally adaptive.

Methodology

LGNN localizes a global GNN model into unique, node-specific models through a mechanism of context-aware weight generation. The framework consists of two primary levels of localization:

  1. Node-level Localization: Instead of a static global weight matrix $W^l$, the model generates a localized weight $W^l_v$ for each node $v$. This is achieved by modulating the global weight using scaling ($a$) and shifting ($b$) vectors.
  2. Context-Aware Generation: To prevent overfitting, these modulation vectors are generated by a secondary hypernetwork. This network takes the local context—the aggregated features of a node's immediate neighborhood—as input, allowing the model to understand the "neighborhood vibe" before processing information.
  3. Edge-level Localization: The model can further refine information flow by adjusting weights for specific edges based on the features of both connected nodes.
Figure 1: Illustration of (a) a toy citation graph with diverse local contexts, (b) a conventional GNN with a global weight matrix, and (c) the proposed node-wise localization where weights are functions of local context.

To ensure stability, a specialized regularization loss is employed to keep localized weights near the global "anchor" model unless local data strongly justifies a divergence. Because LGNN acts as a strategy rather than a standalone architecture, it can be applied as an "agnostic wrapper" to existing models like GCN, GAT, or GIN.

Experimental Results

The researchers evaluated LGNN on four diverse benchmark datasets: Cora, Citeseer, Amazon (co-purchasing), and Chameleon (Wikipedia hyperlinks).

Figure 2: Performance comparison of GCN, GAT, and GIN with and without localization across four benchmark datasets. Localization consistently improves accuracy.

Key findings from the experiments include:

  • Consistent Performance Gains: The localized versions (LGCN, LGAT, LGIN) consistently outperformed their global counterparts. The most significant improvements were observed on the Chameleon dataset, where localization helped capture the complex hyperlink structures more effectively.
  • Strategy over Scale: To confirm that gains weren't merely due to increased parameter counts, LGNN was compared against "wider" versions of standard GNNs. LGNN still demonstrated superior performance, proving that the localization strategy itself is the driver of success.
  • Robustness: LGNN exhibited lower standard deviations in performance across runs, suggesting that the localized approach helps the model find more stable and optimal solutions compared to rigid global models.

Conclusion

LGNN addresses a fundamental limitation of standard GNNs by allowing for node-specific transformations based on local graph topology. By using hypernetworks to generate weights from neighborhood contexts, the model achieves high representational power without the pitfalls of overfitting. While the approach introduces some computational overhead due to the per-node weight calculations, the consistent accuracy gains across multiple benchmarks establish node-wise localization as a powerful tool for graph representation learning. Future work will focus on optimizing these transformations for industrial-scale graphs with millions of edges.

Summary updated

Abstract

Graph neural networks (GNNs) emerge as a powerful family of representation learning models on graphs. To derive node representations, they utilize a global model that recursively aggregates information from the neighboring nodes. However, different nodes reside at different parts of the graph in different local contexts, making their distributions vary across the graph. Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes. To utilize node locality without overfitting, we propose a node-wise localization of GNNs by accounting for both global and local aspects of the graph. Globally, all nodes on the graph depend on an underlying global GNN to encode the general patterns across the graph; locally, each node is localized into a unique model as a function of the global model and its local context. Finally, we conduct extensive experiments on four benchmark graphs, and consistently obtain promising performance surpassing the state-of-the-art GNNs.
BibTeX
@inproceedings{liu2021node,
  author = {Zemin Liu and Yuan Fang and Chenghao Liu and Steven C. H. Hoi},
  title = {{Node-wise Localization of Graph Neural Networks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=MQ1xl5vSU9N},
}
Figure