Abstract
Graph neural networks (GNNs) emerge as a powerful family of representation learning models on graphs. To derive node representations, they utilize a global model that recursively aggregates information from the neighboring nodes. However, different nodes reside at different parts of the graph in different local contexts, making their distributions vary across the graph. Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes. To utilize node locality without overfitting, we propose a node-wise localization of GNNs by accounting for both global and local aspects of the graph. Globally, all nodes on the graph depend on an underlying global GNN to encode the general patterns across the graph; locally, each node is localized into a unique model as a function of the global model and its local context. Finally, we conduct extensive experiments on four benchmark graphs, and consistently obtain promising performance surpassing the state-of-the-art GNNs.
Introduction
Standard Graph Neural Networks (GNNs) typically adopt a "one-size-fits-all" approach, applying a single set of global weights to every node in a graph. While effective for capturing general patterns, this approach fails to account for the inherent heterogeneity of graphs. In a citation network, for instance, a node representing a biology paper exists in a vastly different semantic and structural context than one representing computer science. A single global weight matrix tends to prioritize the most frequent patterns, potentially neglecting niche or localized distributions.
To address this, we introduce the Node-wise Localized GNN (LGNN) framework. LGNN dynamically adapts model weights to each node's specific local neighborhood, allowing for personalized representation learning without the parameter explosion or overfitting risks associated with giving every node an independent weight matrix.
Methodology
LGNN treats the localized model as a function of both a shared global model and the node's specific local context. The framework consists of several key components:
Node-level Localization
Instead of learning unique weights for every node, LGNN utilizes a shared "meta-network" (dense layers). This network processes the mean-pooled features of a node's neighborhood to generate two modulation vectors: a scaling vector ($a_v$) and a shifting vector ($b_v$). The localized weight matrix for a node $v$ is then derived by modulating the global weight matrix $W$:
$$W_v = W \odot a_v + b_v$$
Edge-level Localization and Model Agnosticism
For finer granularity, LGNN can also incorporate edge-level modulation, adjusting information flow based on the features of both source and target nodes. Crucially, LGNN is model-agnostic; it acts as a "wrapper" that can be applied to existing architectures like GCN, GAT, and GIN, resulting in their localized counterparts (LGCN, LGAT, LGIN).
Regularization
To ensure the model remains robust against local noise, a regularization term encourages localized weights to remain close to the global model unless the local context provides strong evidence for divergence.
Experimental Results
We evaluated LGNN on four benchmark datasets: Cora and Citeseer (citation networks), Amazon (co-purchasing network), and Chameleon (Wikipedia hyperlink network).
Performance Gains
As shown in the experimental results, the localized versions of standard GNNs consistently outperformed their base counterparts across all datasets. The improvements were particularly notable on the Chameleon dataset, where LGIN achieved 48.3% accuracy compared to GIN's 45.8%.
Key Insights
- Contextual Awareness: By defining context through neighborhood features, the model effectively "senses" its environment to adjust its filters.
- Efficiency: Parameter sharing via the weight-generator network allows for localization with only a marginal increase in total trainable parameters.
- Robustness: Localized models exhibited smaller standard deviations in performance, suggesting increased stability compared to purely global models.
Conclusion and Future Work
LGNN demonstrates that accounting for node locality significantly enhances GNN performance. While the approach introduces some computational overhead during the forward pass due to weight generation, the gains in accuracy and robustness are substantial. Future research will explore extending this localization strategy to heterogeneous graphs and investigating its efficacy in handling distribution shifts in dynamic graph structures.
Summary updated