Locality Preserving Dense Graph Convolutional Networks with Graph Context-Aware Node Representations

Wenfeng Liu, Maoguo Gong, Zedong Tang, A. Kai Qin

TL;DR

This paper introduces LPD-GCN, a novel Graph Convolutional Network (GCN) architecture designed to improve graph classification by better preserving local node information. It achieves this through an encoder-decoder based local feature reconstruction mechanism and a dense connectivity pattern, leading to superior performance on several benchmark datasets.

What Problem Does This Paper Address?

Graph Convolutional Networks (GCNs) have shown great promise in learning representations from graph-structured data, outperforming traditional methods in many graph classification tasks. However, a significant challenge with existing GCNs is their tendency to lose or "smooth out" crucial local information as the network deepens. This over-smoothing issue is particularly detrimental for graph classification, where understanding the fine-grained local structure of nodes and their immediate neighborhoods is essential for accurate categorization. The paper aims to address this limitation by developing a GCN that explicitly preserves local feature information while still capturing global graph structures.

Methods

The proposed Locality-Preserving Dense GCN (LPD-GCN) introduces several key innovations:

  • Encoder-Decoder for Local Feature Reconstruction: A novel module is incorporated that uses an encoder (multiple GCN layers) to embed node features and a decoder (a simple MLP) to reconstruct the original node features. This process is guided by a "local feature reconstruction loss," acting as a regularizer to ensure that local node information is preserved throughout the network. This is distinct from graph autoencoders that might focus on reconstructing graph structure.
  • Dense Connectivity: Inspired by DenseNets in computer vision, LPD-GCN employs a dense connectivity pattern. This means each convolutional layer and its corresponding readout are connected to all preceding convolutional layers. This allows for flexible integration of information from neighborhoods of varying locality, enhancing the flow of local structural patterns.
  • Graph Context-Aware Node Representations: To further enrich node representations, the output of each convolutional layer is concatenated with the global context from the previous layer's readout. This creates "context-aware" node representations that combine local features with broader graph-level information.
  • Attention-Based Layer-wise Aggregation: A self-attention mechanism is used to aggregate the graph-level representations from different layers. This allows the model to adaptively weigh the importance of features from various depths, leading to a more robust final graph representation.

Datasets

The LPD-GCN model was evaluated on six bioinformatics datasets:

  • MUTAG: Mutagenic aromatic and heteroaromatic nitro compounds.
  • ENZYMES: Protein structures, classified by EC top-level class.
  • PTC: Chemical molecules, classified by carcinogenicity.
  • PROTEINS: Protein structures.
  • NCI1: Chemical compounds screened for anti-cancer activity.
  • D&D: Protein structures, classified by enzyme presence.

These datasets are standard benchmarks for graph classification and vary in size and complexity. The paper notes that social network datasets are generally unsuitable due to the lack of node features.

Key Insights and Ideas

  • Local Information is King for Graph Classification: The paper strongly argues that preserving local node features is crucial for effective graph classification, a point often overlooked by GCNs that prioritize global aggregation.
  • Reconstruction as Regularization: Using a local feature reconstruction loss as an auxiliary task effectively regularizes the GCN, preventing the loss of fine-grained local information and improving generalization.
  • Dense Connections Unlock Multi-Scale Locality: The dense connectivity pattern is not just about efficiency; it's about enabling the model to dynamically leverage information from different neighborhood scales, which is vital for capturing diverse local structures.
  • Context-Aware Representations Bridge Local and Global: By combining local node embeddings with global graph context, the model creates richer representations that are more discriminative for classification.

Key Results

  • Superior Classification Accuracy: LPD-GCN achieved state-of-the-art or highly competitive results on 4 out of 6 benchmark datasets. It significantly outperformed other GCN-based baselines on MUTAG, ENZYMES, PTC, and PROTEINS.
  • Notable Gains: For ENZYMES and PTC, LPD-GCN improved accuracy by at least 4.0% and 7.9% respectively. Across all datasets, it showed gains of at least 2.0% over baselines.
  • Ablation Study Success: Variants of LPD-GCN with removed components (NoLFR, NoDC, NoGCA) showed decreased performance, validating the contribution of each novel module.
  • Faster Convergence: Despite a slightly higher computational cost per epoch due to its complex architecture, LPD-GCN often converged faster than the GIN baseline, indicating improved learning efficiency.
  • Stable Training: Training curves for LPD-GCN showed a smoother and steadier trend compared to GIN, suggesting more stable training dynamics.

Limitations and Future Work

  • Hyperparameter Sensitivity: The performance of LPD-GCN is sensitive to the trade-off parameter ($\lambda$) between the graph classification loss and the local feature reconstruction loss, requiring careful tuning. A dropout ratio of 0.5 was found to be a good general choice.
  • Future Work: The authors plan to extend their model to handle non-attributed graph classification problems. They are also interested in developing a more generic model applicable to other tasks like recommender systems, one-shot/zero-shot learning, and point-cloud classification, aiming to provide insights for training deeper GCNs.

Summary updated

Abstract

Graph convolutional networks (GCNs) have been widely used for representation learning on graph data, which can capture structural patterns on a graph via specifically designed convolution and readout operations. In many graph classification applications, GCN-based approaches have outperformed traditional methods. However, most of the existing GCNs are inefficient to preserve local information of graphs -- a limitation that is especially problematic for graph classification. In this work, we propose a locality-preserving dense GCN with graph context-aware node representations. Specifically, our proposed model incorporates a local node feature reconstruction module to preserve initial node features into node representations, which is realized via a simple but effective encoder-decoder mechanism. To capture local structural patterns in neighbourhoods representing different ranges of locality, dense connectivity is introduced to connect each convolutional layer and its corresponding readout with all previous convolutional layers. To enhance node representativeness, the output of each convolutional layer is concatenated with the output of the previous layer's readout to form a global context-aware node representation. In addition, a self-attention module is introduced to aggregate layer-wise representations to form the final representation. Experiments on benchmark datasets demonstrate the superiority of the proposed model over state-of-the-art methods in terms of classification accuracy.
BibTeX
@article{liu2020locality,
  author = {Wenfeng Liu and Maoguo Gong and Zedong Tang and A. Kai Qin},
  title = {{Locality Preserving Dense Graph Convolutional Networks with Graph Context-Aware Node Representations}},
  journal = {Neural Networks},
  year = {2020},
  doi = {10.1016/j.neunet.2021.05.031},
}
Figure