Graph-Wise Common Latent Factor Extraction for Unsupervised Graph Representation Learning

Thilini Cooray, Ngai-Man Cheung

Introduction

Unsupervised graph representation learning is a critical challenge in fields like drug discovery and social network analysis, where labeled data is scarce. Current state-of-the-art methods predominantly rely on contrastive learning, which requires carefully curated "negative samples" to learn meaningful embeddings. However, the performance of these methods is often sensitive to batch composition. Graph Autoencoders (GAEs) offer an alternative by learning from a single graph, but they typically over-emphasize local node proximity, failing to capture global, graph-wide structural properties.

This paper introduces GCFX (Graph-wise Common latent Factor EXtraction), a framework designed to explicitly disentangle global "common" factors from local "patch-specific" noise within a single graph.

The deepGCFX Framework

The proposed deepGCFX architecture is a Graph Variational Autoencoder (GVAE) that separates a graph's latent space into a single graph-wise common factor ($z_c$) and multiple local factors ($z_l$).

The deepGCFX architecture overview, illustrating the flow from input graph to disentangled latent factors and dual reconstruction.

The architecture features two primary innovations:

  1. ACCUM (Iterative Query-Based Reasoning): Unlike standard pooling, ACCUM iteratively queries node representations to identify and extract features shared across the entire graph. By masking these shared features from local representations in each step, it effectively isolates the global signal.
  2. Dual Decoding Mechanism: To ensure both factor types are informative, the model employs an Aggregation decoder ($D_{agg}$) that uses both $z_c$ and $z_l$ for reconstruction, and a Regularization decoder ($D_{reg}$) that forces $z_c$ to capture essential structural information by reconstructing the graph from the common factor alone.

Disentanglement via Iterative Reasoning

A key insight of GCFX is that global conditions often govern graph generation. By mimicking this process, the model achieves better discriminative power. The effectiveness of the iterative ACCUM module is empirically supported by correlation analysis.

Spearman correlation analysis over iterations: (a) shows the decreasing correlation between common ($z_c$) and local ($Z_l$) factors, confirming successful disentanglement.

As shown in the figure above, the Spearman correlation between the common factor $z_c$ and local factors $Z_l$ decreases significantly over iterations, demonstrating that the masking process successfully separates global information from local noise.

Experimental Results

The authors evaluated deepGCFX on standard benchmarks for graph and node classification.

Graph Classification

On datasets like MUTAG and RED-BIN, the model (specifically the deepGCFX++ variant) achieved state-of-the-art results, reaching 92.2% accuracy on MUTAG and 90.9% on RED-BIN. Notably, it outperformed popular contrastive methods like GraphCL and InfoGraph without requiring negative sampling.

Node Classification on Disassortative Graphs

Standard GNNs often fail on disassortative graphs (where neighbors have different labels) due to over-smoothing. By extracting a global $z_c$ factor, deepGCFX enables long-range dependencies. On the SQUIRREL dataset, deepGCFX++ improved accuracy to 39.20%, up from the 33.05% achieved by standard GVAE baselines.

Conclusion

GCFX presents a robust alternative to contrastive learning by focusing on internal feature differentiation. By explicitly modeling the separation of global and local factors, it captures high-level graph properties that traditional autoencoders miss. While the iterative reasoning adds some computational complexity and requires hyperparameter tuning for the common-local balance, its success on complex, disassortative networks marks a significant step forward for unsupervised graph learning.

Summary updated

Abstract

Unsupervised graph-level representation learning plays a crucial role in a variety of tasks such as molecular property prediction and community analysis, especially when data annotation is expensive. Currently, most of the best-performing graph embedding methods are based on Infomax principle. The performance of these methods highly depends on the selection of negative samples and hurt the performance, if the samples were not carefully selected. Inter-graph similarity-based methods also suffer if the selected set of graphs for similarity matching is low in quality. To address this, we focus only on utilizing the current input graph for embedding learning. We are motivated by an observation from real-world graph generation processes where the graphs are formed based on one or more global factors which are common to all elements of the graph (e.g., topic of a discussion thread, solubility level of a molecule). We hypothesize extracting these common factors could be highly beneficial. Hence, this work proposes a new principle for unsupervised graph representation learning: Graph-wise Common latent Factor EXtraction (GCFX). We further propose a deep model for GCFX, deepGCFX, based on the idea of reversing the above-mentioned graph generation process which could explicitly extract common latent factors from an input graph and achieve improved results on downstream tasks to the current state-of-the-art. Through extensive experiments and analysis, we demonstrate that, while extracting common latent factors is beneficial for graph-level tasks to alleviate distractions caused by local variations of individual nodes or local neighbourhoods, it also benefits node-level tasks by enabling long-range node dependencies, especially for disassortative graphs.
BibTeX
@inproceedings{cooray2022graph,
  author = {Thilini Cooray and Ngai-Man Cheung},
  title = {{Graph-Wise Common Latent Factor Extraction for Unsupervised Graph Representation Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i6.20593},
}
Figure