Deep Multi-modal Graph Clustering via Graph Transformer Network

Qianqian Wang, Haiming Xu, Zihao Zhang, Wei Feng, Quanxue GaoAAAI 2025 · DOI

Abstract

Graph clustering aims to group similar nodes in a network without relying on labels. Traditional methods often struggle with structural blind spots and the effective integration of multi-modal data. We present DMGC-GTN (Deep Multi-modal Graph Clustering via Graph Transformer Network), a novel framework that moves beyond standard neighborhood-aggregation approaches. By integrating graph smoothing with a Transformer architecture that explicitly encodes node intimacy and shortest-path distances, DMGC-GTN captures complex, complementary relationships inherent in multi-modal graph data. Our experimental results across four benchmark datasets demonstrate that DMGC-GTN consistently outperforms existing state-of-the-art methods.

Introduction

Graph clustering is essential for discovering community structures in social networks, citation indices, and biological systems. While deep learning has significantly advanced this field, two major challenges persist. First, standard Graph Neural Networks (GNNs) primarily focus on immediate neighborhoods, often overlooking deeper structural cues like global node intimacy or shortest-path distances. Second, real-world data is frequently multi-modal, requiring models to balance modality-consistent information with modality-specific features. DMGC-GTN addresses these hurdles by treating graph structures as sequences and leveraging the adaptive attention mechanisms of Transformers.

Methodology

The DMGC-GTN framework follows a multi-stage pipeline designed to maximize information extraction from multi-modal graph data.

The DMGC-GTN architecture, illustrating the data processing pipeline and the multi-modal graph transformer module with triple embeddings.

Data Processing and Intimacy Inference

The process begins with Graph Laplacian Smoothing applied to node features to reduce noise and ensure initial local consistency. Simultaneously, the PageRank algorithm is used to calculate an Intimacy Matrix ($S$), which identifies global topological relationships beyond direct edges.

Triple Embedding and Transformer Encoder

The core innovation lies in the Triple Embedding Strategy. For each node, a sequence of its top-$t$ most intimate neighbors is constructed. Three types of embeddings are generated for this sequence:

  1. Feature Embeddings: Smoothed node attributes.
  2. Position Embeddings: Encoding relative closeness based on intimacy scores.
  3. Shortest Path Embeddings: Mapping hop-distances between nodes into the vector space.

These sequences are processed by a Graph Transformer Encoder, which uses self-attention to adaptively weigh the importance of different neighbors and structural features.

Reconstruction and Clustering

A Dual-Task Decoder ensures the latent space's fidelity by reconstructing both the original smoothed features and the graph's intimacy structure. Finally, a specialized clustering layer refines node groupings by minimizing the KL divergence between the predicted and target distributions.

Experimental Setup

DMGC-GTN was evaluated on four benchmark datasets: AMAP (product network), WIKI (co-occurrence network), CITESEER, and CORA (citation networks). For single-modal datasets, additional modalities were generated using Fast Fourier Transform (FFT) to simulate a multi-modal environment. Performance was measured using Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI).

Results and Discussion

DMGC-GTN consistently outperformed state-of-the-art (SOTA) methods such as CMGEC and SGFormer.

Training convergence and performance metrics (Loss, ACC, NMI, ARI) for the Citeseer and Cora datasets.

Performance Benchmarks

On the AMAP dataset, DMGC-GTN achieved an ACC of 0.786, a 2.3% improvement over the previous SOTA. On Cora, the model reached an ACC of 0.763, significantly higher than CO-GCN's 0.735. As shown in the convergence plots, the model demonstrates stable learning and rapid improvement in clustering metrics during the early stages of training.

Ablation Studies

Ablation studies highlighted the critical role of structural embeddings. On the AMAP dataset, using only feature embeddings resulted in an ACC of 0.652. The inclusion of position and shortest-path embeddings increased this to 0.786, confirming that explicit structural context is vital for accurate clustering.

Conclusion

DMGC-GTN demonstrates the power of combining graph-theoretic concepts with Transformer architectures for multi-modal clustering. By treating neighborhoods as sequences and incorporating multi-faceted structural embeddings, the model overcomes the limitations of traditional GNNs. Future work will focus on optimizing the model for massive-scale graphs and exploring its performance on diverse, naturally occurring multi-modal data such as video and sensor streams.

Summary updated

BibTeX
@inproceedings{wang2025deep,
  author = {Qianqian Wang and Haiming Xu and Zihao Zhang and Wei Feng and Quanxue Gao},
  title = {{Deep Multi-modal Graph Clustering via Graph Transformer Network}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i8.32844},
}
Figure