A Survey of Graph Meets Large Language Model: Progress and Future Directions

Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, Jeffrey Xu Yu

Abstract

The integration of Graph Neural Networks (GNNs) and Large Language Models (LLMs) represents a significant shift in data science, bridging the gap between structural intelligence and semantic reasoning. This survey provides a comprehensive taxonomy of the field, categorizing research into three main strategies: LLM as an Enhancer, LLM as a Predictor, and GNN-LLM Alignment. By analyzing various datasets and downstream tasks, we highlight how this synergy overcomes the limitations of "graph-blind" LLMs and "semantically shallow" GNNs, offering a roadmap for future research in text-attributed graphs and beyond.

Introduction

Traditionally, Graph Neural Networks (GNNs) and Large Language Models (LLMs) have been developed in isolation. GNNs are adept at capturing structural dependencies through message passing but often lack deep semantic understanding. Conversely, LLMs possess immense reasoning capabilities but struggle with non-linear graph structures. This survey addresses the urgent need to systematize the merging of these paradigms to create models that understand both the textual context and the structural connections of data.

Overview of the Graph+LLM framework, illustrating the flow from multi-domain graph datasets through integrated models to various downstream tasks.

Methodology: The Three-Pillar Taxonomy

The research landscape is organized into three primary integration strategies based on the role of the LLM:

A comprehensive taxonomy tree of the "Graph Meets Large Language Model" field, detailing specific strategies and representative works.

1. LLM as Enhancer

In this approach, the GNN remains the primary model, while the LLM "upgrades" the data.

  • Explanation-based: LLMs generate natural language descriptions or "pseudo-labels" for nodes to provide richer context.
  • Embedding-based: LLMs act as sophisticated feature extractors, providing high-quality initial vectors for the GNN to process.

2. LLM as Predictor

The LLM serves as the primary decision-maker, often incorporating structural information directly into its reasoning process.

  • Flatten-based: The graph structure is "flattened" into a text string (e.g., "Paper A cites Paper B") and fed directly into the LLM.
  • GNN-based: Structural features from a GNN are injected into the LLM’s input space, typically via a projector layer that translates between modalities.

3. GNN-LLM Alignment

This strategy seeks to synchronize the latent spaces of both models to ensure they share a common mathematical understanding.

  • Symmetrical: Uses contrastive learning to align graph embeddings and text descriptions in a shared space.
  • Asymmetrical: One model acts as a teacher for the other, distilling knowledge (such as structural awareness) across modalities.

Datasets and Downstream Tasks

The survey summarizes benchmarks across several key domains:

  • Citation Networks: Cora, PubMed, and Ogbn-Arxiv for node classification.
  • Molecular Graphs: MoleculeNet and SMILES-based datasets for property prediction.
  • Social & E-commerce: Amazon Co-purchase and social network logs for recommendation and link prediction.
  • Knowledge Graphs: Various datasets used for complex reasoning and link prediction.

Key Insights and Future Directions

Integration strategies consistently outperform standalone models on Text-Attributed Graphs (TAGs). Notable trends include using LLMs as "active annotators" to label nodes and leveraging their zero-shot reasoning for new tasks without retraining.

However, challenges remain, including extending these methods to non-textual graphs (e.g., power grids), mitigating data leakage from LLM pre-training, and improving computational efficiency for large-scale graphs. Future work should focus on "graph-aware" distillation and theoretical analysis of LLM expressive power on graph problems.

Summary updated

BibTeX
@inproceedings{li2024survey,
  author = {Yuhan Li and Zhixun Li and Peisong Wang and Jia Li and Xiangguo Sun and Hong Cheng and Jeffrey Xu Yu},
  title = {{A Survey of Graph Meets Large Language Model: Progress and Future Directions}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=gL1zhtxL84},
}
Figure