Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation

Mufei Li, Siqi Miao, Pan Li

Abstract

Knowledge-Graph-based Retrieval-Augmented Generation (KG-RAG) provides a structured alternative to text-based RAG, yet it often struggles to balance retrieval efficiency with reasoning depth. This paper introduces SubgraphRAG, a framework that utilizes a lightweight Multilayer Perceptron (MLP) retriever combined with Directional Distance Encoding (DDE) to extract relevant subgraphs. By delegating complex reasoning to off-the-shelf Large Language Models (LLMs), SubgraphRAG achieves state-of-the-art performance on multi-hop question-answering benchmarks while significantly reducing latency compared to iterative LLM-based methods.

Introduction

Large Language Models (LLMs) are transformative but limited by hallucinations and a lack of access to private or real-time data. Retrieval-Augmented Generation (RAG) mitigates these issues by grounding models in external knowledge. While text-based RAG is prevalent, Knowledge Graphs (KGs) offer a more structured representation of complex relationships. However, KG-based RAG faces a "Goldilocks" problem: simple retrieval methods often miss multi-hop evidence, while complex iterative searches are too slow for real-time applications. SubgraphRAG addresses these challenges through a specialized "division of labor" between a fast retriever and a powerful reasoner.

Methodology

SubgraphRAG employs a three-step pipeline designed for maximum efficiency and reasoning coverage:

  1. Topic Entity Extraction: The system first identifies core entities within the user's query.
  2. Subgraph Extraction: This is the core innovation. Instead of using expensive Graph Neural Networks (GNNs), the framework uses a lightweight MLP retriever. To make the MLP "graph-aware," the authors introduce Directional Distance Encoding (DDE), which encodes structural distances between topic entities and graph triples. This allows the retriever to understand connectivity without the overhead of message-passing.
  3. LLM Reasoning: Retrieved triples are linearized into text and fed into an unfine-tuned LLM (e.g., GPT-4o or Llama 3.1). The model is prompted to provide both an answer and a step-by-step explanation to ensure grounding.
The SubgraphRAG framework overview, illustrating the three-step pipeline from topic entity extraction to LLM-based reasoning.

Experimental Setup

The framework was evaluated on two major multi-hop KG question-answering benchmarks: WebQSP (WebQuestions Semantic Parses) and CWQ (Complex WebQuestions). To specifically measure "truth-grounding" and the model's ability to avoid hallucinations, the authors introduced WebQSP-sub and CWQ-sub, filtered versions where answers are guaranteed to exist within the KG.

Key Results and Insights

SubgraphRAG demonstrates several critical advantages over existing baselines:

  • Superior Retrieval: The DDE-based MLP retriever outperformed GNNs in identifying relevant triples. The authors suggest that GNNs can introduce "semantic noise" during message passing, whereas DDE provides a cleaner structural signal.
  • State-of-the-Art Accuracy: SubgraphRAG + GPT-4o achieved the highest scores to date on WebQSP and CWQ, surpassing models specifically fine-tuned for these tasks.
  • Efficiency Gains: The system is 10x to 100x faster than iterative LLM-based retrieval methods, reducing processing time from hundreds of seconds to just a few seconds per query.
  • Scalability: Larger models like GPT-4o benefit from increased context (more triples), while smaller models like Llama 3.1-8B perform best with a more curated, smaller set of evidence.
Retrieval effectiveness curves showing SubgraphRAG (star line) consistently outperforming baselines in triple and entity recall across different triple counts.

Conclusion

SubgraphRAG proves that a simple, non-iterative approach to KG-RAG can be highly effective. By combining a fast, structure-aware retriever with the reasoning capabilities of modern LLMs, the framework achieves a new balance of speed and accuracy. Future work may explore iterative retrieval for even more complex queries and improving the robustness of the initial entity linking step.

Summary updated

BibTeX
@inproceedings{li2025simple,
  author = {Mufei Li and Siqi Miao and Pan Li},
  title = {{Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=JvkuZZ04O7},
}
Figure