MS-RAG: Simple and Effective Multi-Semantic Retrieval-Augmented Generation

Xiaozhou You, Yahui Luo, Lihong GuEMNLP 2025

TL;DR

This paper introduces MS-RAG, a novel Retrieval-Augmented Generation (RAG) system that combines knowledge graphs with dense vector embeddings to improve both retrieval accuracy and question-answering performance. By creating a multi-semantic index and employing a "mix recall" algorithm, MS-RAG significantly outperforms existing methods, achieving state-of-the-art results while also offering faster inference speeds.

What Problem Does This Paper Address?

Large Language Models (LLMs) are prone to hallucination, generating factually incorrect or nonsensical information. Retrieval-Augmented Generation (RAG) aims to mitigate this by grounding LLM responses in external knowledge. While graph-based RAG has shown promise by leveraging structured knowledge, existing methods suffer from inefficient indexing, reliance on LLMs for extraction (leading to errors and slow inference), and information loss. This paper addresses these limitations by proposing a more efficient and effective RAG system that integrates the strengths of both knowledge graphs and dense vector search.

Methods

The core of MS-RAG lies in its multi-semantic index and mix recall algorithm.

Indexing Stage: * Multi-Semantic Index: Instead of relying solely on LLMs to build knowledge graphs (which can lead to missing nodes/edges), MS-RAG creates a richer index comprising three levels: * Chunk-level: Stores raw text chunks. * Entity-level: Stores extracted entities. * Relation-level: Stores extracted relationships between entities. All these components are encoded using dense vectors.

  • Entity/Relation Summarization and Disambiguation: To enrich the information and handle variations in entity names, summarization and disambiguation techniques are applied.

Retrieval Stage: * Mix Recall Algorithm: This novel algorithm combines the strengths of dense recall and graph recall. * Dense Recall: Uses vector search to quickly retrieve relevant entities and relations from the multi-semantic index. * Graph Recall: Leverages the knowledge graph structure to find neighboring entities and relations, complementing dense recall and mitigating LLM errors. * Multi-Semantic Rerank Module: A lightweight LLM is used to rerank the retrieved results, further purifying the output and enhancing reasoning capabilities with minimal latency.

Datasets

The authors evaluated MS-RAG on three challenging multi-hop question-answering benchmarks:

  • HotpotQA: A large dataset with questions requiring reasoning over multiple documents.
  • 2WikiMultiHopQA: Provides evidence information and reasoning paths for multi-hop questions.
  • MuSiQue (answerable): A multi-hop QA dataset with questions derived from existing single-hop datasets.

They also conducted experiments on simple QA datasets like NaturalQA, PopQA, and TriviaQA to demonstrate robustness.

Key Insights and Ideas

  • LLM Errors are a Major Bottleneck: The paper highlights that LLMs, while powerful, are prone to errors when constructing knowledge graphs (missing nodes/edges), which significantly degrades the performance of graph-based RAG.
  • Synergy of Dense Vectors and Knowledge Graphs: MS-RAG demonstrates that combining dense vector embeddings (for efficient recall of short entities/relations) with knowledge graphs (for structured semantic associations) leads to superior performance.
  • Chunk-Level Information is Crucial: Including raw text chunks in the index helps alleviate the negative impact of LLM mistakes and enhances semantic representation.
  • Efficient Retrieval is Key for Industry: By replacing LLM-based entity extraction with vector search and optimizing the retrieval pipeline, MS-RAG achieves significantly faster inference speeds, making it more practical for industrial applications.
  • Multi-Semantic Reranking Improves Quality: The reranking module, using a lightweight LLM, effectively refines the retrieved results, leading to better question-answering quality.

Key Results

  • Retrieval Performance: MS-RAG achieves state-of-the-art performance on retrieval tasks, showing improvements of about 10%-30% over existing methods.
  • Question-Answering Performance: MS-RAG significantly outperforms GraphRAG and HippoRAG in question-answering, achieving higher correctness, diversity, and comprehension scores.
  • Inference Speed: MS-RAG offers a substantial speedup, being over 5 times faster than GraphRAG in inference time.
  • Module Effectiveness: Ablation studies confirm the effectiveness of each component, including the multi-semantic index, mix recall algorithm, and rerank module.

Limitations and Future Work

  • Knowledge Graph Construction: While MS-RAG alleviates LLM errors, the authors acknowledge the need to explore more effective ways to build knowledge graphs.
  • Advanced Retrieval Algorithms: Further exploration of more advanced retrieval algorithms within the MS-RAG framework is suggested to maximize performance.

Summary updated

BibTeX
@inproceedings{you2025ms,
  author = {Xiaozhou You and Yahui Luo and Lihong Gu},
  title = {{MS-RAG: Simple and Effective Multi-Semantic Retrieval-Augmented Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure