KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrieval

Chi Minh Bui, Ngoc Mai Thieu, Vinh Van Nguyen, Jason J. Jung, Khac-Hoai Nam BuiEMNLP 2025

TL;DR

This paper introduces KG-CQR, a novel framework that enhances retrieval in Large Language Model (LLM) based systems by enriching queries with contextual information extracted from knowledge graphs (KGs). KG-CQR significantly improves retrieval accuracy and reduces reasoning steps in complex question-answering tasks by leveraging structured relation representations and subgraph completion.

What Problem Does This Paper Address?

Large Language Models (LLMs) excel at generating human-like text but struggle with factual accuracy and reliable knowledge retrieval. While Retrieval-Augmented Generation (RAG) systems aim to address this by incorporating external knowledge, existing methods often face challenges in effectively aligning query and document embeddings, especially in domain-specific contexts. Current approaches like query decomposition or hypothetical document generation have limitations, such as insufficient contextual alignment or reliance on LLM-generated content prone to hallucination. The paper specifically targets the problem of enhancing the retrieval phase in RAG systems by better contextualizing complex queries using knowledge graphs.

Methods

KG-CQR is a three-module framework:

  1. Subgraph Extraction: This module identifies relevant triples from a corpus-centric KG based on an input query. Unlike traditional methods that rely solely on entity linking or keyword similarity, KG-CQR uses textual triplet representations (TTRs) generated by LLMs to measure semantic similarity at a sentence level, enabling more nuanced subgraph extraction.
  2. Subgraph Completion: To enrich the initial subgraph, this module incorporates additional semantically meaningful paths from the KG. It uses a heuristic-guided Beam Search algorithm to efficiently explore and select relevant triplets that expand the contextual information.
  3. Contextual Generation: This module leverages the enriched subgraph to generate a contextually rich query representation. This enhanced query is then encoded in the same embedding space as documents, facilitating better similarity matching.

The framework also employs a Retrieval Fusion Function that combines the original query embedding with the KG-enhanced query embedding using a weighted-sum approach, further improving retrieval outcomes. KG-CQR is designed as a model-agnostic pipeline, allowing it to work with various LLMs without additional training.

Datasets

The paper evaluates KG-CQR on two benchmark datasets:

  • RAGBench: This dataset spans five industry-specific domains and is used for retrieval evaluation.
  • MultiHop-RAG: This dataset includes a knowledge base, multi-hop queries, and supporting evidence, designed for evaluating complex reasoning tasks.

A corpus-centric KG was constructed for each dataset using the LLaMA-3.3-70B model.

Key Insights and Ideas

  • Structured Relations are Key: The core innovation is the use of "structured relation representations" (TTRs) that combine textual descriptions with KG triplets. This allows for a richer understanding of relationships than just entity-based matching.
  • Query-Centric Contextualization: Unlike corpus-centric approaches, KG-CQR focuses on enriching the query itself with KG information, leading to better alignment with documents.
  • Subgraph Completion Enhances Retrieval: Simply extracting relevant triples isn't enough; completing the subgraph by finding related paths significantly boosts performance.
  • Model Agnosticism is Practical: The framework's ability to work with different LLMs without retraining makes it highly adaptable and scalable.
  • HyDE's Limitations: The paper highlights that methods like HyDE, which rely heavily on LLM-generated hypothetical documents, can be sensitive to LLM inaccuracies and may not always provide reliable context.

Key Results

  • Significant Performance Gains: KG-CQR consistently outperforms strong baselines across both RAGBench and MultiHop-RAG datasets. Improvements of up to 4-6% in mAP and 2-3% in Recall@25 were observed.
  • Effectiveness Across Backbones: KG-CQR enhances retrieval performance regardless of the underlying LLM backbone (e.g., BM25, DPR, BGE), demonstrating its generalizability.
  • Reduced Reasoning Steps: In multi-step reasoning tasks, KG-CQR significantly reduces the number of reasoning iterations required, indicating more efficient and accurate problem-solving.
  • Scalability: The proposed KG-CQR with heuristic-guided Beam Search strikes a balance between efficiency and semantic expressiveness, offering a favorable trade-off compared to naive BFS or HyDE approaches.
  • Complementarity: Integrating KG-CQR with corpus-centric methods like HippoRAG2 further improved performance, suggesting that combining query-centric and corpus-centric KG approaches can create more comprehensive retrieval frameworks.

Limitations and Future Work

  • KG Construction Challenges: The reliance on external LLMs for KG construction can introduce errors in entity and relation extraction, potentially impacting subgraph quality, especially in domains with sparse or noisy data.
  • Scalability of Subgraph Extraction: While effective, the subgraph extraction process can be computationally intensive for very large KGs, potentially limiting real-time applications or resource-constrained environments.
  • Limited Evaluation Scope: The current evaluation is based on a few benchmark datasets. Future work should include evaluations on a wider range of datasets, including those with cross-lingual settings or highly domain-specific knowledge, to assess generalizability more rigorously.
  • Contextual Drift and Irrelevant Retrievals: KG-CQR can struggle with queries requiring fine-grained temporal reasoning, comparative analysis, or interpretation of subjective content due to limitations in temporal representation and deep semantic modeling.
  • Limited Multi-hop Coherence: In some complex multi-hop queries, KG-CQR might retrieve disconnected evidence, hindering the formation of a complete answer path.

Summary updated

BibTeX
@inproceedings{bui2025kg,
  author = {Chi Minh Bui and Ngoc Mai Thieu and Vinh Van Nguyen and Jason J. Jung and Khac-Hoai Nam Bui},
  title = {{KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrieval}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure