Abstract
Graph-based Retrieval-Augmented Generation (GraphRAG) is a powerful paradigm for complex reasoning, yet it frequently suffers from information loss during the construction of knowledge graphs (KGs). This "completeness fallacy" results in information sparsity and context loss, where vital semantic nuances are discarded. We present the Triple Context Restoration and Query-driven Feedback (TCR-QF) framework, which treats the KG as a dynamic, evolving structure. By restoring original textual context to triples and iteratively filling knowledge gaps based on user queries, TCR-QF achieves a significant 29.1% improvement in Exact Match over state-of-the-art competitors on multi-hop reasoning tasks.
Introduction
While GraphRAG is the current frontier for helping Large Language Models (LLMs) reason over structured data, most existing methods assume the underlying KG is perfect. In practice, KG construction is inherently lossy. Two primary issues arise:
- Information Sparsity: Extraction algorithms often miss critical entities or relationships.
- Context Loss: Compressing rich prose into discrete
(subject, predicate, object)triples discards vital semantic dependencies.
For example, if a KG records that Einstein "received" a Nobel Prize but fails to capture the specific discovery (the photoelectric effect), the LLM will struggle with nuanced follow-up questions. TCR-QF addresses these limitations by moving away from static retrieval toward a synergistic cycle of knowledge enhancement.
Methodology
The TCR-QF framework consists of two core components designed to restore semantic integrity and ensure graph completeness.
Triple Context Restoration (TCR)
Instead of providing the LLM with isolated triples, TCR "traces back" to the original source text. For every retrieved triple, the system identifies the most relevant sentence from the source documents and attaches it. This ensures that the nuance lost during initial graph construction is available for reasoning.
Query-Driven Feedback (QF)
QF serves as an active learning component. If the initial retrieved subgraph is insufficient to answer a query, the LLM identifies specific "missing knowledge" formulated as sub-questions. The system then performs a targeted search in the original text corpus, extracts new triples, and updates the KG on-the-fly.
Experimental Evaluation
The framework was evaluated on five benchmark multi-hop question-answering datasets: 2WikiMultiHopQA, HotpotQA, MuSiQue-Ans, MuSiQue-Full, and ConcurrentQA. These datasets are specifically designed to test complex, multi-step reasoning across multiple documents.
Results and Discussion
TCR-QF consistently outperformed LLM-only, text-based RAG, and existing GraphRAG methods across all metrics. On the 2WikiMultiHopQA dataset, TCR-QF achieved an Exact Match (EM) score of 0.598, compared to 0.400 for the graph-based ToG method—a relative improvement of nearly 50%.
Key Insights
- The KG as a Living Document: TCR-QF treats the KG as a draft that can be refined during the reasoning process, rather than a static, read-only index.
- Contextual Synergy: Triples provide the structure, while the original text provides the nuance. Combining both allows the LLM to resolve ambiguities that pure graph-based approaches miss.
- Efficient Convergence: Analysis shows that most knowledge gaps are filled within the first 4-6 iterations of the feedback loop, after which performance gains begin to plateau.
Conclusion
TCR-QF effectively mitigates the "broken telephone" problem in GraphRAG by bridging the gap between structured knowledge and textual context. While the iterative nature of the feedback loop increases computational overhead, the gains in accuracy and robustness on complex reasoning tasks are substantial. Future work will focus on improving the efficiency of the search space pruning to reduce latency.
Summary updated