Introduction
Large Language Models (LLMs) have revolutionized natural language processing but continue to struggle with factual hallucinations and a lack of domain-specific, up-to-date knowledge. Retrieval-Augmented Generation (RAG) addresses these issues by grounding model outputs in external data. Knowledge Graphs (KGs) are particularly effective for RAG due to their structured nature and logical relationships.
However, most existing KG-RAG systems operate under a "closed-world" assumption, where the starting "anchor entity" is either provided or easily identified. In real-world scenarios, user queries are often noisy, containing abbreviations, aliases, or typos. This "open-world" setting makes entity linking unreliable. If a system incorrectly identifies the starting node, the subsequent retrieval process is doomed to fail.
The AnchorRAG Framework
To overcome the limitations of single-threaded "link-then-retrieve" methods, we propose AnchorRAG, a Multi-Agent Collaboration (MAC) framework. AnchorRAG decentralizes the retrieval process across three specialized agent roles:
1. The Predictor Agent
The Predictor Agent moves beyond simple string matching. It identifies a set of candidate entities using semantic indexing (FAISS) and employs a "Context-Aware Entity Disambiguation" strategy. By examining the neighboring relations of a node, the agent determines if the local graph structure aligns with the semantic context of the user's question.
2. Retriever Agents
Upon identifying candidate anchors, the framework initializes multiple Retriever Agents to explore the graph in parallel. This "divide and conquer" strategy ensures robustness; if one path leads to a dead end due to an ambiguous anchor, other agents may still find the correct evidence. These agents use LLMs to dynamically score and select the most promising relations and entities to follow.
3. The Supervisor Agent
The Supervisor Agent coordinates the overall process. It synthesizes the paths discovered by the retrievers and validates the evidence. By converting graph paths into natural language, the Supervisor allows the LLM to apply its full reasoning capabilities to verify if the retrieved information truly answers the user's query.
For resource-constrained environments, we also introduce AnchorRAG-LR (Lightweight Retrieval). This version replaces expensive LLM-based ranking with a hybrid BM25 and SBERT approach, significantly reducing latency and token costs while maintaining competitive performance.
Experimental Evaluation
AnchorRAG was evaluated on four major Knowledge Graph Question Answering (KGQA) benchmarks: WebQSP, GrailQA, CWQ, and WebQuestions. To test real-world applicability, we developed "Open-World" variants of WebQSP and GrailQA by introducing semantic noise and typos into the queries.
Key Results
- Performance: AnchorRAG achieved state-of-the-art results, notably improving Hit@1 accuracy on the complex GrailQA dataset by over 20% compared to baselines like Think-on-Graph (ToG).
- Robustness: In noisy open-world settings, AnchorRAG demonstrated superior resilience. While the ToG baseline saw a 23.4% performance drop on GrailQA, AnchorRAG's performance decreased by only 11.6%.
- Efficiency: The AnchorRAG-LR variant reduced token consumption by 77.8% and reasoning time by 63.8% compared to the full model, yet still outperformed existing strong baselines.
Conclusion
AnchorRAG represents a significant step toward practical, open-world KG-RAG. By leveraging multi-agent parallelism and context-aware disambiguation, the framework mitigates the "single point of failure" in entity linking. Future research will focus on refining agent communication protocols and scaling the framework to massive, heterogeneous data structures beyond current benchmarks.
Summary updated