Introduction
Retrieval-Augmented Generation (RAG) has significantly enhanced Large Language Models (LLMs) by grounding them in external, factual data. However, traditional RAG systems often suffer from "conversational amnesia" in multi-turn dialogues. They typically treat each user query in isolation or rely on a static knowledge base, failing to account for the evolving context of a conversation. This leads to responses that, while factually correct, are contextually irrelevant or ambiguous when referring to previous turns.
To address this, we present DH-RAG, a framework that introduces a Dynamic Historical Information Database to mimic human cognitive processes, balancing long-term static knowledge with short-term conversational context.
Methodology
The DH-RAG architecture centers on a Dynamic Historical Information Database ($H$), which stores triples consisting of prior queries, retrieved passages, and generated responses. The system processes new queries through two primary modules.
History-Learning based Query Reconstruction Module (HL-QRM)
This module synthesizes the current query with historical context using three key strategies:
- Historical Query Clustering: Organizes past interactions into clusters to enhance retrieval efficiency.
- Hierarchical Matching: Employs a three-layered tree structure (Category $\rightarrow$ Summary $\rightarrow$ Historical Info) to accurately locate relevant past turns.
- Chain of Thought (CoT) Tracking: Sequentially aligns the current query with historical ones to maintain a logical reasoning chain throughout the dialogue.
Dynamic History Information Updating Module (DHI-UM)
To maintain an efficient "working memory," this module utilizes a TopN pruning strategy. It assigns a comprehensive weight to each historical entry based on Relevance (thematic connection to the current query) and Recency (temporal proximity). This ensures the database remains focused on the most pertinent information.
An attention mechanism finally weighs information from the static vanilla knowledge base against the dynamic historical context to produce a reconstructed query for the LLM.
Experimental Evaluation
DH-RAG was evaluated across several diverse benchmarks designed to test multi-turn dependency:
- MobileCS2: Domain-specific multi-turn mobile customer service interactions.
- PopQA_Mod & TriviaQA_Mod: Standard open-domain QA datasets adapted to simulate conversational flows.
- CoQA & TopiOCQA: Benchmarks specifically designed for conversational context dependency.
Results and Discussion
DH-RAG consistently outperformed baseline RAG methods and standard LLMs (e.g., Llama 3, GPT-4o-mini). Notable results include:
- Conversational QA Performance: On the TopiOCQA dataset, DH-RAG achieved an F1 score of 40.56, a significant improvement over SelfRAG's 3.24.
- Domain-Specific Success: In the MobileCS2 task, the model demonstrated a 215% relative increase in BLEU scores compared to the best-performing baseline.
- Ablation Studies: Removing the dynamic module led to a drastic performance drop (e.g., PopQA BLEU score falling from 44.99 to 3.87), validating the necessity of historical context integration.
Efficiency analysis showed that DH-RAG only increased total runtime by 6.4% compared to SelfRAG, though it did exhibit higher peak memory usage (4192.5 MB vs. 3018.6 MB).
Conclusion
DH-RAG effectively mitigates "conversational amnesia" in RAG systems by integrating a dynamic memory of dialogue history. By mimicking human-like memory management—prioritizing relevant and recent context—it achieves substantial gains in response coherence and accuracy for complex, multi-turn interactions. Future work will focus on optimizing memory overhead and refining the "forgetting" mechanisms for long-term scalability.
Summary updated