TL;DR
This paper explores integrating Retrieval-Augmented Generation (RAG) with Diffusion Language Models (DLMs), a promising but underexplored area. They identify "Response Semantic Drift" (RSD) as a key issue where DLM outputs deviate from the original query's meaning. To combat this, they propose SPREAD, a novel framework that guides DLM denoising using query relevance, significantly improving answer precision and coherence.
What Problem Does This Paper Address?
Large Language Models (LLMs) have been significantly boosted by Retrieval-Augmented Generation (RAG), which grounds their outputs in external knowledge. Diffusion Language Models (DLMs), a newer class of models known for their parallel decoding and fine-grained control, also face challenges like knowledge gaps and hallucinations. However, effectively combining DLMs with RAG has been hindered by their fundamentally different decoding processes compared to LLMs. This paper investigates this integration, specifically addressing a critical performance bottleneck: Response Semantic Drift (RSD). RSD occurs when a DLM's generated answer, while potentially factually correct and derived from retrieved context, gradually drifts away from the original query's core semantics, leading to less precise and sometimes redundant information. This drift is traced back to the local, greedy nature of standard DLM denoising strategies, which prioritize token-level confidence over global semantic alignment with the query.
Methods
The core of this paper is the introduction of Semantic-Preserving REtrieval-Augmented Diffusion (SPREAD). SPREAD modifies the standard DLM denoising process by incorporating a query-relevance-guided denoising strategy.
Here's how it works:
- Contextualized Embedding Extraction: At each denoising step, SPREAD leverages the DLM's own forward pass to extract contextualized hidden layer representations for both the current sequence state and the query. This avoids additional encoding overhead.
- Semantic Similarity Computation: It then computes the cosine similarity between the hidden state of each masked token position and the query's representation. This similarity score is mapped to a normalized relevance score using a sigmoid function.
- Relevance-Guided Token Selection: Instead of solely relying on local model confidence (as in traditional methods), SPREAD uses these relevance scores to guide token selection. It prioritizes decoding tokens that are most semantically relevant to the query, effectively steering the generation process to maintain alignment with the query's semantics throughout all denoising steps. This ensures that the generation remains anchored to the query, suppressing semantic drift.
Datasets
The paper evaluates SPREAD on six widely-used open-domain question-answering (QA) datasets:
- Natural Questions (NQ) and TriviaQA: Focus on factual QA.
- HotpotQA, MuSiQue, and MultiHop-RAG: Multi-hop QA datasets requiring reasoning and synthesis of information from multiple documents.
- UltraDomain: Challenges models with in-depth understanding within specialized vertical domains.
The paper notes that MultiHop-RAG is licensed under ODC-BY 1.0, MuSiQue under CC BY 4.0, and HotpotQA under CC BY-SA 4.0.
Key Insights and Ideas
- RSD is a critical bottleneck for DLM-RAG: The paper convincingly demonstrates that simply applying RAG to DLMs doesn't guarantee optimal results. The inherent "semantic drift" in DLM generation is a significant issue that needs to be addressed.
- Local denoising vs. Global semantic alignment: The core problem lies in the conflict between DLMs' step-wise, locally optimal denoising and the need for global semantic coherence with the query in RAG tasks.
- Query relevance as a guiding principle: SPREAD's key innovation is using query relevance as a direct signal to guide the denoising process, rather than relying solely on internal model confidence. This is a more principled way to ensure the generated output stays on topic.
- High copy rate doesn't equal high quality: The paper shows that while DLMs can achieve high copy rates from retrieved context, this doesn't automatically translate to precise answers. The content being copied matters more than the quantity.
Key Results
- Significant Precision Improvement: SPREAD consistently and significantly improves answer precision across all datasets and models. It achieves an average absolute increase of 15.34% on LLaDA and 30.90% on Dream.
- Substantial RSD Reduction: The method effectively mitigates Response Semantic Drift, with an average relative reduction exceeding 10.92% on LLaDA and 61.18% on Dream.
- Minimal Efficiency Overhead: SPREAD introduces only a negligible increase in computational time (around 0.23s for LLaDA and 0.68s for Dream) because it reuses existing model computations.
- No significant impact on Copy Rate: SPREAD does not drastically alter the copy rate, but it does significantly decrease redundancy, indicating more focused and relevant content is being copied.
- Overall Superiority: Multi-dimensional analysis using metrics like F1, Precision, ROUGE-1, and ROUGE-L shows SPREAD covering a larger area, indicating balanced and effective generation quality.
Limitations and Future Work
- Contextual Relevance Dependency: SPREAD's performance is still contingent on the relevance of the retrieved context. The authors suggest that co-designing the retriever and generator could further enhance performance.
- Extension to Other Tasks: While validated on QA, the authors aim to extend SPREAD to more open-ended generation tasks like story writing.
- Open-ended Generation: The paper focuses on QA, and its application to more creative or free-form generation tasks might require further adaptation.
Summary updated