UniRAG: A Unified RAG Framework for Knowledge-Intensive Queries with Decomposition, Break-Down Reasoning, and Iterative Rewriting

Gun Il Kim, Jong Wook Kim, Beakcheol JangEMNLP 2025

TL;DR

This paper introduces UniRAG, a novel framework that unifies query decomposition, break-down reasoning, and iterative rewriting to significantly improve the accuracy and reliability of Retrieval-Augmented Generation (RAG) for knowledge-intensive queries. By systematically breaking down complex questions, verifying sub-facts, and adaptively refining queries, UniRAG achieves state-of-the-art performance across various LLMs and challenging benchmarks.

What Problem Does This Paper Address?

Knowledge-intensive queries, which require precise answers grounded in factual evidence, pose a significant challenge for current Large Language Models (LLMs) and existing Retrieval-Augmented Generation (RAG) approaches. These systems often struggle with:

  • Query Complexity: Multi-hop questions, queries with multiple constraints, or domain-specific queries that require nuanced external knowledge.
  • Reasoning Errors: Propagating errors from initial retrieval or reasoning steps, leading to inaccurate or incomplete answers.
  • Incomplete/Noisy Retrieval: Relying on insufficient or irrelevant retrieved documents.
  • Hallucinations: LLMs generating plausible but factually incorrect information.

Existing RAG methods, while improving over standalone LLMs, often achieve only modest gains on these complex queries. Approaches like query decomposition and rewriting have limitations, such as requiring substantial supervision or suffering from LLM hallucinations. Knowledge reasoning methods, while promising, can also be susceptible to error propagation and computational overhead. UniRAG aims to address these limitations by creating a more robust and integrated framework.

Methods

UniRAG is a unified framework that synergistically combines three core active retrieval strategies:

  1. Entity-Grounded Query Decomposition:
    • Novelty: Instead of generic decomposition, UniRAG first identifies key entities (persons, locations, dates, etc.) in the original query using a specialized NER model (FLERT).
  2. Process: These entities are then used to prompt an LLM to generate semantically coherent and focused sub-queries. This ensures each sub-query targets a distinct aspect of the original query and maintains semantic alignment.
  3. Break-Down Reasoning:
    • Novelty: This module explicitly verifies each sub-fact derived from the decomposed queries.
  4. Process: * Relevance Filtering: Documents retrieved for both the original query and its sub-queries are combined. A semantic reranking step using the mGTE model filters these documents to ensure high relevance to the original query.
  5. Retrieval Reasoning: The LLM is prompted with "Let's Break It Down" to independently verify each conceptual component (sub-fact) of the query against the filtered documents. This structured reasoning process aims to prevent error propagation and ensure factual completeness.
  6. Confidence-Based Decision: The RAGAS framework is used to evaluate the confidence of the generated answer. If confidence is low, it triggers the iterative rewriting step.
  7. Iterative Query Rewriting:
    • Novelty: This mechanism adaptively refines queries based on identified knowledge gaps during the reasoning phase.
  8. Process: If the confidence-based decision indicates insufficient support for an answer, the LLM is prompted to generate a new, refined query. This rewritten query is informed by the LLM's previous reasoning and explicitly avoids previously used sub-queries, encouraging exploration of alternative search avenues. The process then loops back to query decomposition.

UniRAG operates entirely through prompting, requiring no additional model training, making it flexible and easy to deploy.

Datasets

UniRAG was evaluated on a diverse set of challenging benchmarks:

  • Multi-hop QA: HotPotQA, 2WikiMultihopQA
  • Commonsense Reasoning: StrategyQA
  • Biomedical: MedMCQA, MedQA
  • Fact-Verification: FEVER, SciFact
  • Logical Deduction: LogiQA, FOLIO
  • Arithmetic Problems: GSM8K, MAWPS

The retrieval knowledge base used was primarily Wikipedia abstract dumps and PubMed abstract dumps.

Key Insights and Ideas

  • Entity-Grounding is Key for Decomposition: Simply decomposing queries isn't enough. Grounding the decomposition process with identified entities from the query leads to more semantically relevant and focused sub-queries.
  • Break-Down Reasoning Prevents Error Propagation: Unlike sequential Chain-of-Thought (CoT) prompting, the "break it down" approach allows for parallel verification of sub-facts, significantly reducing the risk of errors cascading through the reasoning process.
  • Iterative Rewriting as Targeted Refinement: Query rewriting, when guided by identified knowledge gaps from the reasoning phase, becomes a powerful tool for adaptively improving retrieval and addressing specific information deficits.
  • Synergistic Effect of Modules: Each component (decomposition, reasoning, rewriting) contributes significantly, but their combined effect is far greater than the sum of their parts, leading to substantial performance gains.
  • Optimal Sub-Query Granularity Varies: While 8 sub-queries proved optimal for multi-hop tasks with LLaMA-3.1-8B, the ideal number of sub-queries can vary by task domain, with simpler fact-verification tasks benefiting from fewer sub-queries.

Key Results

UniRAG consistently outperformed state-of-the-art baselines across various LLMs (GPT-3.5-Turbo, GPT-40, Gemini-1.5-Flash, LLaMA-3.1-8B, Qwen-2.5-7B, Gemma2-9B) and datasets.

  • Significant Improvements: For example, with LLaMA-3.1-8B, UniRAG achieved an EM score of 0.766 on HotPotQA, a substantial leap from 0.346 with just the reasoning module.
  • Generalizability: The framework demonstrated robust performance across diverse tasks, including multi-hop QA, biomedical, and fact-verification.
  • "Let's Break It Down" Prompting: This specific prompt, used in the reasoning phase, showed superior performance compared to standard CoT prompting on biomedical and fact-verification datasets.
  • Module-Wise Performance: Ablation studies showed that while each module contributed, the full UniRAG framework (decomposition + reasoning + rewriting + reranking) yielded the most significant improvements.

Limitations and Future Work

  • Confidence Score Sensitivity: The framework relies on a confidence score decision-making process using RAGAS, which could potentially be sensitive to threshold settings. However, empirical observations suggested the model's performance was not significantly sensitive to this threshold in their experiments.
  • Optimal Sub-Query Number: While an optimal number of sub-queries was identified for specific tasks and models (e.g., 8 for HotPotQA/2WikiMultihopQA with LLaMA-3.1-8B), the paper notes that different task domains benefit from varying levels of decomposition. Future work could explore more dynamic or adaptive methods for determining the optimal number of sub-queries.
  • Computational Overhead: While UniRAG aims for efficiency, iterative processes in RAG can still incur computational costs. The paper suggests that UniRAG achieves a "near-optimal balance of accuracy and efficiency within just a few steps," implying that the iterative nature is manageable.

Summary updated

BibTeX
@inproceedings{kim2025unirag,
  author = {Gun Il Kim and Jong Wook Kim and Beakcheol Jang},
  title = {{UniRAG: A Unified RAG Framework for Knowledge-Intensive Queries with Decomposition, Break-Down Reasoning, and Iterative Rewriting}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure