Abstract
Retrieval-Augmented Generation (RAG) significantly enhances Large Language Models (LLMs) by providing external context, yet it often falters when faced with noisy or disparate information. We introduce HIRAG (Hierarchical-Thought Instruction-Tuning), a framework that shifts RAG from a simple "find-and-copy" mechanism to a structured "think-then-answer" process. By training models on a hierarchy of tasks—filtering noise, combining facts, and performing RAG-specific reasoning—HIRAG enables smaller 8B models to outperform much larger counterparts and specialized RAG systems.
Introduction
While LLMs are powerful, they are prone to hallucinations and outdated knowledge. RAG addresses this by allowing models to query external documents. However, current RAG systems suffer from a "garbage in, garbage out" problem: retrieved documents are frequently noisy, irrelevant, or require complex synthesis across multiple sources. Most existing fine-tuning methods lack a structured approach to handle these nuances, often failing to effectively utilize Chain-of-Thought (CoT) reasoning for retrieved text. HIRAG addresses this by instilling a granular, hierarchical set of skills necessary for mastering the "open-book exam" format of modern RAG tasks.
The HIRAG Framework
The core of HIRAG is a hierarchical strategy that introduces three levels of cognitive ability for processing retrieved information:
- Filtering: The foundational ability to ignore irrelevant "noise" and pinpoint the specific evidence required.
- Combination: The ability to synthesize semantic information across disparate paragraphs or documents to form a cohesive understanding.
- RAG-Specific Reasoning: The advanced ability to apply internal logic to external facts, encompassing comparative, deductive, and causal reasoning.
Methods and Training Pipeline
HIRAG employs a progressive instruction-tuning approach. The researchers generated a synthetic dataset of 120,000 samples using high-capacity models (e.g., GPT-4o) to produce structured "thoughts." These thoughts are encapsulated within special tokens: <|REASON|> for the internal monologue and <|ANSWER|> for the final output.
To ensure robustness, the training data intentionally includes 20-30% noisy documents. Ablation studies revealed an optimal 1:2:2 ratio for Filtering, Combination, and Reasoning tasks, suggesting that complex synthesis should be prioritized during training. This progressive learning—mastering easier tasks before moving to complex reasoning—ensures better model convergence.
Experimental Results
HIRAG was evaluated across diverse benchmarks, including RGB, PopQA, HotpotQA, and PubMedQA. Key findings include:
- State-of-the-Art Performance: HIRAG-8B outperformed established RAG models like RAFT and Self-RAG. Notably, it achieved results comparable to or better than Llama-3-70B-Instruct and GPT-4 on several tasks.
- Multi-hop Reasoning Gains: On the MuSiQue dataset, HIRAG improved accuracy by 7.7 percentage points over previous state-of-the-art models.
- Noise Robustness: In the RGB-noise benchmark, HIRAG maintained high accuracy even when 80% of the context was irrelevant.
- Efficiency: The reasoning step is computationally efficient, adding only approximately 0.14 seconds of latency on an A100 GPU for complex tasks.
Conclusion and Future Work
HIRAG demonstrates that teaching models how to think about their references is more effective than simply increasing model size. While it excels at general reasoning, future work will focus on refining performance in highly specialized vertical domains (e.g., legal or technical fields) and exploring reinforcement learning to further diversify reasoning pathways. By structuring the interaction between internal logic and external evidence, HIRAG sets a new standard for robust and intelligent Retrieval-Augmented Generation.
Summary updated