Abstract
Current Retrieval-Augmented Generation (RAG) research often treats all external data tasks as uniform. This survey introduces a four-level stratification of user queries based on complexity—ranging from simple explicit facts to inferred hidden rationales. By categorizing tasks this way, developers can better select between context-based RAG, specialized small models, and fine-tuning to build more reliable domain-expert applications.
Introduction
Large Language Models (LLMs) frequently encounter "knowledge cutoff" and hallucinations when applied to specialized or private datasets. While RAG and fine-tuning are common solutions, standard pipelines often fail in high-stakes domains like law or medicine. This paper argues that these failures stem from a lack of task decomposition. Asking for a simple date is fundamentally different from asking for a decision based on complex historical patterns. This survey provides a roadmap for defining the reasoning required for diverse data-augmented tasks.
The Four Levels of Query Complexity
The core contribution of this work is the stratification of queries into four distinct levels of increasing difficulty:
- Level 1: Explicit Facts: Direct retrieval of information explicitly stated in the text (e.g., "What is the capital of France?").
- Level 2: Implicit Facts: Requires connecting multiple facts or applying basic common-sense reasoning (e.g., multi-hop questions).
- Level 3: Interpretable Rationales: Following explicit domain-specific rules or workflows provided in the external data (e.g., medical diagnostic guidelines).
- Level 4: Hidden Rationales: Inferring implicit expert logic or decision-making patterns from historical examples where the rules are not explicitly documented.
By identifying the query level first, developers can move away from "plugging leaks" in a generic RAG system and instead build a robust architecture tailored to the specific reasoning depth required.
Benchmarking the Framework
The survey categorizes several well-known benchmarks according to this four-level framework. While many existing datasets focus on Level 1 and Level 2, higher-level tasks often require specialized domain data.
- Level 1 (Explicit): Includes datasets like NQ (Natural Questions), MS MARCO, and SQuAD.
- Level 2 (Implicit): Includes multi-hop and reasoning datasets like HotPotQA, StrategyQA, and MuSiQue.
- Level 3 & 4 (Rationales): These levels typically utilize specialized datasets such as GLOBALCIT for legal reasoning or clinical diagnostic manuals.
Strategic Insights and Technical Solutions
The paper identifies three primary paths for injecting external data:
- Context: High interpretability but constrained by the model's context window.
- Small Model Assistance: Using specialized smaller models for routing or error correction.
- Fine-tuning: High capacity for capturing "hidden" knowledge, though it risks degrading general reasoning capabilities.
For Level 1 and 2 queries, hybrid retrieval (combining BM25 with dense vector search) remains the gold standard. Level 3 tasks benefit significantly from Chain-of-Thought (CoT) prompting and prompt tuning to ensure adherence to provided rules. Level 4 tasks, the most challenging, require In-Context Learning (ICL) with expert examples or Supervised Fine-Tuning (SFT) to capture nuanced "expert intuition."
Conclusion and Future Directions
A key takeaway is that RAG is generally superior for factual updates, while fine-tuning should be reserved for adjusting style and logic. Future research should focus on "routing pipelines" capable of automatically detecting a query's complexity level to trigger the appropriate sub-system. Additionally, improving evaluation metrics for fragmented data synthesis remains a critical challenge for advancing Level 4 reasoning.
Summary updated