Typed-RAG: Type-Aware Decomposition of Non-Factoid Questions for Retrieval-Augmented Generation

DongGeon Lee, Ahjeong Park, Hyeri Lee, Hyeonseo Nam, Yunho MaengACL 2025

TL;DR

This paper introduces TYPED-RAG, a novel framework that tackles the challenge of non-factoid question answering (NFQA) by decomposing complex questions into smaller, type-specific sub-queries. This approach significantly improves the quality and relevance of answers generated by retrieval-augmented generation (RAG) systems, outperforming existing methods. The authors also introduce Wiki-NFQA, a new benchmark dataset to facilitate further research in this area.

What Problem Does This Paper Address?

Traditional question answering (QA) systems have largely focused on "factoid" questions, which have single, verifiable answers (e.g., "When was Google founded?"). However, real-world information needs often extend to "non-factoid" questions (NFQs). These questions are more complex, requiring nuanced, multi-faceted, and interpretive responses that integrate multiple perspectives. Examples include comparative questions ("What is the difference between X and Y?"), experience-based queries ("What are the best Portuguese wines?"), or debate-style questions ("Is Trump a good president?").

Existing retrieval-augmented generation (RAG) systems, while improving contextuality, often produce overly homogeneous answers for NFQs, lacking the depth and multi-faceted nature required. This is because NFQs vary significantly in their intent, aspect directionality, and degree of contrast. Current approaches struggle to adapt retrieval and generation strategies to these diverse characteristics. The core challenge is to effectively handle the inherent complexity and heterogeneity of NFQs.

Methods

TYPED-RAG addresses this challenge by integrating question type classification directly into the RAG paradigm. The framework operates as follows:

  1. Type Classification: An NFQ is first classified into one of six predefined types (Evidence-based, Comparison, Experience, Reason, Instruction, Debate) using a pre-trained classifier.
  2. Multi-Aspect Decomposition: Based on the identified question type, a "Multi-aspect Decomposer" breaks down the question into focused, single-aspect sub-queries. This decomposition is tailored to the specific requirements of each NFQ type.
  3. For Evidence-based questions, a straightforward RAG approach is used as they are inherently single-aspect.
  4. For Comparison questions, a Keyword Extractor identifies the comparison type (differences, similarities, superiority) and the items being compared.
  5. For Experience questions, key topics and entities are extracted to guide retrieval.
  6. For Reason/Instruction questions, the original query is decomposed into multiple distinct sub-queries that preserve the core intent.
  7. For Debate questions, the debate topic and key perspectives are identified, and biased sub-queries are generated for each perspective.
  8. Targeted Retrieval: Each decomposed sub-query is then processed by one or more retrievers to find highly relevant passages. Optionally, these passages are re-ranked.
  9. Answer Generation: A generator produces answers for each sub-query.
  10. Answer Aggregation: An "Answer Aggregator" synthesizes the individual sub-responses into a comprehensive, contextually aligned final answer. This aggregation process is also type-specific.

The novelty lies in the type-aware decomposition and the subsequent tailored retrieval and generation strategies for each NFQ type, moving beyond generic RAG approaches.

Datasets

The authors introduce Wiki-NFQA, a new benchmark dataset specifically curated for NFQA. It is derived from existing Wikipedia-based datasets like Natural Questions (NQ), SQuAD (SQD), TriviaQA (TQA), 2Wiki-MultiHopQA (2WMH), HotpotQA (HQA), and MuSiQue (MSQ). The dataset was filtered to extract NFQs and then enriched with diverse, high-quality reference answers to support evaluation metrics like LINKAGE.

Key Insights and Ideas

  • Type-Awareness is Crucial for NFQA: The fundamental insight is that non-factoid questions are not monolithic. Their diverse types (comparison, experience, debate, etc.) necessitate distinct handling strategies for effective retrieval and generation.
  • Decomposition Unlocks Nuance: Breaking down complex NFQs into single-aspect sub-queries allows for more focused retrieval and, consequently, richer, more nuanced answers that capture multiple perspectives.
  • RAG's Noise Problem: Direct RAG can introduce noise from irrelevant retrieved documents, hindering response generation for NFQs. TYPED-RAG mitigates this by structuring retrieval around specific facets of the question.
  • Unified Framework for Diverse NFQs: TYPED-RAG provides a single, adaptable framework that can handle various NFQ types, unlike previous approaches that might focus on only one or two specific types.

Key Results

TYPED-RAG consistently outperformed both LLM-only and standard RAG baselines across various language models (Llama-3.2-3B, Mistral-7B) and scorer LLMs (Mistral-7B, GPT-40 mini) on the Wiki-NFQA dataset.

  • MRR and MPR Improvements: TYPED-RAG achieved significant gains in both Mean Reciprocal Rank (MRR) and Mean Percentile Rank (MPR), indicating better answer quality and ranking. For example, on SQUAD-NF with Mistral-7B and GPT-40 mini scorer, TYPED-RAG achieved an MRR of 0.7444, significantly outperforming LLM-only (0.4222) and RAG-based (0.3817) baselines.
  • Robustness: The framework demonstrated robustness across different NFQ categories and scorer LLM strictness.
  • Handling Complexity: TYPED-RAG showed particular strength in datasets requiring multi-hop reasoning (HotpotQA-NF, MuSiQue-NF) and complex reasoning chains, effectively navigating these challenges through its decomposition strategy.

Limitations and Future Work

  • Absence of Direct Comparison: A key limitation is the lack of direct comparison with existing query rewriting and decomposition methodologies. The authors suggest future work should incorporate benchmark evaluations against these established techniques.
  • Self-Evaluation Bias: The current evaluation setup uses LLMs to assess answer quality, which might introduce bias. Future work could explore human assessments or ensemble scoring methods.
  • Future Directions: The authors suggest extending TYPED-RAG to incorporate more fine-grained question types, refine decomposition strategies, and apply the approach to other specific domains. Integrating more sophisticated retrieval mechanisms is also a promising avenue.

Summary updated

BibTeX
@inproceedings{lee2025typed,
  author = {DongGeon Lee and Ahjeong Park and Hyeri Lee and Hyeonseo Nam and Yunho Maeng},
  title = {{Typed-RAG: Type-Aware Decomposition of Non-Factoid Questions for Retrieval-Augmented Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure