Tree-of-Quote Prompting Improves Factuality and Attribution in Multi-Hop and Medical Reasoning

Justin Xu, Yiming Li, Zizheng Zhang, Augustine Luk, Mayank Jobanputra, Samarth Oza, Andrew W. Murray, Meghana Reddy Kasula, Andrew Parker, David W. Eyre2025 · DOI

Abstract: Justin Xu, Yiming Li, Zizheng Zhang, Augustine Yui Hei Luk, Mayank Jobanputra, Samarth Oza, Ashley Murray, Meghana Reddy Kasula, Andrew Parker, David W Eyre. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

2025.

Introduction

Large Language Models (LLMs) often struggle with hallucination and poor attribution, particularly in complex multi-hop reasoning tasks where errors can cascade. In high-stakes domains like medicine, the "black box" nature of these models is a significant barrier to adoption. To address this, researchers have developed Tree-of-Quote (ToQ) prompting, a framework that integrates internal quoting as a fundamental part of the reasoning process rather than a post-hoc addition.

The Tree-of-Quote (ToQ) Framework

ToQ structures reasoning as a dynamic graph (implemented via LangGraph), ensuring that every step in a chain of thought is grounded in verifiable evidence. The process is managed through four primary nodes:

  1. Initialization: The model decomposes a complex question into simpler sub-questions.
  2. Quoting: The model generates internal quotes from its training data (e.g., "According to PubMed...") to answer sub-questions without external retrieval.
  3. Scoring: Quotes are evaluated using the QUIP (Quoted Information Precision) metric. If a quote falls below a quality threshold, the system triggers a retry or prunes that reasoning branch.
  4. Questioning: The model assesses if it has sufficient context for a final answer or if further sub-questions are required.
The Tree-of-Quote (ToQ) reasoning process involving decomposition, attribution via quoting, and scoring for factual verification.

Evaluation and Datasets

The framework was evaluated across five diverse datasets, including general multi-hop QA (StrategyQA, 2WikiMultiHopQA, MuSiQue, MoreHopQA) and medical reasoning (MedQA). To validate the results, the researchers introduced the FQ-Score (Factuality with Quoting Score), a holistic metric combining correctness, attribution quality, and logical flow. This metric was validated through reader studies involving clinicians to ensure alignment with expert human judgment.

Key Results and Insights

ToQ consistently outperforms standard prompting techniques like Chain-of-Thought (CoT) and Chain-of-Verification (CoVe). On StrategyQA, ToQ achieved 81.7% accuracy with GPT-4o. In medical contexts, it provided a transparent "paper trail," with human evaluators confirming the accuracy of 37 out of 45 generated quotes.

Performance comparison of ToQ against other prompting methods using the DeepSeek-Chat backend.

A core insight of ToQ is that the act of quoting serves as a factual filter, catching hallucinations early in the reasoning chain. The study also demonstrates that LLMs often possess the necessary facts within their weights; they simply require structured prompting to access them reliably.

Limitations and Future Directions

While effective, ToQ is computationally more expensive than standard prompting, requiring roughly 2x more LLM calls and tokens due to its iterative loops. Future work aims to integrate ToQ with external Retrieval-Augmented Generation (RAG) systems and optimize the framework for smaller, more efficient language models to reduce operational costs.

Summary updated

BibTeX
@misc{xu2025tree,
  author = {Justin Xu and Yiming Li and Zizheng Zhang and Augustine Luk and Mayank Jobanputra and Samarth Oza and Andrew W. Murray and Meghana Reddy Kasula and Andrew Parker and David W. Eyre},
  title = {{Tree-of-Quote Prompting Improves Factuality and Attribution in Multi-Hop and Medical Reasoning}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.285},
}
Figure