T: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu

TL;DR

This paper introduces T² (Think-to-Think), a novel framework that allows Large Language Models (LLMs) to dynamically adapt their reasoning depth based on question complexity for Contextual Question Answering (CQA). By learning from similar examples, T² achieves higher accuracy and significantly reduces computational overhead compared to existing methods that use a one-size-fits-all approach.

What Problem Does This Paper Address?

Current LLMs excel at CQA, but they often struggle with efficiency and adaptability. Many models employ a fixed reasoning strategy for all questions, whether simple or complex. This "one-size-fits-all" approach leads to two main issues:

  1. Inefficiency: Simple questions that could be answered directly or with minimal reasoning are subjected to lengthy, verbose, and often redundant step-by-step explanations. This wastes computational resources and increases deployment costs.
  2. Lack of Adaptability: While complex questions benefit from detailed reasoning, simpler ones might be hindered by overly elaborate processes. Existing "test-time scaling" methods that try to address this by setting budgets or early stopping often introduce human bias and fail to fully leverage the LLM's inherent reasoning capabilities.

The paper aims to bridge this gap by developing a system that can intelligently choose the appropriate level of reasoning for each question, balancing accuracy with computational efficiency.

Methods

T² (Think-to-Think) is a framework that dynamically adapts reasoning strategies. It operates in four key steps:

  1. Question Decomposition: The original question is broken down into its structural elements (framework tokens) and replaceable entities (placeholders). These placeholders are then assigned semantic types (e.g., person, location) to create a question template.
  2. Similar Examples Generation: Based on the question template, T² generates a diverse set of similar example questions that share the same structure but have different entities. For each similar question, it also constructs a corresponding reasoning strategy (a sequence of reasoning skills) and identifies relevant document segments. This process draws from a taxonomy of fundamental human reasoning skills.
  3. Multi-Criteria Selection: The framework uses a multi-criteria matching process to select the most appropriate reasoning strategy from the generated examples. This involves evaluating strategies based on:
  4. Skill Uniqueness: Prioritizing rarer reasoning skills.
  5. Skill Coverage: How well the strategy covers the required reasoning skills for the original question.
  6. Structural Similarity: How closely the example question's structure matches the original.
  7. Reasoning Strategy-Guided Answering: The selected reasoning strategy and relevant document segments are used to guide the LLM in answering the original question. This involves extracting specific text segments relevant to each reasoning step and formatting a prompt that instructs the LLM to follow the chosen strategy.

The core novelty lies in adaptively selecting reasoning strategies based on learned patterns from similar examples, rather than relying on pre-defined templates or human-imposed biases.

Datasets

The paper evaluates T² on seven diverse Contextual Question Answering (CQA) datasets:

  • SQuAD: General-domain questions from Wikipedia.
  • HotpotQA: Multi-hop questions requiring reasoning across multiple facts.
  • BioASQ: Biomedical queries requiring specialized knowledge.
  • NewsQA: News-related passages.
  • GAOKAO: Exam-oriented dataset with academic coverage.
  • HQA: Historical questions focusing on chronology and figures.
  • TriviaQA: Wikipedia-based trivia.

Key Insights and Ideas

  • Leveraging Similar Examples is Key: The core insight is that if an LLM can solve similar questions using a particular reasoning strategy, it can apply that same strategy to the original question. This allows for dynamic adaptation without needing to pre-train for every possible complexity.
  • Structure Over Content: T² emphasizes preserving the structure of reasoning paths from similar examples, even if the specific entities or content differ. This allows for generalization across diverse questions.
  • Balancing Accuracy and Efficiency: The framework's multi-criteria selection mechanism is crucial for finding the sweet spot between thoroughness (for complex questions) and conciseness (for simple questions), leading to both higher accuracy and reduced computational cost.
  • Adaptive Reasoning Reduces Redundancy: By selecting appropriate reasoning paths, T² effectively eliminates unnecessary steps that plague many current LLM reasoning approaches, leading to significant efficiency gains.

Key Results

  • Improved Accuracy: T² achieves superior accuracy compared to baseline methods, with up to a 21.3% increase on some benchmarks.
  • Reduced Computational Overhead: The framework reduces computational requirements by up to 25.2% (e.g., token reduction). This efficiency is particularly pronounced for simpler questions.
  • Effective Reasoning Skill Utilization: Ablation studies show that combining skill coverage and uniqueness in the selection mechanism significantly boosts performance, outperforming approaches that use only one component or random sampling.
  • Reduced Retrace Rate: T² helps models achieve correct answers more directly, minimizing the need for backtracking or revising answers within the reasoning chain.

Limitations and Future Work

  • Reliance on High-Quality Examples: The effectiveness of T² depends on the availability of high-quality example reasoning strategies for similarity matching. In domains with limited annotated data or novel questions, the framework might struggle.
  • Textual Reasoning Focus: The current implementation is primarily focused on textual reasoning tasks. Extending T² to multimodal contexts (e.g., visual question answering) would require architectural modifications.
  • Future Work: The authors suggest exploring T²'s application in multimodal reasoning and further investigating its adaptability to different types of reasoning tasks.

Summary updated

Abstract

Recent advances in large language models have demonstrated remarkable performance on Contextual Question Answering (CQA). However, prior approaches typically employ elaborate reasoning strategies regardless of question complexity, leading to low adaptability. Recent efficient test-time scaling methods introduce budget constraints or early stop mechanisms to avoid overthinking for straightforward questions. But they add human bias to the reasoning process and fail to leverage models’ inherent reasoning capabilities. To address these limitations, we present T2: Think-to-Think, a novel framework that dynamically adapts reasoning depth based on question complexity. T2 leverages the insight that if an LLM can effectively solve similar questions using specific reasoning strategies, it can apply the same strategy to the original question. This insight enables to adoption of concise reasoning for straightforward questions while maintaining detailed analysis for complex problems. T2 works through four key steps: decomposing questions into structural elements, generating similar examples with candidate reasoning strategies, evaluating these strategies against multiple criteria, and applying the most appropriate strategy to the original question. Experimental evaluation across seven diverse CQA benchmarks demonstrates that T2 not only achieves higher accuracy than baseline methods but also reduces computational overhead by up to 25.2%.
BibTeX
@inproceedings{zhao2025t,
  author = {Zhengyi Zhao and Shubo Zhang and Zezhong Wang and Huimin Wang and Yutian Zhao and Bin Liang and Yefeng Zheng and Binyang Li and Kam-Fai Wong and Xian Wu},
  title = {{T: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=u2bBYe0rgQ},
}

Loading related content...

Figure