T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering.

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu

Abstract

Large Language Models (LLMs) often struggle with the "accuracy-vs-efficiency dilemma" in Contextual Question Answering (CQA), either over-thinking simple tasks or under-thinking complex ones. We present T² (Think-to-Think), an adaptive test-time scaling framework that dynamically adjusts reasoning depth. By decomposing questions and matching them to synthetic reasoning strategies based on a taxonomy of seven fundamental skills, T² optimizes the reasoning path for specific queries. Experimental results across seven benchmarks, including HotpotQA and SQuAD, demonstrate that T² improves accuracy by up to 21.3% while reducing token consumption by 25% and inference time by nearly 50% compared to standard test-time scaling methods.

Introduction

In Contextual Question Answering (CQA), the complexity of queries varies significantly. A simple factual extraction (e.g., "What is the capital of France?") requires minimal processing, whereas multi-hop logical reasoning (e.g., "Which actor from Inception was born in the same city as the director?") demands a deeper cognitive load.

Current LLMs typically operate at two extremes: "quick-thinking" direct generation, which often fails on complex logic, or "slow-thinking" Chain-of-Thought (CoT) reasoning, which can be unnecessarily verbose for simple questions. This verbosity leads to wasted tokens and increased latency. While existing Test-Time Scaling (TTS) methods attempt to address this via budget constraints or early-exit mechanisms, they often introduce human bias and fail to leverage the model's inherent understanding of a question's specific needs. T² addresses this by allowing the model to "think" about how it should "think" before executing a final answer.

Methodology

The T² framework employs a four-stage process to dynamically scale reasoning depth:

  1. Question Decomposition: Using a fine-tuned RoBERTa model, the system breaks a question into its structural "skeleton" (logical tokens) and "placeholders" (entities like names or dates).
  2. Similar Example Generation: The LLM generates a diverse set of synthetic "similar" questions based on the identified skeleton. These are paired with reasoning strategies derived from a taxonomy of seven skills: Deductive, Inductive, Abductive, Analogical, Causal, Critical, and Creative reasoning.
  3. Multi-Criteria Matching: This core innovation evaluates which synthetic reasoning strategy best fits the original question using two metrics:
  4. Skill Uniqueness Scoring: Weighting rarer, specialized reasoning skills higher to ensure complex logic is prioritized.
  5. Skill Coverage Assessment: Ensuring the selected strategy covers all necessary logical steps required by the query.
  6. Reasoning-Guided Answering: The model applies the selected "roadmap" to the document, focusing only on relevant segments and avoiding "retrace" (backtracking or mid-stream self-correction).

Experimental Setup

The researchers evaluated T² across seven diverse CQA benchmarks to ensure domain-agnostic performance:

  • SQuAD: General Wikipedia-based extraction.
  • HotpotQA: Complex multi-hop reasoning.
  • BioASQ: Specialized biomedical queries.
  • NewsQA: News-related passage comprehension.
  • GAOKAO: High-level academic exam questions.
  • HQA: Chronological historical questions.
  • TriviaQA: Wikipedia-based trivia.

The framework was tested using Qwen2.5-32B and GPT-4o, comparing it against heavyweights like Tree-of-Thoughts (ToT) and Monte Carlo Tree Search (MCTS).

Key Results and Insights

Performance and Efficiency

T² consistently outperformed established baselines in both accuracy and resource management. On the HotpotQA benchmark, T² achieved a 21.3% increase in accuracy compared to other test-time scaling methods.

Efficiency gains were equally significant:

  • Token Consumption: Reduced by 25.2% compared to QwQ-32B-Preview and 14.8% compared to Qwen2.5-32B using standard self-consistency.
  • Inference Speed: Total inference time was reduced by 41% to 47% across different models. This proves that the initial overhead of "thinking about thinking" is offset by the efficiency of the final execution.

The "Goldilocks" Zone of Reasoning

A critical finding of the study is that increasing reasoning length does not linearly correlate with performance. For simpler tasks, longer reasoning chains introduce noise and lead to errors. T² identifies the "Goldilocks" zone—the optimal reasoning length for a specific task. Furthermore, T² significantly reduces the "Retrace" phenomenon, where models realize an error mid-generation and attempt to correct themselves, leading to cleaner and more reliable outputs.

The Importance of Skill Rarity

The authors found that weighting reasoning skills by their rarity (Skill Uniqueness) is more effective than uniform sampling. When a question requires a rare skill, such as Abductive Reasoning, the model prioritizes examples that demonstrate that specific logic, leading to higher success rates on niche queries.

Discussion and Limitations

While T² represents a significant step forward in adaptive reasoning, the authors note several limitations:

  • Example Dependency: The system relies on the LLM's ability to generate high-quality synthetic examples. In extremely low-resource or niche domains, the model may struggle to find an appropriate reasoning template.
  • Text-Only Focus: The current iteration is designed for text-based CQA. Extending this to multimodal contexts (e.g., Visual Question Answering) would require architectural changes to handle non-textual inputs while maintaining efficiency.
  • Initial Overhead: There is a minor computational cost associated with the decomposition and strategy selection phases, though this is largely mitigated by the efficiency of the single-pass final answer.

Conclusion

T² offers a robust solution to the accuracy-vs-efficiency dilemma in LLMs. By dynamically scaling reasoning paths based on the specific logical requirements of a question, it achieves superior accuracy with significantly lower computational costs, paving the way for more efficient and intelligent automated reasoning systems.

Summary updated

BibTeX
@inproceedings{zhao2025t,
  author = {Zhengyi Zhao and Shubo Zhang and Zezhong Wang and Huimin Wang and Yutian Zhao and Bin Liang and Yefeng Zheng and Binyang Li and Kam-Fai Wong and Xian Wu},
  title = {{T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering.}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=u2bBYe0rgQ},
}
Figure