Abstract
Contextual Question Answering (CQA) often suffers from an imbalance between reasoning depth and computational efficiency. We present Think-to-Think ($T^2$), a novel framework that enables Large Language Models (LLMs) to dynamically scale their reasoning based on question complexity. By generating synthetic examples to determine optimal reasoning strategies, $T^2$ improves accuracy while reducing computational overhead by up to 25.2% and inference time by 56.3% compared to existing slow-thinking methods.
Introduction
Current CQA systems face a significant "accuracy-vs-efficiency" trade-off. Direct answering often fails on complex, multi-hop questions, while uniform Chain-of-Thought (CoT) reasoning leads to "overthinking" on simple tasks. This overthinking results in verbose, redundant outputs that are not only costly but can also degrade performance. Existing Test-Time Scaling (TTS) methods typically rely on fixed budgets or early-stopping, which often fail to capture the inherent difficulty of a task. $T^2$ addresses this by allowing the model to self-evaluate and adapt its reasoning strategy at test time.
Methodology
The $T^2$ framework operates through a four-stage pipeline designed to match the reasoning depth to the specific requirements of the input question.
- Question Decomposition: The system uses a fine-tuned RoBERTa model to extract the "skeleton" of a question, separating structural tokens from specific entities.
- Similar Examples Generation: The LLM generates synthetic questions based on the extracted structure and constructs corresponding reasoning strategies (sub-questions and specific reasoning skills).
- Multi-Criteria Matching: The system evaluates synthetic examples using Skill Coverage and Skill Uniqueness to identify the most appropriate strategy for the original question.
- Strategy-Guided Answering: The selected strategy guides the model to extract relevant document segments and produce a concise answer.
Experimental Evaluation
The framework was validated across seven diverse CQA benchmarks, including SQuAD, HotpotQA, BioASQ, NewsQA, GAOKAO, HQA, and TriviaQA. These datasets cover general knowledge, multi-hop reasoning, specialized domains (biomedical, history), and academic testing.
Results and Discussion
$T^2$ consistently outperforms traditional baselines in both accuracy and efficiency. For example, on the SQuAD dataset, $T^2$ improved the ROUGE-L score of Qwen2.5-32B from 73.41 to 81.86.
Key findings include:
- Efficiency Gains: $T^2$ reduced token consumption by up to 25.2% compared to QwQ-32B-Preview and accelerated inference by 56.3% over MCTS-based methods.
- Reasoning Quality: The framework significantly reduces "retracing" (mid-output revisions), leading to cleaner reasoning paths. As shown in the performance analysis, $T^2$ achieves a higher "Hit" rate in retrieving required facts while minimizing the introduction of spurious information (Error rate).
- Adaptability: The "Think-to-Think" principle allows the model to recognize when a direct approach is sufficient, avoiding the pitfalls of forced long-chain reasoning.
Conclusion and Future Work
$T^2$ demonstrates that adaptive test-time scaling is a viable path toward more efficient and accurate LLM reasoning. While highly effective in textual CQA, future work will explore extending this framework to multimodal contexts and addressing potential structural biases in synthetic example generation.
Summary updated