Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities

Weixiang Zhao, Xingyu Sui, Jiahe Guo, Yulin Hu, Yang Deng, Yanyan Zhao, Bing Qin, Wanxiang Che, Tat-Seng Chua, Ting Liu

Abstract

Recent advancements in Large Reasoning Models (LRMs), such as OpenAI's o1/o3 and DeepSeek-R1, have demonstrated remarkable performance in specialized reasoning tasks through human-like deliberative thinking and long chain-of-thought reasoning. However, our systematic evaluation across various model families (DeepSeek, Qwen, and LLaMA) and scales (7B to 671B) reveals that acquiring these deliberative reasoning capabilities significantly reduces the foundational capabilities of LRMs, including notable declines in helpfulness and harmlessness, alongside substantially increased inference costs. Importantly, we demonstrate that adaptive reasoning -- employing modes like Zero-Thinking, Less-Thinking, and Summary-Thinking -- can effectively alleviate these drawbacks. Our empirical insights underline the critical need for developing more versatile LRMs capable of dynamically allocating inference-time compute according to specific task characteristics.

Introduction: The Rise of the "Reasoning Tax"

The AI community has recently pivoted from standard "Chat" models to Large Reasoning Models (LRMs) that utilize extensive Chain-of-Thought (CoT) processing to solve complex problems. While these models set new benchmarks in mathematics and coding, their performance on "foundational" tasks—such as general knowledge, strict instruction following, and safety—remains poorly understood. This study identifies a significant "reasoning tax": the process of training a model for deep deliberation often causes it to "over-think" simple tasks, leading to hallucinations in the reasoning trace and a failure to adhere to negative constraints.

The Hidden Costs of Deliberation

Systematic evaluation across DeepSeek, Qwen, and LLaMA families reveals a stark trade-off. As models gain reasoning prowess, their foundational capabilities—measured by benchmarks like IFEval (instruction following) and WildJailbreak (safety)—often degrade. For instance, the distilled model s1.1-32B showed a 47.38% decrease in instruction-following capability compared to its base chat counterpart, despite a 250% increase in inference cost.

Figure 2: Comparison of Foundational vs. Reasoning performance and the associated token length costs across various model families.

This degradation is often linked to the length of the reasoning trace. In general tasks, failed samples frequently exhibit significantly longer reasoning traces than successful ones. Excessive self-reflection can lead the model away from the user's original instructions, a phenomenon termed the "over-thinking penalty."

Adaptive Reasoning as a Solution

To mitigate these trade-offs, the researchers introduced several Adaptive Reasoning Modes designed to dynamically manage the model's deliberative process:

  • Zero-Thinking: Bypassing the <think> block entirely for simple queries.
  • Less-Thinking: Truncating the reasoning process at a specific percentage of its total length.
  • Summary-Thinking: Condensing the long reasoning trace into a concise summary before generating the final answer.
  • Summary-Thinking-Plus: Preserving the critical first sentence of the reasoning trace while summarizing the remainder.
Figure 4: Performance of s1.1-32B and QwQ-32B across different benchmarks as the "Thinking Ratio" is adjusted.

As shown in Figure 4, different tasks respond differently to thinking ratios. While complex reasoning tasks (like MMLU-Pro) benefit from higher ratios, safety benchmarks (like WildJailbreak) often see performance peaks at lower ratios. For R1-Distill-Llama-70B, switching to "Zero-Thinking" mode caused its jailbreak resistance to surge from 28.25 to 89.10, demonstrating that internal reasoning can sometimes brainstorm harmful steps that leak into the final output.

Conclusion: Toward Dynamic Compute Allocation

The primary takeaway for future LRM development is the necessity of Dynamic Compute Allocation. Rather than a fixed deliberative approach, future models should autonomously categorize input difficulty and decide whether to employ "System 1" (fast, intuitive) or "System 2" (slow, deliberative) reasoning. By adopting adaptive modes like Summary-Thinking, models can recover lost foundational skills and improve inference efficiency without sacrificing their core reasoning strengths.

Summary updated

Abstract

Recent advancements in Large Reasoning Models (LRMs), such as OpenAI's o1/o3 and DeepSeek-R1, have demonstrated remarkable performance in specialized reasoning tasks through human-like deliberative thinking and long chain-of-thought reasoning. However, our systematic evaluation across various model families (DeepSeek, Qwen, and LLaMA) and scales (7B to 671B) reveals that acquiring these deliberative reasoning capabilities significantly reduces the foundational capabilities of LRMs, including notable declines in helpfulness and harmlessness, alongside substantially increased inference costs. Importantly, we demonstrate that adaptive reasoning -- employing modes like Zero-Thinking, Less-Thinking, and Summary-Thinking -- can effectively alleviate these drawbacks. Our empirical insights underline the critical need for developing more versatile LRMs capable of dynamically allocating inference-time compute according to specific task characteristics.
BibTeX
@inproceedings{zhao2026trade,
  author = {Weixiang Zhao and Xingyu Sui and Jiahe Guo and Yulin Hu and Yang Deng and Yanyan Zhao and Bing Qin and Wanxiang Che and Tat-Seng Chua and Ting Liu},
  title = {{Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i41.40802},
}
Figure