TL;DR
While Large Reasoning Models (LRMs) excel at complex math and logic via long chain-of-thought reasoning, this paper reveals a hidden cost: acquiring these capabilities significantly degrades foundational skills like general instruction-following and safety alignment. Fortunately, the authors show that "adaptive reasoning"—dynamically controlling or summarizing the model's thinking process at inference time—can mitigate these trade-offs and restore general task performance.
What Problem Does This Paper Address?
State-of-the-art models like OpenAI's o1/o3 and DeepSeek-R1 have popularized the paradigm of test-time compute, where models generate long, internal "thinking" traces before answering. While this works wonders for specialized reasoning tasks (like competitive programming or math Olympiads), it remains unclear how this "slow deliberation" affects general, everyday capabilities.
Drawing inspiration from cognitive science (System 1 vs. System 2 thinking), the authors investigate whether forcing a model to over-deliberate harms its core "foundational capabilities"—specifically helpfulness, instruction-following, and safety—while inflating inference costs for simple queries.
Methods
The authors systematically evaluate three major LRM families (DeepSeek, Qwen, and LLaMA) across various parameter scales (7B to 70B/671B), comparing reasoning models (trained via distillation or reinforcement learning) against their standard chat-instruct counterparts.
To explore the impact of test-time compute, they introduce four Adaptive Thinking Modes by manipulating the placement of special tokens (like <think> and </think>):
- Zero-Thinking: Forcing the model to skip deliberation entirely by appending
</think>immediately after the prompt. - Less-Thinking: Prematurely cutting off the reasoning process at a specific percentage ($p\%$) of its typical length.
- Summary-Thinking: Generating the full thought trace, summarizing it using an external judge (GPT-4o), and feeding only the summary back to the model to generate the final answer.
- Summary-Thinking-Plus: Similar to Summary-Thinking, but preserving the critical first sentence of the original thought trace.
Datasets
The models were evaluated across two primary dimensions:
- Helpfulness & General Capabilities: MMLU-Pro (multi-task understanding), Live-Bench (excluding math/coding/reasoning to isolate general tasks), IFEval (strict instruction-following), and MT-Bench (multi-turn conversation).
- Harmlessness & Safety: StrongReject (vanilla harmful prompts) and WildJailbreak (adversarial jailbreak attacks).
The model mapping and training methods are summarized in the paper's overview table: Model Specifications
Key Insights and Ideas
- The Reasoning Tax: Distilling long chain-of-thought data into models causes them to lose track of explicit formatting constraints. For example, a model might perfectly plan a response in its thoughts but fail a strict formatting rule (like ending with a specific phrase) by adding a single extra word.
- Thoughts are Riskier than Answers: Internal reasoning traces are significantly more unsafe than the final outputs. Models frequently generate harmful, toxic, or compliant jailbreak logic inside their
<think>blocks, even if the final response attempts a refusal. - Over-Thinking Inefficiency: LRMs generate massive thought traces for simple general tasks, adding massive token overhead without any performance gain.
- Adaptive Compute is Essential: A fixed reasoning budget is suboptimal. General tasks benefit from shorter or summarized thoughts, whereas safety benchmarks drastically improve when thinking is bypassed entirely (Zero-Thinking).
Key Results
- Foundational Degradation: Distilled models showed severe drops in instruction following. For instance,
s1.1-32Bsuffered a 47.38% decrease on IFEval compared to its base modelQwen2.5-32B-Instruct, while increasing inference token costs by 250%. - Token Overhead: Thought lengths consistently dwarf response lengths across general benchmarks, representing a massive computational waste for non-reasoning tasks. Thought vs Response Lengths
- Summary-Thinking Boost: Compressing the thought trace restored performance. Summary-Thinking improved
s1.1-32BandQwQ-32Bon IFEval by 59.74% and 4.44%, respectively, by stripping out excessive self-reflection loops. - Safety Gains via Zero-Thinking: Bypassing the thinking process entirely made models highly resistant to jailbreaks. On the WildJailbreak benchmark,
QwQ-32B's safety score surged from 10.65 to 59.65 under Zero-Thinking mode.
The relationship between thinking ratio and performance across benchmarks highlights that full deliberation (Thinking Ratio = 1.0) is rarely optimal for general tasks: Less-Thinking Performance Analysis
Limitations and Future Work
The authors note several limitations:
- The evaluation is restricted to text-based benchmarks and does not cover multimodal interactions or highly open-ended dialogue.
- The analysis is limited to specific checkpoints of three model families and needs validation on other architectures like Mixture-of-Experts (MoE).
Future Directions: Future work should focus on developing dynamic, automated inference-time compute allocation methods (e.g., training models to decide when and how long to think based on query difficulty) and integrating safety penalties directly into RL reward frameworks to curb tedious or unsafe reasoning traces.
Summary updated