The modern AI paradigm operates under a powerful assumption: to make a model smarter, you must force it to deliberate. Post-training techniques like Chain-of-Thought (CoT) prompting, supervised fine-tuning on long reasoning traces, and test-time compute scaling have unlocked state-of-the-art problem-solving capabilities.
However, a growing body of research reveals a critical counter-phenomenon: reasoning-induced performance degradation. When highly capable models are forced to engage in explicit, multi-step deliberation for simple, factual, or perceptual tasks, their accuracy frequently plummets compared to direct, zero-shot responses. As [Ryan Liu] et al. demonstrate in Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse, this behavior closely mirrors human cognitive psychology. In domains like implicit statistical learning, mandating internal deliberation introduces unnecessary complexity, turning a straightforward task into a significant performance liability.
Why Deliberation Fails on Simple Tasks
To understand why explicit reasoning hurts performance on low-complexity tasks, we must look at how autoregressive language models generate text. When a model is forced to output a reasoning chain before arriving at a final answer, it changes the statistical path to the solution. This introduces three primary failure modes:
1. Textual Complexity as a Distractor
When a model generates an intermediate rationale, it produces a dense sequence of tokens that it must subsequently condition upon to generate the final answer. [Yidan Zhang] et al. show in Rationales for Answers to Simple Math Word Problems Confuse Large Language Models that this textual complexity acts as a distractor. If a model lacks the capacity to perfectly verify each logical transition it just wrote, the sheer volume of intermediate text introduces confusing elements, degrading its ability to isolate the correct final outcome.
2. Instruction Over-Decomposition
Highly capable models often exhibit a "complexity bias," assuming that a task requires sophisticated planning when a direct action would suffice. In NatureGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory Dataset, [Zihan Zheng] et al. identify an "over-planning paradox." Advanced models frequently decompose simple, one-step instructions into redundant or excessively verbose sub-actions, leading to execution inefficiencies and compounding errors in low-complexity scenarios.
3. Error Propagation and Circular Logic
Forcing extended reasoning paths on simple tasks frequently introduces redundant steps and circular logic. As [Wenkai Yang] et al. observe in Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning, this unnecessary complexity increases the frequency of erroneous intermediate steps. These errors pollute the context window, distracting the model from a straightforward solution and causing its internal logic to collapse.
Standard Prompting:
Input ──> [Direct Retrieval / Perception] ──> Correct Answer
Chain-of-Thought Prompting on Simple Tasks:
Input ──> [Unnecessary Sub-steps] ──> [Erroneous Token / Hallucination] ──> [Circular Logic] ──> Incorrect Answer
Domain-Specific Vulnerabilities
The overthinking penalty is highly dependent on task difficulty and domain type, manifesting prominently in three core areas:
- Visual and Spatial Perception: In Vision-Language Models (VLMs), forcing structured reasoning across all difficulty levels can actively damage basic visual understanding. [Jierun Chen] et al. explore this in The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs, finding that supervised fine-tuning on long reasoning traces induces excessive verbosity that degrades accuracy on simple perception tasks. Similarly, [Dongzhi Jiang] et al. show in MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency that explicit step-by-step reasoning introduces hallucinations or contradictions that override the model's initial, correct visual interpretation.
- Factual Retrieval and Simple Classification: For tasks that primarily require direct mapping or factual retrieval, intermediate rationales act as a penalty. In Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability, [Benfeng Xu] et al. demonstrate that training models to generate intermediate steps is counterproductive for low-complexity tasks like sentiment analysis or paraphrase detection, where the extra tokens distract from the correct label.
- Expert Guidelines: Even in specialized domains, over-analysis can backfire. [Yu-Min Tseng] et al. find in Evaluating Large Language Models as Expert Annotators that applying CoT to expert-level annotation tasks causes models to over-analyze or misinterpret highly specific domain guidelines rather than following them strictly.
Adaptive Compute: Learning When Not to Think
To resolve this tension, recent work focuses on teaching models when to deploy their reasoning capabilities. Rather than treating CoT as a default strategy, systems must adaptively scale their test-time compute based on problem difficulty.
For instance, [Zhuosheng Zhang] et al. propose the SelF-Reasoner framework in Mitigating Misleading Chain-of-Thought Reasoning with Selective Filtering, which uses a dedicated filter to assess the validity of a reasoning chain and bypasses CoT entirely if the logic appears shaky. To systematically evaluate this trade-off, [Seungone Kim] introduced OptimalThinkingBench: Evaluating Over and Underthinking in LLMs, a unified benchmark designed to measure overthinking and encourage the development of models that dynamically allocate their internal reasoning budgets. By balancing direct execution with structured deliberation, future architectures can avoid the performance tax of overthinking while preserving deep reasoning for the tasks that truly require it.
Key Foundational Work
CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks (2023) — contributed 3 concept elements to this direction.