The Inverted U-Curve of Chain of Thought Reasoning

This direction is merged into another direction: optimal reasoning depth. Its last version stays readable and citable here.

Version 1 of 1123 papers178 conceptsfrozen 2026-10-04

In the evolution of Large Language Models (LLMs), the prevailing dogma has been that "thinking longer" leads to "thinking better." The success of models like OpenAI o1 and DeepSeek-R1, which use extensive Chain-of-Thought (CoT) traces to solve complex problems, reinforced the idea that scaling inference-time compute is a reliable path to higher accuracy.

However, a growing body of research suggests that this relationship is not linear. Instead, LLM performance typically follows an inverted U-shaped curve relative to reasoning length. Beyond a certain "Goldilocks zone," additional reasoning steps introduce noise, logical drift, and "overthinking," which can actively degrade the model's performance.

The Mechanics of Overthinking

Why does more computation sometimes lead to worse results? The core issue is the accumulation of stochastic noise. As Yifei Wang and Yuyang Wu demonstrate in When More is Less: Understanding Chain-of-Thought Length in LLMs, while initial decomposition helps a model tackle high-entropy tasks, every additional step in a reasoning chain is an opportunity for a minor error to propagate.

This phenomenon is particularly visible in inductive reasoning—inferring rules from examples. Haibo Jin and colleagues found that for these tasks, CoT can actually be counterproductive. In Evaluating the Inductive Abilities of Large Language Models, they show that extended reasoning steps amplify hallucinated logic, causing "Large Reasoning Models" to underperform standard LLMs that provide direct answers.

Theoretically, this is framed by Yong Liu in CoT-Space as a trade-off between underfitting (thinking too little to capture task complexity) and overfitting (thinking so much that the model captures noise and irrelevant patterns).

Finding the Optimal Reasoning Depth

If more is not always better, how much reasoning is enough? Research indicates that the "optimal" length is a function of both task difficulty and model capability.

  1. The Simplicity Bias: Counter-intuitively, more powerful models often require shorter reasoning chains to reach peak accuracy. Yuyang Wu observed that high-parameter models exhibit a "simplicity bias," where they achieve their best results with dense, efficient logic rather than verbose exploration.
  2. Mathematical Scaling: Amrut Nadgir and Pratik Chaudhari derived a mathematical scaling law for reasoning depth. They argue there is a "sweet spot" for the number of sub-problems a task should be broken into. If the steps are too large, the model fails the logic; if the steps are too many and too small, the cumulative error of the sequence overwhelms the result.
  3. Diminishing Returns: In tasks like causal inquiry, WHY-Chain researchers found that a depth of roughly three "Why?" questions provides the best balance. Beyond this, the complexity of the reasoning chain introduces noise that degrades logical consistency.

Strategies for Length Calibration

To combat "overthinking" and reduce computational waste (which can be as high as 80% in verbose models), researchers are developing methods to force models toward the efficiency frontier.

Reinforcing Brevity

The ShorterBetter framework, introduced by Sida Li and Jingyang Yi, uses reinforcement learning to reward the shortest correct response among multiple samples. This pushes the model to find the minimum necessary reasoning path, slashing token usage by 50–80% while maintaining accuracy. Similarly, Ruosen Li proposed AALC (Adaptive Accuracy-Length Control), which dynamically balances correctness and brevity to identify the inflection point where further length reduction would cause accuracy to collapse.

Structural Optimization

Instead of just shortening the output, some methods focus on aligning the reasoning structure with the problem's actual complexity. Runquan Gui developed CoSMo, which uses a split-merge algorithm to ensure the number of reasoning segments matches the logical hops required by the task. This prevents "structural redundancy" where a model generates five steps for a two-step problem.

Predictive Stopping

Can a model know when to stop thinking? Aditya Golatkar introduced Re-FORC, which trains a lightweight adapter to predict the "marginal benefit" of the next reasoning token. If the expected reward for further thinking is low, the system terminates the sequence early.

Measuring Effort Beyond Token Count

A critical realization in this field is that token count is a poor proxy for reasoning quality. A model can generate thousands of tokens of "repetitive thinking" without making logical progress.

To address this, Wei-Lin Chen and colleagues developed the Deep-Thinking Ratio (DTR). Instead of counting words, DTR measures how much a model "revises" its internal predictions across its layers. This metric provides a much more robust signal for correctness than raw length, allowing for inference strategies like Think@n that achieve state-of-the-art performance with half the compute.

By shifting the focus from "thinking long" to "thinking deep," these researchers are moving toward a more efficient and reliable generation of AI reasoning—one where the model knows exactly how much deliberation a problem deserves.

Papers in this version

Showing 60 of 123 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_the_inverted_u_curve_of_chain_of_thought_02e8f184,
  title = {The Inverted U-Curve of Chain of Thought Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-inverted-u-curve-of-chain-of-thought-reasoning/rdr_02e8f1849e8f4ceea6837d538aa016d2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_the_inverted_u_curve_of_chain_of_thought_02e8f184_v1,
  title = {The Inverted U-Curve of Chain of Thought Reasoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-inverted-u-curve-of-chain-of-thought-reasoning/rdr_02e8f1849e8f4ceea6837d538aa016d2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}