Abstract
Chain-of-thought (CoT) prompting enhances reasoning in large language models (LLMs) but often leads to verbose and redundant outputs, thus increasing inference cost. We hypothesize that many reasoning steps are unnecessary for producing correct answers. To investigate this, we start with a systematic study to examine what is the minimum reasoning required for a model to reach a stable decision. We find that on math reasoning tasks like math, models typically converge to their final answers after 60% of the reasoning steps, suggesting substantial redundancy in the remaining content. Based on these insights, we propose three inference-time strategies to improve efficiency: (1) early stopping via answer consistency, (2) boosting the probability of generating end-of-reasoning signals, and (3) a supervised method that learns when to stop based on internal activations. Experiments across five benchmarks and five open-weights LLMs show that our methods significantly reduce token usage with little or no accuracy drop. In particular, on NaturalQuestions, Answer Consistency reduces tokens by over 40% while further improving accuracy. Our work underscores the importance of cost-effective reasoning methods that operate at inference time, offering practical benefits for real-world applications.
Introduction
While Chain-of-Thought (CoT) prompting has significantly boosted the reasoning capabilities of Large Language Models (LLMs), it comes with a literal cost: high inference latency and token usage. Models like DeepSeek-R1 or QwQ often produce massive "thinking" blocks. This paper identifies an "overthinking phenomenon" where models continue to generate reasoning steps even after their predicted answer has stabilized. This redundancy doesn't just waste computational resources; in some cases, it actually hurts performance, as the model might "talk itself out" of a correct initial intuition through unnecessary self-reflection or revision.
The Concept of Answer Convergence
The researchers first conducted a systematic study to define the Answer Convergence Ratio (ACR)—the proportion of the reasoning chain required before the final answer remains unchanged. They found that for many tasks, models converge after only 60% of their total reasoning steps.
Proposed Inference-Time Strategies
To capitalize on the observed redundancy, three strategies were proposed:
- Answer Consistency (Unsupervised): The model's CoT is split into sentence-level chunks. After each chunk, the model is prompted to provide a "greedy" answer. If the answer remains identical for $k$ consecutive chunks, the process terminates.
- Think Token Adjustment (Unsupervised): This method applies a linear logit transformation to the
</think>token during decoding. By boosting the probability of this "end-of-reasoning" signal based on the "peakedness" of the model's output distribution, the model is encouraged to stop when confident. - Learn-to-Stop (Supervised): A lightweight LSTM classifier is trained on the LLM’s internal hidden activations ($h_t$) to predict the optimal stopping point. This self-supervised approach uses the model’s own final converged answer as the training target.
Experimental Results
The methods were evaluated across five benchmarks: NaturalQuestions (NQ), GSM8K, MATH-500, GPQA-Diamond, and AIME’24.
Key Insights
- Significant Token Savings: On NaturalQuestions, token usage was reduced by over 40% across multiple models. On GSM8K, the "Learn-to-Stop" method reduced tokens by 44.5% with only a negligible 0.2% drop in accuracy.
- Accuracy Gains: On the NQ dataset, the "Learn-to-Stop" method actually improved accuracy (e.g., from 41.6% to 43.0% on QwQ-32B), suggesting that extra reasoning steps can sometimes introduce noise.
- Model Scaling: Larger models (like Llama-70B) tend to stabilize on answers earlier in their reasoning chain compared to smaller models, indicating more efficient internal processing.
Conclusion and Future Work
This work demonstrates that answer convergence is a powerful signal for early stopping in LLM reasoning. While a stable answer is not always correct—particularly in high-complexity tasks like AIME—the proposed methods offer a practical way to reduce inference costs. Future research will explore hybrid strategies that combine answer stability with "faithfulness-aware" signals to ensure truncated reasoning remains logically sound.
Summary updated