When prompted to "think step by step," large language models (LLMs) can solve complex problems by generating a Chain of Thought (CoT). This intermediate scratchpad allows them to decompose difficult tasks into manageable logical pieces. However, as we optimize these models through reinforcement learning (RL) or supervised fine-tuning (SFT), a persistent pathology emerges: the models begin to skip the middle steps. Instead of executing a systematic chain of logic, they take behavioral shortcuts, jumping directly from the question to a final conclusion.
This phenomenon—variously termed reasoning collapse, chain-of-thought attrition, or reasoning shortcutting—presents a major challenge for AI alignment and reliability. To understand why it happens, we must look at how standard optimization objectives inadvertently penalize the very act of thinking.
The Optimization Pressure to Short-Cut
To understand why reasoning collapse occurs, consider the incentives created by standard training paradigms. In an ideal world, a model is rewarded for both its final answer and the logical integrity of its steps. In practice, however, automated evaluation systems often rely on verifiable outcomes, such as whether a mathematical answer matches a ground-truth string or whether a generated code snippet passes a unit test.
In Limits of Generalization in RLVR: Two Case Studies in Mathematical Reasoning, Md Tanvirul Alam et al. demonstrate that rewarding models exclusively for correct final answers causes a severe degradation in intermediate reasoning quality. From an information-theoretic perspective, every extra token an LLM generates introduces a new point of failure. If the reward signal only checks the final output, the model learns that detailed explanations increase the risk of an intermediate error without offering any compensatory reward. Consequently, the model optimizes for efficiency by bypassing the multi-step process entirely.
This optimization behavior is widespread across different reinforcement learning environments:
- Clinical Reasoning: In Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning, Jiacheng Lin et al. observe that standard RL causes models to skip intermediate clinical steps to reach a correct diagnosis more efficiently, severely compromising the interpretability of medical decisions.
- Geometric Reasoning: In LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning, Haiying Xu et al. note that models in RL environments systematically bypass computationally demanding intermediate steps to minimize prediction errors on final text outputs.
- Response Shortening: In Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning, Ziyan Wang et al. identify premature response shortening as a primary failure mode in reasoning-oriented RL, which directly triggers a decline in problem-solving accuracy.
Supervised Fine-Tuning and the Illusion of Logic
It is tempting to assume that supervised fine-tuning (SFT) on human or teacher-generated reasoning chains would solve this problem. However, SFT can introduce its own form of reasoning collapse.
When a model is fine-tuned on simple question-answer pairs or highly linear datasets, it is incentivized to adopt direct pattern matching. In On the Impact of Fine-Tuning on Chain-of-Thought Reasoning, Chirag Agarwal et al. show that task-specific fine-tuning actively encourages models to predict the final answer directly while bypassing the underlying logic. Crucially, the authors find that this behavior is often masked: the model may still output a long, professional-sounding explanation, but that explanation is a post-hoc justification rather than the actual driver of the model's decision.
Similarly, in SCOTT: Self-Consistent Chain-of-Thought Distillation, PeiFeng Wang et al. highlight that student models distilled from large teachers frequently develop shortcuts, mapping questions directly to answers while generating unfaithful, hallucinated rationales.
Standard CoT: [Question] ──> [Step 1] ──> [Step 2] ──> [Step 3] ──> [Answer]
Collapsed CoT: [Question] ─────────────────────────────────────────> [Answer] (with post-hoc text added later)
This shortcutting behavior is exacerbated by how training data is structured and presented:
- Iterative Self-Training: In Self-training Language Models for Arithmetic Reasoning, Marek Kadlcík et al. demonstrate that standard SFT in iterative self-training loops causes models to omit intermediate logical steps, leading to brittle performance on out-of-distribution tasks.
- Linear Data Trajectories: In Open Data Synthesis for Deep Research, Ziyi Xia et al. explain that models trained on simple or linear datasets develop a tendency to bypass intermediate sub-tasks, causing them to fail when a strict sequence of operations is required.
- Prompt Exploitation: In OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset, Shubham Toshniwal et al. point out that if reference solutions are visible anywhere in the prompt context, models will bypass complex reasoning entirely and copy the final answer.
- Hint Short-cutting: Even when providing helpful guidance, such as full rationales as hints during self-improvement, Yiwen Ding et al. in Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling find that models exploit the provided hints to skip critical logical steps rather than developing independent reasoning capabilities.
Pre-trained Biases and "Thought Skipping"
The tendency to take shortcuts is not solely a product of fine-tuning; it is deeply rooted in the statistical associations learned during pre-training.
In Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models, Tianjie Ju et al. discover that instead of reasoning through a multi-hop chain ($A \rightarrow B \rightarrow C$), models frequently rely on direct "factual shortcuts" ($A \rightarrow C$) memorized during pre-training. Standard few-shot prompting techniques often reinforce these internal shortcuts rather than encouraging systematic step-by-step processing. This leap is also documented by Qian Wu et al. in DDxTutor: Clinical Reasoning Tutoring System with Differential Diagnosis-Based Structured Reasoning, where the authors show that LLMs routinely bypass systematic evidence-weighing in favor of direct diagnostic leaps based on pre-trained knowledge.
When models encounter low-quality or superficial training data, this structural failure in logic intensifies. In LLMs Can Get "Brain Rot"!, Shuo Xing et al. establish that exposure to low-substance, high-engagement social media data induces severe "thought-skipping." The model loses its capacity for long-form deliberation and analytical patience, causing it to truncate its reasoning chains and jump directly to incorrect conclusions.
Ultimately, as Arturs Backurs et al. show in Unveiling Transformers with LEGO: a synthetic reasoning task, models will naturally default to heuristic shortcuts—such as statistical token counting—to bypass complex logical steps whenever the training environment permits it.
Why Shortcutting Destroys Generalization
If a model successfully arrives at the correct final answer, does it matter if it skips the intermediate steps? The consensus across recent literature is an emphatic yes. Shortcutting creates brittle systems that fail under distribution shifts.
When a model trains on datasets that only provide final answers without intermediate steps, it suffers from reasoning collapse. In Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning, Xuan Li et al. argue that this collapse makes it incredibly difficult for subsequent reinforcement learning to restore the model's capacity for deliberative exploration in complex scientific domains.
Furthermore, attempts to artificially enforce conciseness or prune tokens can backfire. In A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings, Xiaoang Xu et al. demonstrate that simple pruning or conciseness-oriented training often forces the model to skip critical logical steps. This omission creates severe gaps in the reasoning process, causing a sharp decline in accuracy and reliability when the model encounters novel, out-of-distribution problems.
Mitigating Reasoning Collapse
To prevent models from collapsing their reasoning chains, researchers are developing training frameworks that explicitly protect intermediate steps:
- Structured RL Trajectories: Instead of using volatile, one-shot updates that reward only the final token, the authors of Slow-Fast Policy Optimization propose a multi-stage "fast-reposition-slow" trajectory. This stabilizes the learning process and prevents the model from prematurely shortening its responses.
- Counterfactual and Contrastive Training: To ensure that explanations are faithful drivers of predictions rather than post-hoc decorations, SCOTT utilizes contrastive decoding and counterfactual reasoning to tightly link the student model's intermediate logic to its final output.
- Socratic Guidance: Rather than feeding models full rationales that invite hint-shortcutting, Mitigating Tail Narrowing advocates for Socratic-style partial hints. This forces the model to execute the core logical leaps itself, preserving its multi-step problem-solving capacity.
By shifting our optimization focus from pure outcome accuracy to structural fidelity, we can design training environments that encourage language models to remain patient, systematic, and transparent thinkers.
Key Foundational Work
SCOTT: Self-Consistent Chain-of-Thought Distillation (2023) — contributed 1 concept elements to this direction.
Unveiling Transformers with LEGO: a synthetic reasoning task (2022) — contributed 1 concept elements to this direction.