TL;DR
Large Language Models often "know" the correct answer internally long before they finish generating a lengthy Chain-of-Thought (CoT). By identifying a specific "task-solving direction" within the model's hidden states, researchers can steer a direct-answer prompt to achieve higher accuracy than CoT while using 90% fewer tokens.
What Problem Does This Paper Address?
Chain-of-Thought (CoT) prompting is the gold standard for complex reasoning in LLMs, but it has two major drawbacks: it is computationally expensive (generating hundreds of tokens) and it isn't foolproof—sometimes the model reasons its way to a wrong answer.
This paper investigates a fascinating discrepancy: the information needed to solve a task is often present in the model's internal hidden states even when the generated text is incorrect. The authors aim to understand if these "useful" internal states can be systematically identified and used to skip the long-winded reasoning chain entirely, moving straight to a correct direct answer.
Methods
The researchers employ a technique called activation patching to probe the model's internal representations.
- Labeling States: They run a model with a CoT prompt (the "source") and cache all hidden states. They then run the same question with a direct-answer prompt (the "target") and replace its final hidden state with one from the CoT run. If this "patch" causes the direct run to produce the correct answer, that CoT state is labeled recovering; otherwise, it is failed.
- Linear Probing: They train linear classifiers (probes) to see if these two classes of states are linearly separable in the model's representation space.
- Finding the Direction: For each question, they calculate a task-solving direction ($d$). This is essentially the vector pointing from the average of the "failed" states toward the average of the "recovering" states ($d \propto c^+ - \lambda c^-$).
- Spherical Intervention: To test this direction, they take a fresh direct-answer run and rotate its final hidden state toward the task-solving direction using Spherical Linear Interpolation (SLERP). This changes the direction of the state while preserving its original norm (magnitude).
Datasets
The authors evaluated their approach across 25 tasks spanning six reasoning types: arithmetic, logical, temporal, spatial, causal, and linguistic.
- GSM8K: Grade school math word problems.
- StrategyQA: Questions requiring multi-step implicit reasoning.
- BIG-Bench Hard (BBH): 23 subtasks including Boolean Expressions, Logical Deduction, and Geometric Shapes.
- Models: Evaluations were performed on LLaMA 3.1 8B and Qwen 2.5 7B.
Key Insights and Ideas
- Correctness is Linearly Decodable: The fact that a linear probe can distinguish "recovering" from "failed" states (with a mean AUROC of ~0.84) suggests that the model's internal "knowledge" of the correct path is structured and accessible, even if the output decoder misses it.
- Directions are Question-Specific: A major finding is that there is no single "reasoning direction" for an entire task. The task-solving direction is specific to each individual question. Averaging these directions across a task results in a vector that fails to improve performance.
- Generalization Across Reasoning Types: Probes trained on one task can often identify recovering states in other tasks, especially those of the same reasoning type (e.g., logical tasks generalize to other logical tasks). This suggests the model uses similar internal "circuits" for related types of problems.
Key Results
The intervention—rotating a single hidden state—produced dramatic improvements over both direct prompting and standard CoT:
- Accuracy Boost: On LLaMA 3.1 8B, accuracy jumped from 46% (direct) to 85% (steered), significantly outperforming CoT (69%). Qwen 2.5 7B saw a similar rise from 52% to 86% (CoT: 77%).
- Extreme Token Efficiency: The steered models reached these correct answers using an average of only 19 tokens, compared to the 197–245 tokens typically generated during a full CoT process.
- Fixing "Broken" Reasoning: The method successfully recovered correct answers for 64-70% of the questions where both the direct prompt and the CoT prompt originally failed.
Limitations and Future Work
The primary limitation is that this method currently requires generating the CoT chain first to "find" the task-solving direction, making it an analysis tool rather than a drop-in replacement for inference.
However, the authors have created a valuable dataset of question-direction pairs. They suggest that future work could involve training a separate, smaller model (a regressor) to predict this task-solving direction directly from the input prompt. If successful, this would allow models to achieve CoT-level reasoning (or better) at the speed and cost of direct answering.
Summary updated