Abstract
Chain-of-Thought (CoT) reasoning is a pivotal technique for enhancing the performance of Large Language Models (LLMs) on complex tasks. However, the internal mechanisms that allow simple prompts like "let’s think step by step" to unlock these capabilities remain poorly understood. This paper introduces an explainable framework grounded in the "Hopfieldian view" from cognitive neuroscience, which treats cognition as transformations within representational spaces. We propose a "Read-and-Control" approach that treats prompts as stimuli to activate latent concepts. Our framework enables the localization of reasoning errors and the mathematical steering of models toward correct reasoning paths, demonstrating significant accuracy improvements across multiple reasoning benchmarks.
Introduction: The Mystery of Chain-of-Thought
While Chain-of-Thought (CoT) prompting has dramatically improved LLM performance, the field lacks a fundamental, top-down explanation for how these prompts alter a model's internal state. Most research focuses on empirical trial-and-error or external knowledge retrieval. This paper addresses two core mysteries:
- Zero-shot CoT: Why does a simple instruction like "let’s think step by step" fundamentally change the model's output?
- Few-shot CoT: Why do a few preceding examples drastically improve the model's logical consistency?
The Hopfieldian Framework and Read-and-Control
The authors build their framework on the Hopfieldian view, shifting the focus from individual neurons to the movement of representations through high-dimensional space. Their "Read-and-Control" approach treats LLM prompts as biological-style stimuli that shift the model's internal geometry into a more logical "action space."
The framework consists of three main components:
- Concept Modeling & Simulation: Identifying latent concepts (e.g., "careful reasoning") learned during pre-training that are activated by specific stimuli.
- Representations Reading: Using Linear Artificial Tomography (LAT) and Principal Component Analysis (PCA) to extract a "reading vector" that represents the direction of a specific concept within the model's hidden layers.
- Representations Controlling: Once a reading vector is identified, it can be mathematically injected back into the model's hidden states during inference. This "nudges" the model toward a more accurate reasoning path without needing to change the input prompt.
Key Results and Interpretability
The framework was evaluated across seven datasets covering arithmetic (GSM8K, SVAMP, AQuA), commonsense (StrategyQA), and symbolic reasoning. The "Read-and-Control" approach consistently outperformed standard baselines on models like Mistral-7B and LLaMA-3-8B, with accuracy gains of up to 3.81% on AQuA.
Reasoning Error Localization
A major contribution of this work is the ability to pinpoint exactly where a model’s reasoning goes wrong. By comparing the model's hidden states against the reading vector, the framework generates a "salience map."
As shown in the visualization, green and red bars indicate which parts of a reasoning chain are logically sound or flawed. This allows for "rectifying" reasoning paths—for instance, correcting a failed comparison between the depth of Mount Fuji and the Sea of Japan by applying the control vector.
Overcoming the "Stereotype" Trap
The authors discovered that few-shot examples can create "stereotypes," where the model forces a problem into a learned pattern even if the facts do not fit. The "Control" method proved capable of breaking these biases, demonstrating that the underlying logic is often already present in the model but requires precise activation.
Conclusion
By bridging neuroscience and AI, this paper provides a new lens for understanding LLM behavior. The results suggest that reasoning capabilities are latent within pre-trained models; CoT prompts are merely one way to access them. The "Read-and-Control" framework offers a more direct, mathematical method to steer these models toward logical accuracy. Future work will explore extending this approach to multi-modal data and automating the discovery of concept stimuli.
Summary updated