Understanding Reasoning in Chain-of-Thought from the Hopfieldian View

Lijie Hu, Liang Liu, Shu Yang, Xin Chen, Zhen Tan, Muhammad Asif Ali, Mengdi Li, Di Wang

Introduction

While Chain-of-Thought (CoT) prompting has significantly enhanced the reasoning capabilities of Large Language Models (LLMs), the underlying mechanisms remain poorly understood. Current research often treats CoT as a black-box engineering feat. This paper proposes a shift from the traditional "Sherringtonian" view—which focuses on individual neurons—to a Hopfieldian perspective. This view treats reasoning as the collective, dynamic movement of neural populations within a low-dimensional representation space.

Bridging Neuroscience and LLM Mechanics

The authors establish a formal mapping between cognitive neuroscience and LLM dynamics to explain how CoT functions. In this framework, CoT instructions (e.g., "Let's think step by step") act as external stimuli, while the generated tokens represent behavioral actions.

The Hopfieldian view maps cognitive brain dynamics to LLM representation spaces, treating reasoning as a trajectory influenced by instructions.

By analyzing the difference in hidden state activations between standard and CoT-enhanced prompts, the researchers identify low-dimensional manifolds using Principal Component Analysis (PCA). These manifolds represent the "reasoning direction" within the model's internal state. Reasoning is thus viewed as a trajectory through this space; if the model's state drifts out of the "reasoning manifold," it begins to hallucinate or lose logic.

Representation-of-Thought (RoT)

Building on this theoretical foundation, the authors introduce Representation-of-Thought (RoT). Unlike standard CoT, which relies purely on textual prompts, RoT actively "steers" the model by injecting the identified reasoning direction directly into the hidden states of the last few layers during inference. This method offers two primary advantages:

  1. Reasoning Error Localization: By monitoring the alignment of token activations with the reasoning subspace, the framework can pinpoint logical missteps (e.g., unit conversion errors) before the final answer is generated.
  2. Robustness: RoT filters out the noise associated with specific high-dimensional prompt templates, making the model more resilient to variations in phrasing or example ordering.

Experimental Evaluation

The framework was evaluated across three diverse reasoning domains—Arithmetic, Commonsense, and Symbolic—using benchmarks like GSM8K, SVAMP, and StrategyQA. The experiments primarily utilized Llama-2 and Llama-3 models.

Performance comparison of Base, CoT, and RoT methods across various reasoning benchmarks and model scales.

Key Results

  • Performance Gains: On Llama-3-8B-Instruct, RoT improved accuracy over the base model by 1.52% on GSM8K and 2.66% on SVAMP. Notably, it outperformed standard zero-shot CoT, which occasionally degraded performance on certain tasks (indicated by red values in the table).
  • Superior Robustness: RoT demonstrated significantly lower variance across different prompt styles. In few-shot scenarios where standard CoT is sensitive to example shuffling, RoT maintained consistent accuracy.
  • Interpretability: The error localization method provided a "heat map" of reasoning, successfully identifying specific logical flaws, such as failing to convert hours to minutes in a math problem.

Conclusion and Future Work

This research provides a robust theoretical framework for understanding CoT through the lens of population dynamics. While the current approach uses linear approximations (PCA), it opens the door for exploring more complex non-linear dynamics and "attractor" states in LLM reasoning. Future work will focus on dynamic layer selection and automated hyperparameter tuning to further refine the RoT framework.

Summary updated

Abstract

Large Language Models have demonstrated remarkable abilities across various tasks, with Chain-of-Thought (CoT) prompting emerging as a key technique to enhance reasoning capabilities. However, existing research primarily focuses on improving performance, lacking a comprehensive framework to explain and understand the fundamental factors behind CoT's success. To bridge this gap, we introduce a novel perspective grounded in the Hopfieldian view of cognition in cognitive neuroscience. We establish a connection between CoT reasoning and key cognitive elements such as stimuli, actions, neural populations, and representation spaces. From our view, we can understand the reasoning process as the movement between these representation spaces. Building on this insight, we develop a method for localizing reasoning errors in the response of CoTs. Moreover, we propose the Representation-of-Thought (RoT) framework, which leverages the robustness of low-dimensional representation spaces to enhance the robustness of the reasoning process in CoTs. Experimental results demonstrate that RoT improves the robustness and interpretability of CoT reasoning while offering fine-grained control over the reasoning process.
BibTeX
@misc{hu2024understanding,
  author = {Lijie Hu and Liang Liu and Shu Yang and Xin Chen and Zhen Tan and Muhammad Asif Ali and Mengdi Li and Di Wang},
  title = {{Understanding Reasoning in Chain-of-Thought from the Hopfieldian View}},
  year = {2024},
  eprint = {2410.03595},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.03595},
}
Figure