Abstract
Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy based on their inherent structure and horizontal thinking strategy based on Chain of Thought to achieve multi-step reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts them through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75% reduction in the training time required for the GPT-2 model to achieve generalization capability on the PrOntoQA dataset. These findings provide new insights into understanding the mechanisms of large language models.
Introduction: The Reasoning Gap in LLMs
Large Language Models (LLMs) are surprisingly adept at many tasks, but they often stumble when required to perform multi-step logical reasoning in a single forward pass—a capability termed Vertical Thinking Strategy (VTS). While techniques like Chain-of-Thought (CoT) facilitate reasoning by allowing the model to "think out loud" (Horizontal Thinking Strategy or HTS), this approach is computationally expensive. The core challenge lies in understanding how a Transformer passes a logical result from one layer to the next without losing or mixing information.
Investigating Reasoning via Symbolic Tasks
To isolate the mechanics of multi-step logic, the researchers utilized a custom-built symbolic reasoning task. This task requires models to derive conclusions from a series of rules (e.g., $[a] \to [b]$).
As shown in the accuracy table above, even advanced models like GPT-4 and Claude 3.5 Sonnet struggle significantly as the number of reasoning steps increases. While Claude 3.5 Sonnet achieves 100% accuracy on 2-step reasoning, its performance drops to just 12% for 4-step reasoning, highlighting the inherent difficulty of vertical multi-step logic.
The Buffer Mechanism: VTS vs. HTS
The study introduces the Buffer Mechanism theory to explain internal information propagation. In this framework, the model stores intermediate reasoning steps in distinct subspaces of its hidden layers (buffers) and uses the attention mechanism as an "extractor" to pull information from these buffers.
Figure 6 illustrates the difference between the two strategies. In VTS (a), the model must manage multiple distinct buffers across different layers simultaneously to solve a problem in one forward pass. This requires precise "same-token matching" where the model attends to its own previous layer's output for the same token. In HTS (b), the model can reuse the same buffer for every step by generating a new token, making the reasoning process easier for the architecture to learn but more time-consuming to execute.
Random Matrix-Based Algorithm (RMBA)
To bridge the reasoning gap, the authors proposed the Random Matrix-Based Algorithm (RMBA). Instead of requiring the model to learn complex buffer alignments from scratch, RMBA adds a small set of trainable parameters (only 132) that nudge the model toward an efficient buffer-matching structure using fixed random matrices.
Key Results and Insights
- Performance Boost: On the RuleTaker dataset, the RMBA-enhanced model achieved 83.5% accuracy, compared to the baseline's 67.0%.
- Efficiency Gains: RMBA significantly reduced training costs. For datasets like PrOntoQA, the training time required to reach generalization dropped by over 65%.
- Orthogonal Memory: Successful models were found to maintain nearly orthogonal (mathematically independent) internal buffers, preventing "information interference" between reasoning steps.
Conclusion
This research provides a mechanistic look at how Transformers handle logic. By identifying the "same-token matching" operation as a critical bottleneck for vertical thinking, the authors have developed a method to significantly improve reasoning efficiency with minimal parameter overhead. Future work will explore how this buffer framework scales to non-symbolic and more creative reasoning tasks.
Summary updated