Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism

Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, Zhi-Qin John Xu

TL;DR

This paper investigates how large language models (LLMs) perform symbolic multi-step reasoning by introducing the concept of a "buffer mechanism." The authors propose that LLMs store intermediate reasoning results in distinct buffers and selectively retrieve them, offering a new perspective on their internal reasoning processes and leading to a method that significantly improves performance on multi-step reasoning tasks.

What Problem Does This Paper Address?

LLMs have shown impressive capabilities across various tasks, including logical reasoning. However, they still struggle with complex, multi-step reasoning problems, such as mathematical problem-solving. Understanding the internal mechanisms that enable LLMs to perform such reasoning is crucial for designing more effective models and training strategies. This paper aims to uncover how Transformer models process and integrate logical information across multiple layers to tackle multi-step reasoning, moving beyond simply stacking single-step reasoning capabilities.

Methods

The authors propose a buffer mechanism to explain how Transformer models handle multi-step reasoning. They hypothesize that information is stored in distinct "buffers" within the model's layers and accessed via query-key matrices. To enhance reasoning ability, they introduce a random matrix-based algorithm that modifies specific weight matrices within the Transformer architecture. This algorithm adds only 132 trainable parameters but aims to improve the model's capacity to manage and retrieve intermediate reasoning steps. They also explore two reasoning strategies: Vertical Thinking Strategy (VTS), where reasoning progresses sequentially through layers, and Horizontal Thinking Strategy (HTS), where reasoning is completed within a single layer (e.g., Chain-of-Thought).

Datasets

The paper utilizes several symbolic multi-step reasoning datasets for training and evaluation, including:

  • Clutrr
  • RuleTaker
  • StepGame
  • LogicInference
  • LogicAsker
  • PararulePlus
  • PrOntoQA

They also designed an abstract symbolic multi-step reasoning task with both in-distribution (ID) and out-of-distribution (OOD) data splits to test the model's generalization capabilities.

Key Insights and Ideas

  • Buffer Mechanism: The core idea is that LLMs don't just process information linearly; they store intermediate results in distinct "buffers" (represented by weight matrices) and selectively retrieve them. This is analogous to how a computer uses memory buffers.
  • "Same-Token Matching": The authors observe that the model effectively matches information within a token by attending to itself, facilitated by the buffer mechanism. This is crucial for consolidating diverse information.
  • HTS vs. VTS and Buffers: Horizontal Thinking Strategies (like CoT) seem to enhance reasoning by promoting buffer reuse, allowing models to perform multi-step reasoning without needing to allocate new buffers for each step.
  • OOD Generalization: The buffer mechanism, particularly the diagonal structure it induces in certain matrices, is linked to the model's ability to generalize to unseen reasoning patterns.
  • RMBA Algorithm: A simple algorithmic modification (adding random matrices) can significantly improve reasoning performance by enhancing the buffer mechanism, demonstrating that even small architectural changes can have a large impact.

Key Results

  • The proposed random matrix-based algorithm (RMBA) led to significant performance improvements on 7 multi-step reasoning datasets.
  • For example, on the LogicAsker, PararulePlus, and PrOntoQA datasets, RMBA achieved nearly 100% accuracy, matching the base model, but with a substantial reduction in training cost (e.g., over 65% fewer epochs required to reach 95% accuracy).
  • The study found that LLMs exhibit a "buffer mechanism" for symbolic multi-step reasoning, where information is stored and retrieved from distinct buffers.
  • Causal intervention experiments confirmed that specific buffers (e.g., $W_{v0}^{(l-1)}$ in layer $l$) are crucial for storing intermediate reasoning results, and modifying them directly impacts the output.
  • The research also showed that LLMs can perform multi-step reasoning beyond their training sequence length using techniques like Chain-of-Thought (CoT).

Limitations and Future Work

The authors acknowledge that their current work lacks in-depth theoretical analysis. For future work, they aim to conduct deeper theoretical modeling for multi-step reasoning problems and extend the buffer mechanism to other types of multi-step reasoning tasks. They also suggest that the buffer mechanism could be further explored in the context of other LLM architectures and reasoning paradigms.

Summary updated

Abstract

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy based on their inherent structure and horizontal thinking strategy based on Chain of Thought to achieve multi-step reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts them through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75\% reduction in the training time required for the GPT-2 model to achieve generalization capability on the PrOntoQA dataset. These findings provide new insights into understanding the mechanisms of large language models.
BibTeX
@inproceedings{wang2025understanding,
  author = {Zhiwei Wang and Yunji Wang and Zhongwang Zhang and Zhangchen Zhou and Hui Jin and Tianyang Hu and Jiacheng Sun and Zhenguo Li and Yaoyu Zhang and Zhi-Qin John Xu},
  title = {{Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure