The Buffer Mechanism for Multi-Step Information Reasoning in Language Models

Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, Zhi-Qin John XuSubmission to ICLR 2025 · Rejected

TL;DR

This paper reveals that Transformers execute multi-step reasoning via a buffer mechanism, where intermediate conclusions are stored in distinct subspaces defined by value-output projection matrices ($W^{vo}$) and retrieved by query-key attention heads ($W^{qk}$). This perspective explains why Chain-of-Thought (CoT) is so effective—it allows models to repeatedly reuse a small set of buffers across generated tokens rather than burning through network layers. Exploiting this insight, the authors introduce a Random Matrix-Based Algorithm (RMBA) that accelerates generalization on the PrOntoQA benchmark by 75%.

What Problem Does This Paper Address?

While Large Language Models (LLMs) solve complex problems when prompted with step-by-step scratchpads like Chain-of-Thought (horizontal thinking), their ability to solve multi-step logic in a single forward pass (vertical thinking) degrades rapidly—often collapsing after just three steps.

Prior mechanistic interpretability work identified single-step primitives (such as induction heads), but multi-step reasoning is not a simple linear cascade of single steps. How Transformers store, isolate, and route intermediate logical deductions across layers without causing cross-talk has remained an open question. Understanding this internal circuit is essential for designing architectures and initialization strategies that learn logical deduction efficiently.

Methods

  • The Buffer Mechanism: The authors formalize the intuition that attention value-output projections ($W^{vo(l)} = W^{v(l)} W^{o(l)}$) act as a collection of nearly orthogonal buffers. Intermediate results are mapped into separate subspaces ($[a]W^{vo(l)}$), while query-key heads ($W^{qk(l)} = W^{q(l)} W^{k(l)\top}$) serve as "extractors." Reasoning occurs when an extractor aligns with a buffer such that their product forms a diagonal kernel ($Ker^{(l)} \approx I$), triggering "same-token matching" to link an intermediate deduction to the premise of the next step.
  • Buffer-Level Causal Intervention: Rather than perturbing entire token representations, the authors surgically swapped values within individual buffer subspaces across layers to confirm where intermediate reasoning states reside.
  • Analytical Weight Construction: Using the buffer formulation, the authors proved and demonstrated that an $L$-layer Transformer can execute $(L-1)$-step reasoning zero-shot without any training, merely by setting $W^{qk}$ and $W^{vo}$ to predetermined permutation and random matrices.
  • Random Matrix-Based Algorithm (RMBA): To improve data efficiency during training, prior work proposed adding identity matrices to attention projections (Identity Matrix-Based Algorithm, or IMBA). However, the authors show that in multi-step reasoning, adding identity matrices creates severe informational interference. Instead, RMBA regularizes the weights by adding fixed Gaussian random matrices: $$W^{qk(l)} + \alpha^{(l)} Z^{(l-1)}, \quad W^{vo(l)} + \beta^{(l)} Z^{(l)}$$ where $\alpha, \beta$ are learnable scalars and $Z \sim \mathcal{N}(0, 1/d_m)$. This injects fresh, mutually orthogonal buffer subspaces from the start of training.

Datasets

  1. Synthetic Symbolic Multi-Step Reasoning Dataset: An abstract benchmark serialized as token pairs representing directed reasoning edges (e.g., $A \to B, B \to C$). The authors created distinct in-distribution (ID) and out-of-distribution (OOD) splits using modular arithmetic splits, forcing the model to perform in-context reasoning over unseen relation pairs.
  2. PrOntoQA: A 0.5M synthetic question-answering dataset evaluating 3-step first-order logic deduction, formatted as natural language statements concluding in True/False questions.
  3. LLM Benchmarking: Tested closed and open models (GPT-4, GPT-4o, Claude 3.5 Sonnet/Haiku, and Phi-3) on multi-step reasoning chains.

Key Insights and Ideas

  • Why CoT is Computationally Superior (Buffer Reuse): In a single forward pass (vertical thinking), each step consumes layer depth because the model must allocate a new, orthogonal buffer ($W^{vo(l)}$) to avoid overwriting earlier deductions. In contrast, horizontal thinking (CoT) outputs each intermediate deduction as a new text token. This resets the deduction into the base residual stream buffer ($I$), allowing a tiny 2-layer Transformer to solve arbitrarily long reasoning chains by repeatedly cycling through the exact same two buffers.

  • Why Identity Regularization Fails for Multi-Step: Adding identity matrices ($W^{vo} + \beta I$) causes representation leakage between premises and deductions ($[a]W^{vo} + \beta [a] + [b]$). Adding independent random matrices (RMBA) preserves subspace orthogonality, preventing catastrophic feature interference.

  • Evidence in Production LLMs: Inspecting pre-trained Phi-3 reveals that $W^{qk}$ and $W^{vo}$ matrices in middle layers (layers 5–20) naturally align into diagonal kernel structures, and their flattened $W^{vo}$ matrices are pairwise orthogonal, confirming the buffer mechanism emerges in real-world models.

Key Results

  • Sample Efficiency: On PrOntoQA, training a 12-layer GPT-2 small with RMBA reduced the number of iterations required to achieve out-of-distribution generalization by ~75% (generalization began around 0.25 epochs, compared to ~1.0 epoch for baseline GPT-2).
  • Distractor Robustness: When random distractor tokens were inserted into test inputs, RMBA retained baseline-level accuracy (>85%), whereas IMBA dropped steeply to ~25–35%.
  • Zero-Shot Analytical Construction: An untrained 8-layer Transformer constructed using random buffer weights achieved 100% accuracy on 7-step reasoning tasks across natural, reversed, and scrambled premise orders.
  • CoT Scaling: A minimal 2-layer model trained exclusively on 1-step sequences of length 13 generalized to 6-step reasoning chains via autoregressive feedback with >57.6% accuracy.

Limitations and Future Work

  • Simplified Assumptions: The analytical derivation relies on the assumption that hidden dimension $d_m$ is sufficiently large for near-perfect random orthogonality, whereas real-world models operate under tight dimension bottlenecks.
  • Feedforward Network (FFN) Omission: The core theoretical formulation simplifies attention by abstracting away non-linear feedforward layers, although empirical evaluations included standard Transformer blocks.
  • Domain Scope: Experiments primarily focus on formal symbolic chains and synthetic logical deduction (PrOntoQA); verifying whether buffer mechanisms govern fuzzy, semantic, or multi-hop natural language tasks remains for future work.

Summary updated

Abstract

Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these models can help us design better model architectures and training strategies, ultimately enhancing their reasoning capability. In this study, we constructed a symbolic dataset to investigate the mechanisms by which Transformer models employ vertical thinking strategy based on their inherent structure and horizontal thinking strategy based on Chain of Thought to achieve multi-step reasoning. We introduced the concept of buffer mechanism: the model stores various information in distinct buffers and selectively extracts them through the query-key matrix. We proposed a random matrix-based algorithm to enhance the model's reasoning ability, resulting in a 75\% reduction in the training time required for the GPT-2 model to achieve generalization capability on the PrOntoQA dataset. These findings provide new insights into understanding the mechanisms of large language models.
BibTeX
@inproceedings{wang2025buffer,
  author = {Zhiwei Wang and Yunji Wang and Zhongwang Zhang and Zhangchen Zhou and Hui Jin and Tianyang Hu and Jiacheng Sun and Zhenguo Li and Yaoyu Zhang and Zhi-Qin John Xu},
  title = {{The Buffer Mechanism for Multi-Step Information Reasoning in Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}

Loading related content...

Figure