Chain-of-Thought Tokens are Computer Program Variables

Fangwei Zhu, Peiyi Wang, Zhifang SuiarXiv 2025 · arXiv · OpenReview

TL;DR

This paper investigates the "black box" of Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) by treating CoT tokens as computer program variables. They find that intermediate results, rather than the specific wording, are crucial, and these can even be represented in latent forms. Interventions reveal a causal link between CoT values and the final output, akin to variable manipulation in code, though LLMs can exhibit "shortcut" behaviors.

What Problem Does This Paper Address?

Chain-of-Thought (CoT) prompting has revolutionized LLMs' ability to tackle complex reasoning tasks, from math problems to code generation. However, the underlying mechanism of how CoT works and the precise role of the generated intermediate "thought" tokens remain largely mysterious. This lack of understanding hinders our ability to optimize CoT, design more efficient reasoning strategies, and fully trust LLM outputs. This paper aims to demystify CoT by empirically investigating the function of its tokens, specifically hypothesizing that they act like variables in a computer program.

Methods

The authors employ a series of empirical studies to probe the nature of CoT tokens. Their core methodology involves:

  • Task-Specific Analysis: They focus on two compositional tasks requiring serial computations: multi-digit multiplication and dynamic programming (specifically, maximum path sum in a grid).
  • Token Manipulation: They experiment with removing non-result tokens from CoT, compressing intermediate results into latent tokens (using one-hot vectors), and intervening (modifying) values within CoT sequences.
  • Model Architecture Augmentation: For latent token experiments, they introduce input projection modules and latent output heads to allow the model to process and predict these latent representations.
  • Intervention Experiments: They randomly alter intermediate values within CoT sequences and observe the impact on the final output, treating this as a form of debugging or variable manipulation.
  • Probing Classifiers: They train linear probes on LLM hidden states to understand how intermediate variable information is represented across different layers.

Datasets

The paper uses synthetic datasets generated for the specific tasks of multi-digit multiplication and dynamic programming. For each task, they generate 100,000 data entries, split into training and testing sets with a 90%/10% ratio. The specific details of prompt and dataset construction are provided in the appendices.

Key Insights and Ideas

  • Intermediate Results are Key: The most surprising finding is that preserving only the tokens containing intermediate numerical results (and not descriptive text) in CoT achieves comparable, and sometimes even better, performance than full CoT. This suggests that the semantic "language" of the thought process is less critical than the actual values being computed.
  • CoT Tokens as Program Variables: Intervening in CoT values (e.g., changing a carry value) directly and predictably alters subsequent steps and the final answer. This behavior strongly mirrors how modifying a variable in a computer program affects its execution.
  • Latent Representation is Viable: Intermediate results can be effectively represented in alternative, compressed forms, such as latent tokens. This indicates that the LLM doesn't necessarily need to "understand" the number in a human-readable format; a compressed representation is sufficient for its reasoning process.
  • Computational Complexity Limits: While latent representations work well, there's a limit. Aggressively compressing intermediate values leads to performance degradation, especially in dynamic programming tasks with larger numbers. This suggests a computational complexity ceiling for the LLM's internal processing of these "variables."
  • LLMs Can Take Shortcuts: In simpler subproblems, LLMs may learn "shortcuts" where certain intermediate variables are effectively ignored, leading to unfaithful reasoning paths and failed interventions. This highlights a potential brittleness in their reasoning process.

Key Results

  • Removing non-result tokens from CoT leads to comparable performance, and in some cases, slight improvements, compared to full CoT.
  • Representing intermediate results using latent tokens (with a dimension of 20 for multiplication and 50 for DP) does not significantly harm performance on multiplication tasks and even surpasses full CoT on 5x5 problems. However, it leads to a ~9% accuracy drop on 4x5 DP problems.
  • Intervention experiments show a high success rate (e.g., 73.83% success in multiplication, as per Table 1), demonstrating the causal link between CoT values and the final output.
  • Error analysis in interventions reveals that "shortcut error" is a significant issue (1291 instances in multiplication), indicating LLMs may bypass intermediate steps.
  • Probing experiments show that information about intermediate variables is primarily accessible in later layers of the LLM, regardless of overall accuracy.

Limitations and Future Work

  • Theoretical Proof: The paper relies on empirical evidence to support the "CoT tokens as program variables" hypothesis, lacking a formal theoretical proof.
  • Synthetic Tasks: Experiments are conducted on two synthetic tasks (multi-digit multiplication and DP) using a specific model (Qwen-2.5-1.5B). Generalizing these findings to real-world datasets (like GSM8K or Math) and other models requires further investigation.
  • Future Work: The authors suggest exploring:
  • Generalization of CoT "programs" to a wider range of tasks and identifying meta-programs.
  • Developing methods to identify intermediate variable tokens in more general tasks.
  • Estimating the computational complexity limits between variable tokens more precisely.
  • Investigating alternative CoT forms and designing more efficient CoTs by compressing simple reasoning steps and reserving capacity for complex ones.

Summary updated

Abstract

Chain-of-thoughts (CoT) requires large language models (LLMs) to generate intermediate steps before reaching the final answer, and has been proven effective to help LLMs solve complex reasoning tasks. However, the inner mechanism of CoT still remains largely unclear. In this paper, we empirically study the role of CoT tokens in LLMs on two compositional tasks: multi-digit multiplication and dynamic programming. While CoT is essential for solving these problems, we find that preserving only tokens that store intermediate results would achieve comparable performance. Furthermore, we observe that storing intermediate results in an alternative latent form will not affect model performance. We also randomly intervene some values in CoT, and notice that subsequent CoT tokens and the final answer would change correspondingly. These findings suggest that CoT tokens may function like variables in computer programs but with potential drawbacks like unintended shortcuts and computational complexity limits between tokens. The code and data are available at https://github.com/solitaryzero/CoTs_are_Variables.
BibTeX
@misc{zhu2025chain,
  author = {Fangwei Zhu and Peiyi Wang and Zhifang Sui},
  title = {{Chain-of-Thought Tokens are Computer Program Variables}},
  year = {2025},
  eprint = {2505.04955},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.04955},
}
Figure