Limitations of Language Models in Arithmetic and Symbolic Induction

Jing Qian, Hong Wang, Zekun Li, Shiyang Li, Xifeng Yan

Abstract

Large Language Models (LLMs) demonstrate remarkable capabilities in natural language tasks but frequently fail at basic symbolic induction, such as multi-digit addition or string copying. This paper investigates the "generalization gap" where models fail to perform on inputs longer than those seen during training. We identify a fundamental "locating" failure in self-attention mechanisms, particularly in the presence of repeating symbols. We evaluate several mitigation strategies, including positional markers and scratchpads, and introduce LM with Tutor, a method that mimics algorithmic execution to achieve 100% accuracy on out-of-distribution symbolic tasks.

Introduction

Despite their scale, models like GPT-3 struggle with simple arithmetic when the problem size exceeds training distributions. This "Attention Deficit" is characterized by two primary hurdles: Out-of-Distribution (OOD) generalization and the difficulty of handling repeating symbols. Transformers rely on content-based self-attention, which struggles to distinguish between identical tokens (e.g., multiple '1's in a sequence) as sequence length grows. This leads to "locating" errors where the model loses track of its position within the input.

Performance degradation of standard LLMs on arithmetic and copying tasks as sequence length and symbol repetition increase.

Methodology

The study utilizes synthetic datasets for three core symbolic induction tasks: copying, reversing, and addition. Models (GPT-3, T5, and DeBERTa) were trained on short sequences (1–5 digits) and tested on OOD sequences (up to 80 digits for copying and 30 for addition).

Several mitigation strategies were tested:

  • Positional Markers: Explicit labels (e.g., A 1 B 1) to help track token positions.
  • Fine-grained Computation Steps: Breaking problems into minimal units (e.g., column-by-column addition).
  • Callable Programs: Allowing the model to offload sub-tasks to external functions (e.g., a Python add() function).
  • LM with Tutor: A novel approach where the model is taught the physical process of solving a problem. It mimics a multi-tape Turing machine by generating action sequences (e.g., "move cursor," "copy digit") rather than just intermediate results. This simplifies the hypothesis space by teaching the algorithm of movement.

Results and Discussion

Baseline models show a sharp decline in accuracy to near 0% as soon as input length exceeds the training window or symbol repetition increases. While "Scratchpad" prompting provides some benefit, it fails to solve the OOD problem fundamentally. Even with disentangled attention mechanisms, models struggle with absolute "pointing" when the sequence length grows beyond the training window.

Comparative accuracy of various mitigation strategies on addition tasks, highlighting the robustness of the LM with Tutor approach.

The LM with Tutor method achieved 100% accuracy across all tasks, including OOD data. By transforming reasoning into a sequence-following task guided by a "tutor" that pinpoints exact digit origins, the model learns a robust algorithm invariant to input length. Callable programs also improved performance but remained susceptible to locating errors when selecting which digits to pass to the function.

Conclusion

The failure of LLMs in basic math is rooted in the limitations of self-attention for absolute pointing in long or repetitive sequences. The "Tutor" method effectively bypasses these limitations by teaching algorithmic movement and copying. However, this approach requires highly detailed demonstrations and results in long sequence traces, posing challenges for context window limits and scaling to more abstract reasoning tasks where algorithms are less linear.

Summary updated

Abstract

Recent work has shown that large pretrained Language Models (LMs) can not only perform remarkably well on a range of Natural Language Processing (NLP) tasks but also start improving on reasoning tasks such as arithmetic induction, symbolic manipulation, and commonsense reasoning with increasing size of models. However, it is still unclear what the underlying capabilities of these LMs are. Surprisingly, we find that these models have limitations on certain basic symbolic manipulation tasks such as copy, reverse, and addition. When the total number of symbols or repeating symbols increases, the model performance drops quickly. We investigate the potential causes behind this phenomenon and examine a set of possible methods, including explicit positional markers, fine-grained computation steps, and LMs with callable programs. Experimental results show that none of these techniques can solve the simplest addition induction problem completely. In the end, we introduce LMs with tutor, which demonstrates every single step of teaching. LMs with tutor is able to deliver 100% accuracy in situations of OOD and repeating symbols, shedding new insights on the boundary of large LMs in induction.
BibTeX
@inproceedings{qian2023limitations,
  author = {Jing Qian and Hong Wang and Zekun Li and Shiyang Li and Xifeng Yan},
  title = {{Limitations of Language Models in Arithmetic and Symbolic Induction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=sctKg1VD4S},
}
Figure