Optimal inference of a generalised Potts model by single-layer transformers with factored attention

Riccardo Rende, Federica Gerace, Alessandro Laio, Sebastian GoldtarXiv 2023 · arXiv · OpenReview

Abstract

Transformers are the type of neural networks that has revolutionised natural language processing and protein science. Their key building block is a mechanism called self-attention which is trained to predict missing words in sentences. Despite the practical success of transformers in applications it remains unclear what self-attention learns from data, and how. Here, we give a precise analytical and numerical characterisation of transformers trained on data drawn from a generalised Potts model with interactions between sites and Potts colours. While an off-the-shelf transformer requires several layers to learn this distribution, we show analytically that a single layer of self-attention with a small modification can learn the Potts model exactly in the limit of infinite sampling. We show that this modified self-attention, that we call ``factored'', has the same functional form as the conditional probability of a Potts spin given the other spins, compute its generalisation error using the replica method from statistical physics, and derive an exact mapping to pseudo-likelihood methods for solving the inverse Ising and Potts problem.

Introduction

Despite the massive success of Transformers in natural language processing and protein folding, a fundamental understanding of what these models learn remains elusive. Most theoretical work treats neural networks as black boxes, yet Transformers possess a specific architecture—self-attention—trained with a specific objective—Masked Language Modeling (MLM). This paper addresses which probability distributions self-attention can learn efficiently, the sample complexity required for generalization, and the interpretability of learned weights.

Methods

The researchers utilize tools from the statistical physics of learning to analyze a simplified Transformer. They treat "sentences" as sequences of interacting "spins" (tokens/colors) following a Generalized Potts Model. In this model, tokens interact based on both their position in the sequence and their semantic content (color).

A key contribution is the introduction of Factored Self-Attention, a variant where the treatment of positions and embeddings is decoupled. In "vanilla" attention, these are often mixed; the authors show that separating them allows the model to perfectly mimic the conditional probabilities of the Potts model.

Comparison of Vanilla and Factored Attention architectures and their mapping to the Potts model.

The study proves that training a factored self-attention layer via MLM is mathematically identical to solving the "inverse Potts problem" using the pseudo-likelihood method, a well-known technique in physics for inferring interactions from observed data. To calculate generalization error analytically, they employ the "replica method" from the physics of disordered systems, allowing for exact calculations in the high-dimensional limit.

Datasets

The study primarily uses synthetic datasets generated via Gibbs sampling from a generalized Potts Hamiltonian. This provides a "ground truth" interaction matrix ($J$) and color similarity matrix ($U$). Typical setups involved a vocabulary size ($C$) of 20, sequence lengths ($L$) of 20, and varying numbers of training samples ($M$) up to 3,000.

Results

The most significant finding is that the attention matrix in a Transformer is not merely a heuristic weighting—it is a direct estimator of the physical interaction matrix between sites in a system. Factored attention allows a single layer to achieve optimal generalization, whereas a standard 1-layer Transformer fails to reach this limit.

Performance comparison and interaction matrix reconstruction across different model architectures.

Key results include:

  • Perfect Reconstruction: A single layer of factored self-attention recovers the original interaction matrix $J$ perfectly as training progresses, as seen in the comparison between ground truth and the factored model's output.
  • The Power of Decoupling: While a standard 3-layer Transformer can eventually learn the same structure, it remains a "black box," whereas the factored single layer remains perfectly interpretable.
  • Interpolation Peak: The authors confirmed a "peak" in test loss when the number of samples ($M$) equals the sequence length ($L$), marking the threshold where the model begins to overfit perfectly.
  • Statistical Consistency: MLM is shown to be statistically consistent; with sufficient data, the Transformer converges to the exact parameters of the data-generating process.

Discussion and Limitations

While the mapping provides deep insight, it currently only covers two-body interactions. Real-world language involves higher-order dependencies that likely require deeper architectures. Furthermore, the analytical replica analysis assumes specific matrix properties that may be more complex in biological or linguistic data. Future work will focus on the learning dynamics and extending the framework to higher-order interactions.

Summary updated

Abstract

Transformers are the type of neural networks that has revolutionised natural language processing and protein science. Their key building block is a mechanism called self-attention which is trained to predict missing words in sentences. Despite the practical success of transformers in applications it remains unclear what self-attention learns from data, and how. Here, we give a precise analytical and numerical characterisation of transformers trained on data drawn from a generalised Potts model with interactions between sites and Potts colours. While an off-the-shelf transformer requires several layers to learn this distribution, we show analytically that a single layer of self-attention with a small modification can learn the Potts model exactly in the limit of infinite sampling. We show that this modified self-attention, that we call ``factored'', has the same functional form as the conditional probability of a Potts spin given the other spins, compute its generalisation error using the replica method from statistical physics, and derive an exact mapping to pseudo-likelihood methods for solving the inverse Ising and Potts problem.
BibTeX
@misc{rende2023optimal,
  author = {Riccardo Rende and Federica Gerace and Alessandro Laio and Sebastian Goldt},
  title = {{Optimal inference of a generalised Potts model by single-layer transformers with factored attention}},
  year = {2023},
  eprint = {2304.07235},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.07235},
}
Figure