Self-attention Networks Localize When QK-eigenspectrum Concentrates

Han Bao, Ryuichiro Hataya, Ryo Karakida

Abstract

This paper provides a mathematical bridge between the way Transformers focus on specific tokens ("localization") and their overall performance. By analyzing the eigenspectrum (the distribution of eigenvalues) of the Query-Key (QK) parameter matrices, the authors show that concentrating these eigenvalues around a non-zero mean prevents two major training failure modes—rank collapse and entropy collapse—simultaneously.

Introduction: The Dual Collapses of Attention

In the world of Transformers, "attention" is the mechanism that allows the model to decide which parts of an input sequence are important. Ideally, a model should localize: it should pick out a few highly relevant tokens rather than treating every token the same (uniform attention).

However, researchers have long struggled with two seemingly contradictory failure modes:

  1. Rank Collapse: When attention is too uniform, all token embeddings become nearly identical, making the network "shallow" and less expressive.
  2. Entropy Collapse: When attention becomes extremely peaked (low entropy), the training process often hits a "plateau" where the loss stops improving, leading to poor trainability.

Previously, these were studied as separate issues. This paper asks: When exactly does attention localize, and can we find a mathematical condition that prevents both of these collapses at once?

Methods and Theoretical Framework

The authors tackle this using a mix of theoretical derivation and a new regularization technique:

  • Softmax Approximation: To make the math tractable, they use a piecewise linear approximation of the softmax function. This allows them to calculate "Gaussian moments" (mean and variance) of the attention scores.
Comparison of the standard softmax function and its piecewise linear approximation.
  • Signal Propagation Probability ($\rho_i$): Instead of just looking at attention weights, the authors define a metric that measures how likely a specific input token's signal is to propagate through to the gradient during training.
Signal propagation probability curves across relative token positions under various parameter settings.
  • Random Walk Data Model: They analyze the system under the assumption that input tokens follow a "random walk" in the embedding space, which is a semantically plausible way to model how topics evolve in a sentence.
  • LOCATER (LOCalized ATtEntion Regularization): Based on their findings, they propose a new regularization term. It encourages the QK-matrix to have a small eigenspectrum variance (concentration) while keeping the mean of the eigenvalues at a fixed, non-zero level.

Key Insights and Results

  • The Eigenspectrum is the Key: The "localization" of a Transformer isn't just a random byproduct of training; it is directly controlled by the eigenvalues of the $W_{QK}$ matrix (scaled by the data's covariance).
  • Concentration is the Solution: The "aha moment" is that minimizing the variance of the eigenspectrum while keeping a non-zero mean is the "sweet spot."
  • Reconciling the Collapses: The authors prove that this specific condition (small eigenspectrum variance) is a unified solution. It keeps the rank high (preventing rank collapse) and keeps the attention entropy from dropping too low (preventing entropy collapse/training plateaus).
  • Middle-Site Focus: Their math shows that as the eigenspectrum concentrates, the model naturally tends to focus on tokens in the "middle" of the context window, which helps in balancing early and late information.

Empirical Validation

The paper validates its theories using synthetic data (generated Gaussian random walks) to verify the theoretical "signal propagation" curves, and the WikiText-2 dataset to test 1-layer, 3-layer, and 6-layer Transformer architectures.

  • Theoretical Proof: The authors derived a closed-form approximation showing that localization occurs when the ratio of the trace of the QK-matrix to the trace of its square ($tr(W) / \sqrt{tr(W^2)}$) is large.
  • Improved Perplexity: On the WikiText-2 dataset, models trained with the LOCATER regularization achieved significantly better (lower) perplexity compared to standard training.
  • Stability: LOCATER successfully maintained higher attention entropy throughout training. This prevented the "loss plateaus" that usually plague models when their attention becomes too peaked.

Limitations and Future Work

  • Random Walk Assumption: While semantically grounded, real-world language is far more complex than a Gaussian random walk. The authors acknowledge that their model doesn't perfectly align with real data distributions.
  • Depth of Analysis: The formal mathematical analysis is primarily restricted to 1-layer Transformers. While empirical results on 6-layer models look promising, the theoretical extension to very deep networks (where gradient explosion/vanishing is a factor) remains for future work.
  • The "Why" of Localization: While the paper explains how to achieve localization and that it helps performance, the fundamental reason why localized attention is inherently better for language tasks (beyond avoiding collapse) is still an open research question.

Summary updated

Abstract

The self-attention mechanism prevails in modern machine learning. It has an interesting functionality of adaptively selecting tokens from an input sequence by modulating the degree of attention localization, which many researchers speculate is the basis of the powerful model performance but complicates the underlying mechanism of the learning dynamics. In recent years, mainly two arguments have connected attention localization to the model performances. One is the rank collapse, where the embedded tokens by a self-attention block become very similar across different tokens, leading to a less expressive network. The other is the entropy collapse, where the attention probability approaches non-uniform and entails low entropy, making the learning dynamics more likely to be trapped in plateaus. These two failure modes may apparently contradict each other because the rank and entropy collapses are relevant to uniform and non-uniform attention, respectively. To this end, we characterize the notion of attention localization by the eigenspectrum of query-key parameter matrices and reveal that a small eigenspectrum variance leads attention to be localized. Interestingly, the small eigenspectrum variance prevents both rank and entropy collapse, leading to better model expressivity and trainability.
BibTeX
@inproceedings{bao2024self,
  author = {Han Bao and Ryuichiro Hataya and Ryo Karakida},
  title = {{Self-attention Networks Localize When QK-eigenspectrum Concentrates}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=aRZjRj41WQ},
}
Figure