Self-attention Networks Localize When QK-eigenspectrum Concentrates

Han Bao, Ryuichiro Hataya, Ryo Karakida

Abstract

The self-attention mechanism prevails in modern machine learning. It has an interesting functionality of adaptively selecting tokens from an input sequence by modulating the degree of attention localization, which many researchers speculate is the basis of the powerful model performance but complicates the underlying mechanism of the learning dynamics. In recent years, mainly two arguments have connected attention localization to the model performances. One is the rank collapse, where the embedded tokens by a self-attention block become very similar across different tokens, leading to a less expressive network. The other is the entropy collapse, where the attention probability approaches non-uniform and entails low entropy, making the learning dynamics more likely to be trapped in plateaus. These two failure modes may apparently contradict each other because the rank and entropy collapses are relevant to uniform and non-uniform attention, respectively. To this end, we characterize the notion of attention localization by the eigenspectrum of query-key parameter matrices and reveal that a small eigenspectrum variance leads attention to be localized. Interestingly, the small eigenspectrum variance prevents both rank and entropy collapse, leading to better model expressivity and trainability.

Introduction

Transformers have revolutionized artificial intelligence, yet the internal mechanics of self-attention remain partially obscured. A critical area of study is "attention localization"—the ability of a model to focus on specific tokens. Historically, researchers have identified two failure modes related to this:

  1. Rank Collapse: Token embeddings become nearly identical, leading to uniform attention and a loss of expressive power.
  2. Entropy Collapse: Attention becomes excessively "peaked" on a single token, resulting in low entropy that can stall the learning process.

These phenomena seem paradoxical: one involves too much uniformity, the other too much focus. This research reconciles these views by analyzing the eigenspectrum (the set of eigenvalues) of the Query-Key (QK) parameter matrices, identifying the specific conditions under which attention localizes effectively without triggering collapse.

Figure 5: Attention entropy lower bound vs. $\nu$, illustrating how entropy can collapse as the product of weight and input norms increases.

Methodology

The study employs a combination of theoretical derivation and empirical validation:

  • Signal Propagation Probability ($\rho_i$): A novel metric introduced to quantify the likelihood of an input token's signal reaching the gradient during training. High $\rho_i$ for a subset of tokens indicates localization.
  • Piecewise Linear Approximation: To handle the non-linearity of the softmax function, the authors utilize a linear approximation to derive tractable theoretical bounds.
  • Eigenspectrum Analysis: The focus is on the mean ($\xi$) and variance ($\eta$) of the QK matrix eigenvalues.
  • LocAteR (LOCalized ATtEntion Regularization): A proposed regularization technique that constrains the QK matrix to maintain a specific eigenvalue mean while minimizing variance (scale).

Experiments were conducted on synthetic Gaussian Random Walk data to verify lemmas and on the WikiText-2 dataset to evaluate real-world language modeling performance.

Key Insights: The "Goldilocks Zone"

The central finding is that a small eigenspectrum variance is essential for healthy attention. It serves as a bridge, preventing rank collapse by maintaining a non-zero mean and avoiding entropy collapse by keeping the eigenvalue scale small.

Figure 2: Signal propagation probability $\rho(\theta)$ for various eigenspectrum means ($\xi$) and scales ($\eta$). Localization increases with $\xi$, while excessive variance ($\eta$) causes the signal to vanish.

As shown in Figure 2, localization is a spectrum. When the eigenspectrum concentrates around a non-zero mean with low variance, the signal propagation probability becomes sharply peaked, indicating effective localization. If the variance is too high, the signal degenerates. Furthermore, the theory suggests that higher embedding dimensions ($d$) facilitate this beneficial localization.

Results and Performance

Theoretical proofs established the conditions for localization based on the product of eigenvalue mean and scale. Empirically, the LocAteR regularization consistently improved perplexity on WikiText-2 across 1-layer, 3-layer, and 6-layer Transformer architectures compared to standard training baselines. The regularization encourages the formation of stable attention patterns, focusing on informative tokens throughout the training process, whereas standard models often remain noisy or uniform.

Conclusion and Future Work

This work provides a unified framework for understanding attention localization through eigenspectrum concentration. While the Gaussian random walk assumption simplifies the complex correlations of natural language, it provides a rigorous foundation for future study. Future research is needed to extend this analysis to much deeper networks where gradient dynamics are more complex and to further explore the fundamental link between localization and superior language modeling performance.

Summary updated

Abstract

The self-attention mechanism prevails in modern machine learning. It has an interesting functionality of adaptively selecting tokens from an input sequence by modulating the degree of attention localization, which many researchers speculate is the basis of the powerful model performance but complicates the underlying mechanism of the learning dynamics. In recent years, mainly two arguments have connected attention localization to the model performances. One is the rank collapse, where the embedded tokens by a self-attention block become very similar across different tokens, leading to a less expressive network. The other is the entropy collapse, where the attention probability approaches non-uniform and entails low entropy, making the learning dynamics more likely to be trapped in plateaus. These two failure modes may apparently contradict each other because the rank and entropy collapses are relevant to uniform and non-uniform attention, respectively. To this end, we characterize the notion of attention localization by the eigenspectrum of query-key parameter matrices and reveal that a small eigenspectrum variance leads attention to be localized. Interestingly, the small eigenspectrum variance prevents both rank and entropy collapse, leading to better model expressivity and trainability.
BibTeX
@inproceedings{bao2024self,
  author = {Han Bao and Ryuichiro Hataya and Ryo Karakida},
  title = {{Self-attention Networks Localize When QK-eigenspectrum Concentrates}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=aRZjRj41WQ},
}
Figure