TL;DR
This paper provides a rigorous mathematical proof for the "two-clustering" behavior in causal attention, demonstrating that tokens in autoregressive models naturally collapse into two opposing clusters aligned with the value matrix's dominant eigenvector. By introducing a physics-inspired "instability within the flow" framework, the authors explain how high-dimensional token trajectories simplify into a predictable two-dimensional structure, even when model parameters change over time.
What Problem Does This Paper Address?
While Transformers have revolutionized AI, our theoretical understanding of their internal "token dynamics"—how representations evolve across layers—is still catching up to their empirical success. Previous research into "full attention" (where every token sees every other token) often found that tokens converge toward a single cluster, a phenomenon linked to "oversmoothing" where models lose expressive power.
However, modern generative models (like GPT-4 or Llama) use causal attention, where tokens can only attend to their predecessors. This masking breaks the symmetry of full attention, making it much harder to analyze. A recent conjecture suggested that causal attention leads to a "two-clustering" regime, but a formal proof was missing, particularly for the realistic scenario where model weights might vary across layers (time-dependent parameters). This paper aims to bridge that gap, explaining why tokens don't just collapse into one point, but rather split into two.
Methods
The authors treat the Transformer as a dynamical system, viewing individual tokens as interacting particles moving through a high-dimensional embedding space. Their approach combines several novel techniques:
- Instability within the Flow: This is the paper's central analytical contribution. Inspired by stability theory in physics, the authors visualize token evolution as a small ball rolling on a "hill" (the attention landscape). Crucially, in deep models, this hill itself moves over time. They prove that a stationary hill forces tokens toward two extremal states, while a rapidly moving one can trap them in "interior" clusters.
- Hierarchical Suppression: They introduce the "error-to-traverse ratio" to show how the system progressively eliminates non-dominant directions, forcing the high-dimensional trajectory to collapse onto a lower-dimensional manifold.
- Dimensional Reduction: They prove that despite the complexity of the embedding space, token trajectories eventually settle into a two-dimensional coordinate plane governed by a few dominant directions.
Datasets
As a primarily theoretical work, the authors use a mix of synthetic simulations and established benchmarks to validate their proofs:
- Synthetic Simulations: Token trajectories were tested in 2D, 3D, 32D, and 128D spaces to visualize clustering behavior.
- WikiSQL and MultiNLI: The authors performed fine-tuning experiments on GPT-3 (175B) using LoRA. They compared the performance of fine-tuning different weight matrices ($Q, K, V$) to see if the theoretical importance of the Value matrix ($V$) held up in practice.
Key Insights and Ideas
- The $V$ Matrix is the "Boss": The two clusters align perfectly with the eigenvectors associated with the largest eigenvalue ($\lambda_{max}$) of the value transformation matrix. If $\lambda_{max} \le 0$, the two-cluster regime generally fails.
- The "Déjà Vu" Effect: The authors identify a recurrence phenomenon where, even as tokens stabilize, the final tokens are repeatedly forced to re-align with the dominant direction. This suggests that "past" dominant information is never truly forgotten in the infinite-depth limit.
- Rigidity in High Dimensions: One of the most surprising findings is that token dynamics are "rigid." Even in a 1024-dimensional space, the long-term behavior is effectively two-dimensional. This provides a theoretical justification for why low-rank adaptation (LoRA) is so effective.
Key Results
The paper successfully resolves the two-clustering conjecture for causal attention. Key findings include:
- Global Convergence: They prove that for almost every initial condition, tokens converge to the two clusters $\{\xi_{max}, -\xi_{max}\}$.
- GQA Validation: The theory holds up in Group Query Attention (GQA), a popular mechanism in modern LLMs like Llama 3. Figure 1 (below) shows tokens organizing into two opposing clusters on a circle after processing.
- Fine-tuning Superiority: In their GPT-3 experiments, fine-tuning combinations that included the $V$ matrix consistently outperformed those that only tuned $Q$ or $K$. For example, on WikiSQL, tuning $Q$ and $V$ reached 73.7% accuracy, compared to only 70.4% for $Q$ alone. This confirms that the $V$ matrix plays a critical role in shaping the attractors of token dynamics.
Limitations and Future Work
- Structural Assumptions: The proof for time-dependent parameters relies on a specific "block structure" assumption for the Query-Key matrix. While this covers common "weight-tying" scenarios, it may not perfectly describe every pre-trained Transformer in the wild.
- Symmetric $V$ Matrix: Much of the analysis assumes the value matrix $V$ is symmetric. Extending these proofs to non-symmetric matrices—which are common in practice—remains an open challenge.
- Multi-Head Interactions: The current work focuses primarily on single-head dynamics. Future research could explore how multiple attention heads interact to create even more complex clustering regimes.
Summary updated