Transformer Normalisation Layers and the Independence of Semantic Subspaces

Stephen Menary, Samuel Kaski, Andre FreitasarXiv 2024 · DOI · arXiv

Abstract

Modern Transformer architectures predominantly utilize Pre-Norm, where normalization occurs before attention heads. This study reveals that Pre-Norm imposes a restrictive geometric structure—orthogonal spheres—on internal semantic subspaces to prevent interference. We identify a phenomenon termed "circuit collapse," where attention mechanisms fail due to subspace coupling. While we propose QKV-Norm as a theoretically more flexible alternative that allows for linear independence, empirical results on synthetic tasks suggest that the constraints of Pre-Norm may provide a beneficial inductive bias for generalization.

Introduction: The Challenge of Semantic Interference

Transformers are increasingly analyzed through the lens of "circuits"—computational paths that perform specific logical operations. These circuits operate within semantic subspaces: independent slices of the high-dimensional latent space encoding distinct concepts such as token type or position.

A fundamental conflict arises in the standard Pre-Norm architecture. Because the $L_2$-normalization factor is calculated across the entire vector, changes in one subspace (e.g., position) inherently affect the magnitude of others (e.g., semantic content). This coupling creates interference that can destabilize the logical gates essential for Transformer operations.

Methods and Theoretical Modeling

The researchers investigated this interference through a combination of theoretical proofs and empirical probing. They modeled the propagation of information through "Match & pass" mechanisms, where tokens are matched based on specific features and information is passed into new subspaces to build complex representations.

The 'Match & pass' mechanism illustrating how semantic information is embedded and processed through subspaces.

To address the limitations of Pre-Norm, the authors proposed QKV-Norm. In this variant, normalization is applied after the linear projections ($Q, K, V$) within the attention head. This shift theoretically decouples the subspaces, allowing them to remain merely linearly independent rather than strictly orthogonal spheres.

Experimental Setup

The study utilized a synthetic base-10 integer addition task (e.g., [453+16+17-N846=1332]). This controlled environment allowed for precise modulation of task complexity and the measurement of "In-Distribution" versus "Extrapolation OOD" (out-of-distribution) performance, emphasizing multi-step contextual reasoning and "match & pass" logic.

Key Insights and Theoretical Results

The core theoretical finding is that for Pre-Norm to maintain independent subspaces, the model must converge on a specific geometric structure: every subspace must be a sphere, and all spheres must be mutually orthogonal.

Comparison of representation structures and degrees of freedom across different normalization strategies.

As shown in the table above, Pre-Norm and Pre-RMSNorm force representations into orthogonal spheres, reducing the attendable degrees of freedom (d.o.f.) compared to No-Norm or QKV-Norm, which allow for linearly independent subspaces.

Circuit Collapse

The authors identified a phase transition in attention stability. While sparse attention (where a head focuses heavily on one token) is resilient to minor noise, exceeding a specific threshold causes "circuit collapse." In this state, the attention distribution shifts abruptly and incorrectly, leading to catastrophic failure in logical tasks. This collapse is modeled as a consequence of random noise on the $L_2$-norms of vectors propagating through the attention mechanism.

Key Results and Performance Trade-offs

Empirical validation confirmed that trained Pre-Norm models adhere strictly to the "constant-norm" structure, with 90% of embedding norms falling within a narrow $\pm 20\%$ interval. QKV-Norm models, by contrast, exhibited a significantly wider spread.

However, a notable trade-off emerged:

  • Sensitivity: Pre-Norm models are surprisingly fragile; adding just 1% artificial noise to the norms resulted in a 10% accuracy drop.
  • Generalization: Despite the theoretical flexibility of QKV-Norm, Pre-Norm performed significantly better on extrapolation tasks (66.7% vs 46.8% accuracy). This suggests that the "orthogonal spheres" constraint acts as a powerful regularizer, guiding the model toward more robust and generalizable solutions.

Conclusion and Future Work

This work highlights a hidden geometric cost of the Pre-Norm architecture. While it enforces a restrictive representation structure that is sensitive to norm perturbations, this very restriction appears to facilitate better out-of-distribution generalization in mathematical reasoning tasks.

Future research should investigate whether these findings scale to massive language models and explore whether "polysemanticity"—where single neurons represent multiple concepts—serves as a strategy models use to navigate these normalization-induced constraints.

Summary updated

BibTeX
@misc{menary2024transformer,
  author = {Stephen Menary and Samuel Kaski and Andre Freitas},
  title = {{Transformer Normalisation Layers and the Independence of Semantic Subspaces}},
  year = {2024},
  eprint = {2406.17837},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.17837},
}
Figure