What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information

Yiting Chen, Junchi YanNeurIPS 2024 · OpenReview

Researchers have discovered that the geometric relationship between weight pairs in Rotary Position Embeddings (RoPE) acts as a "syntax vs. semantics" switch within Large Language Models (LLMs). By identifying and freezing weights that handle basic syntax while only training those that handle high-level semantics, the authors introduce Angle-based Weight Selection (AWS)—a method that significantly reduces fine-tuning overhead while often boosting model performance.

The Black Box of Position and Meaning

While Transformer-based LLMs are incredibly powerful, their internal mechanics often remain opaque. We know they use position encodings like RoPE to understand the sequential order of words, but it has been unclear how the model's internal weights interact with these encodings to prioritize either the position of a word (syntax) or its meaning (semantics).

Furthermore, fine-tuning these massive models is computationally expensive. Current Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA help reduce the burden, but they do not distinguish between weights that are already "good enough" at basic language structure and those that actually need to adapt to new, task-specific concepts. This paper bridges the gap between architectural theory and fine-tuning efficiency.

The Mechanics of RoPE and Weight Angles

To understand how LLMs separate syntax from semantics, the authors dive deep into the mechanics of Rotary Position Embedding (RoPE). RoPE works by dividing query and key vectors into 2D pairs and rotating them to inject positional information.

The researchers theoretically demonstrate that the angle ($\alpha$) between the weight vectors generating these pairs determines how the model behaves:

  1. Non-orthogonal pairs (high cosine similarity): These pairs are less sensitive to the actual text input and focus heavily on specific relative positions. This behavior corresponds to basic syntax (e.g., looking for a preposition a few tokens back).
  2. Orthogonal pairs (near-zero cosine similarity): These pairs are highly flexible and sensitive to the input content, corresponding to high-level semantic processing.
Illustration of RoPE and the impact of the angle between weight vectors.

Key Insights: Geometry is Destiny

By analyzing these angles across different layers of the network, the authors uncovered several fascinating properties of LLMs:

  • Geometry Dictates Function: The angle between weight pairs is a reliable indicator of a neuron's "job." Large absolute cosine similarity values essentially lock the attention to certain positions regardless of the specific word, which is exactly what is needed for structural syntax.
  • The Layer-wise Pipeline: LLMs follow a "syntax-first, semantics-later" approach. Shallow layers are packed with non-orthogonal (syntactic) pairs, while deep layers are dominated by orthogonal (semantic) pairs.
  • Fine-tuning is for Meaning: The most striking discovery was that fine-tuning barely changes the "syntactic" weights. The pre-trained model already understands grammar; fine-tuning is almost entirely about shifting the "semantic" weights to handle new task-specific meanings.
Attention maps comparing heads with the largest and smallest average cosine similarity, demonstrating the shift from rigid syntactic attention to flexible semantic attention.

Angle-based Weight Selection (AWS)

Based on these geometric insights, the authors propose Angle-based Weight Selection (AWS) (also referred to as Angle-based Weight Masking). During fine-tuning, the system calculates the cosine similarity of all query and key weight pairs. If a pair is non-orthogonal (above a certain threshold), it is masked and frozen. Only the nearly orthogonal pairs—the ones handling "meaning"—are updated.

Key Results and Efficiency Gains

The researchers validated AWS using Alpaca fine-tuning on Llama-2 (7B and 13B) and evaluated the models across benchmarks like TruthfulQA, GSM8K, HellaSwag, and WikiText-2.

  • Massive Parameter Reduction: AWS can freeze over 92-93% of the query/key weight pairs in Llama-2 models without losing performance.
  • Performance Boost: Surprisingly, freezing the syntactic weights often improved performance, acting as a regularizer that prevents the model from overfitting on basic language structures. On TruthfulQA, Llama-2-7b fine-tuned with LoRA + AWS achieved a score of 35.00, outperforming standard LoRA (33.41) despite training far fewer parameters.
  • Complementary to LoRA: AWS is orthogonal to existing PEFT methods like LoRA, meaning it can be used alongside them to further prune the number of trainable parameters.
  • Broad Consistency: The "syntax-to-semantics" layer progression was observed across every major model family tested, including Llama-2, Llama-3, Mistral-7B, and Phi-2.

Limitations and Future Work

The primary limitation of AWS is that it currently only applies to the query and key weights, as it relies on the specific mathematical structure of RoPE. It does not yet address the value (V) weights or the Multi-Layer Perceptron (MLP) blocks, which constitute a large portion of the model's parameters.

Future research directions include extending this angle-based analysis to multi-modal vision-language models, investigating if similar geometric indicators exist for position information in image processing tasks, and exploring how this perspective can be used to further compress models for edge deployment.

Summary updated

Abstract

Transformer-based large language models (LLMs) have successfully handled various tasks. As one fundamental module in Transformers, position encoding encodes the positional information of tokens in a sequence. Specifically, rotary position embedding (RoPE), one of the most widely used techniques, encodes the positional information by dividing the query or key value with $d$ elements into $d/2$ pairs and rotating the 2d vectors corresponding to each pair of elements. Therefore, the direction of each pair and the position-related rotation jointly determine the attention score. In this paper, we show that the direction of the 2d pair is largely affected by the angle between the corresponding weight vector pair. We theoretically show that non-orthogonal weight vector pairs lead to great attention on tokens at a certain relative position and are less sensitive to the input which may correspond to basic syntactic information. Meanwhile, the orthogonal weight vector pairs are more flexible regarding the relative position, which may correspond to high-level syntactic information. Empirical evidence supports the hypothesis that shallow layers of LLMs focus more on local syntax and deep layers focus more on high-level semantics. Furthermore, we show that LLMs fine-tuning mainly changes the pairs of weight vectors that are nearly orthogonal, i.e., the weight corresponding to high-level semantics, which enables the reduction of the number of trainable parameters during fine-tuning without sacrificing performance. We propose a method namely Angle-based Weight Selection (AWS) to reduce the fine-tuning overhead and verify the effectiveness of the proposed method on widely used Alpaca fine-tuned Llama-2.
BibTeX
@inproceedings{chen2024what,
  author = {Yiting Chen and Junchi Yan},
  title = {{What Rotary Position Embedding Can Tell Us: Identifying Query and Key Weights Corresponding to Basic Syntactic or High-level Semantic Information}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=e5Mv7iWfVW},
}
Figure