The Low-Dimensional Linear Geometry of Contextualized Word Representations

Evan Hernandez, Jacob Andreas

Abstract: Contextualized word representations from models like BERT and ELMo encode a wealth of linguistic information. This paper investigates the geometric structure of this information, demonstrating that linguistic features are organized into extremely low-dimensional linear subspaces. Through rank-constrained probing and nullspace projection, we show these subspaces are hierarchical, distributed across neurons, and causally influence model behavior.

Introduction

Linguistic features in models like BERT and ELMo aren't just scattered throughout the hidden states; they are neatly organized into extremely low-dimensional linear subspaces (often fewer than 10 dimensions). These subspaces are hierarchical, distributed across many neurons rather than pinned to specific ones, and—most importantly—causally drive the model's behavior.

For years, researchers have used "probes"—simple classifiers trained on top of frozen model representations—to show that Large Language Models (LLMs) "know" syntax, parts of speech, and morphology. However, knowing that a model encodes a feature is different from knowing how it encodes it. This paper seeks to map the "linear geometry" of these representations to understand the physical shape of linguistic knowledge inside LLMs.

Methods: Rank-Constrained Probing

The authors introduce a technique called rank-constrained probing. Instead of training a standard probe on the full 768-dimensional (BERT) or 1024-dimensional (ELMo) hidden state, they force the probe to pass through a bottleneck: a low-rank projection matrix $\Pi$.

Conceptual overview of linguistic feature encoding in low-dimensional subspaces, illustrating dimensionality, hierarchy, alignment, and causality.
  1. Dimensionality Search: They sweep through different ranks ($d$) to find the smallest subspace that maintains near-optimal accuracy for a task.
  2. Hierarchical Probing: To see if subspaces are nested, they first project data into a "Noun" subspace and then try to find an even smaller "Proper Noun" subspace within it.
  3. Ablation via Nullspace Projection: To test causality, they "delete" a specific linguistic feature by projecting the model's hidden states onto the nullspace of that feature's subspace. If the model’s performance on a specific task drops only when the relevant subspace is removed, it proves the model actually uses that geometric direction to "think."

Key Results

The empirical findings reveal that linguistic features are remarkably "skinny." For Part-of-Speech (POS) and Dependency Label Prediction (DLP) tasks, a subspace of $d < 10$ was sufficient to reach within 5% of the best possible accuracy. Even the complex task of predicting dependency edges (DEP) required only $d < 21$.

Accuracy of probes as a function of projection dimension across different BERT layers and tasks, showing rapid convergence in low dimensions.

Key Insights

  • Geometry mirrors linguistics: The representations are hierarchical. The subspace for "Proper Nouns" is a subset of the subspace for "Nouns," which is a subset of the general "Part of Speech" subspace. The model's internal geometry reflects the tree-like taxonomy of human language.
  • Information is distributed, not local: The authors found that these subspaces are not axis-aligned. You can't just point to "the verb neuron." Instead, the information is smeared across many neurons; it is the direction in the vector space that matters.
  • Causal Control: By ablating the 4-dimensional "noun" subspace in BERT's final layer, the researchers could specifically confuse the model about whether a subject was singular or plural, without breaking its ability to recognize verbs or other parts of speech.

Limitations and Future Work

The study focuses on linear subspaces. While the authors show that linear probes capture most of the relevant information, there may be "residual" linguistic information stored in non-linear manifolds that this method misses. Furthermore, the paper focuses on ELMo and BERT (encoder-only models). It remains to be seen if the same low-dimensional linear geometry holds for massive decoder-only models like GPT-4 or Llama-3.

Conclusion

Linguistic knowledge in LLMs is stored in a highly structured, low-dimensional, and linear manner. This geometric perspective provides a powerful tool for understanding and potentially controlling the internal reasoning of large-scale language models.

Summary updated

BibTeX
@inproceedings{hernandez2021low,
  author = {Evan Hernandez and Jacob Andreas},
  title = {{The Low-Dimensional Linear Geometry of Contextualized Word Representations}},
  booktitle = {CoNLL},
  year = {2021},
  doi = {10.18653/v1/2021.conll-1.7},
}
Figure