Abstract
By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimensional, can be explained using relatively few degrees of freedom. An open question is whether contemporary language models (LMs) reflect the intrinsic simplicity of language that is enabled by compositionality. We take a geometric view of this problem by relating the degree of compositionality in a dataset to the intrinsic dimension (ID) of its representations under an LM, a measure of feature complexity. We find not only that the degree of dataset compositionality is reflected in representations' ID, but that the relationship between compositionality and geometric complexity arises due to learned linguistic features over training. Finally, our analyses reveal a striking contrast between nonlinear and linear dimensionality, showing they respectively encode semantic and superficial aspects of linguistic composition.
Introduction
Human language is remarkably efficient: a finite vocabulary and a small set of syntactic rules allow for the generation of an infinite variety of meaningful expressions. This property, known as compositionality, suggests that while the space of possible sentences is vast, the actual degrees of freedom required to describe them are relatively few. The Manifold Hypothesis posits that high-dimensional data, such as text embeddings, resides on a much lower-dimensional manifold. This research investigates whether Large Language Models (LLMs) exploit this geometric structure to represent linguistic rules and how this geometry evolves during training.
The Geometric Dichotomy: Form vs. Meaning
To analyze the internal states of Transformers, we employ two distinct geometric metrics:
- Nonlinear Intrinsic Dimension ($I_d$): Estimated using the TwoNN algorithm, $I_d$ measures the minimal degrees of freedom needed to describe the data manifold without information loss. It serves as a proxy for the complexity of underlying semantic concepts.
- Linear Effective Dimension ($d$): Calculated via Principal Component Analysis (PCA), $d$ measures the number of linear directions required to capture 99% of the data's variance, representing how the model distributes information in its high-dimensional embedding space.
Our findings reveal a "geometric dichotomy." Linear dimensionality ($d$) correlates strongly with surface-level complexity (e.g., Kolmogorov complexity or gzip compression). In contrast, nonlinear $I_d$ tracks the semantic degrees of freedom. When sentence meaning is destroyed by shuffling words, $I_d$ collapses because the semantic structure is lost, whereas $d$ often increases as the model can no longer linearly compress the input into a structured manifold.
Scaling Laws and Compositionality
A critical observation is how these dimensions scale with the model's hidden dimension ($D$). While the linear space ($d$) expands linearly with $D$, the nonlinear $I_d$ remains remarkably stable and low (typically $O(10)$). This suggests that once a model reaches a sufficient capacity, it recovers a consistent number of underlying linguistic degrees of freedom regardless of its total width.
As shown in the figure above, increasing the "coupling" between words (which reduces compositionality and semantic degrees of freedom) leads to a measurable decrease in $I_d$, while $d$ continues to scale with the ambient dimension $D$.
The Learning Phase Transition
The geometric structure of an LLM is not static; it emerges through a distinct phase transition during training. Around 1,000 training steps, models across different sizes (410m to 6.9b parameters) exhibit a sudden drop and redistribution of $I_d$ across layers. This geometric shift aligns precisely with a sharp increase in the model's performance on linguistic benchmarks like PIQA and LAMBADA.
This transition indicates that the model has moved beyond mere memorization of word-level statistics and has begun to extract the structured, low-dimensional features that characterize compositional language.
Conclusion
By analyzing the geometry of LLM representations, we demonstrate that models naturally compress compositional language into a low-dimensional nonlinear manifold. The distinction between linear and nonlinear dimensionality provides a powerful tool for separating surface-level form from deep semantic meaning. The emergence of this structure during a specific training phase suggests that $I_d$ can serve as a geometric signature of linguistic competence.
Summary updated