Abstraction Induces the Brain Alignment of Language and Speech Models

Emily Cheng, Aditya R. Vaidya, Richard J. Antonello

TL;DR

This paper investigates why intermediate layers of language and speech models are better at predicting brain activity than output layers. They found that the "intrinsic dimension" (a measure of feature complexity) of these intermediate layers, which reflects semantic abstraction, is the key driver of this brain-model alignment, rather than next-word prediction capabilities. This suggests that the brain and AI models align because they both learn to abstract meaning from language.

What Problem Does This Paper Address?

For years, researchers have observed that the internal "thoughts" of large language models (LLMs) and speech models, particularly those in the middle layers, can predict human brain activity when listening to or reading language. This is fascinating because it implies a shared computational basis for understanding language. However, it's been unclear why this happens. Why are the intermediate layers, not the final output layers that are trained to predict the next word, so good at this? This paper aims to unravel this mystery by exploring the representational properties of these models and their relationship to brain activity.

Methods

The core of this research lies in analyzing the "intrinsic dimension" (Id) of model representations. Id is a geometric measure that quantifies the complexity of the feature space within a given layer. The authors hypothesize that a peak in Id in the intermediate layers signifies a phase of rich semantic abstraction. They then correlate this Id with:

  • Brain Encoding Performance: How well a model's layer representations predict fMRI and ECoG brain signals using linear models.
  • Next-Token Prediction (Surprisal): How well a layer's representations can predict the next word in a sequence.
  • Linguistic Probing: Using established benchmarks to test what kind of linguistic information (surface-level vs. semantic) is captured by each layer.
  • Causal Intervention: Finetuning speech models to better predict brain activity and observing how this affects their Id and semantic content.

Datasets

  • fMRI Data: Open-source functional magnetic resonance imaging (fMRI) data from three subjects (UTS01, UTS02, UTS03) listening to English podcast stories.
  • ECoG Data: Open-source electrocorticography (ECoG) data from nine subjects listening to podcasts.
  • Language Models: Six mid-sized LLMs (OPT and Pythia) and three speech models (WavLM and Whisper).
  • Training Data for Models: The Pile (for LLMs) and LibriSpeech (for speech models).

Key Insights and Ideas

  • Abstraction, Not Just Prediction: The strong alignment between AI models and the brain isn't driven by the models' ability to predict the next word. Instead, it's the abstraction of meaning that matters.
  • The "Sweet Spot" of Intrinsic Dimension: Intermediate layers, characterized by a peak in intrinsic dimension, are where this semantic abstraction happens and where brain predictivity is highest. This suggests a "sweet spot" in model processing for capturing brain-like representations.
  • Intrinsic Dimension as a Proxy for Semantic Richness: A higher intrinsic dimension in a layer strongly correlates with its ability to capture rich semantic features, which in turn leads to better prediction of brain activity.
  • Brain Tuning Enhances Abstraction: When models are finetuned to better predict brain responses, their intrinsic dimension and semantic content increase, providing causal evidence for the link between abstraction and brain alignment.
  • Language Modeling as a Driver of Abstraction: The complex task of language modeling, by its nature, forces models to build abstract representations of meaning, which then align with how the brain processes language.

Key Results

  • The layer with the highest intrinsic dimension consistently showed the best brain encoding performance across various LLMs and speech models.
  • The correlation between intrinsic dimension and brain predictivity was significantly positive (e.g., ρ = 0.72 for fMRI, ρ = 0.43 for ECoG).
  • Next-token prediction (surprisal) showed a decreasing trend over layers, with the lowest surprisal in the final layers, which were not the best predictors of brain activity.
  • Finetuning a speech model to better predict fMRI responses not only increased its encoding performance but also its semantic content and intrinsic dimension.
  • Regions of the brain that were better predicted by LLMs (fronto-temporal language areas) showed a stronger correlation between layerwise intrinsic dimension and encoding performance.

Limitations and Future Work

  • Interpreting Intrinsic Dimension: While intrinsic dimension is a powerful indicator, it doesn't explicitly tell us what specific linguistic features (e.g., syntactic, semantic) are being encoded. Future work could focus on explicitly describing these features.
  • Causal Story: While the finetuning experiments provide causal evidence, the full causal story of how intrinsic dimension relates to encoding performance is complex and rife with interdependencies.
  • Generalizability: The authors acknowledge that their work doesn't fully provide a detailed interpretable account of which specific features Ia tends to encode. Further research is needed to disentangle the precise mechanisms.
  • Beyond Next-Token Prediction: The paper strongly suggests that autoregressive language modeling is not the sole driver of brain-model similarity. Future work could explore other training objectives or architectures that might also lead to similar abstraction.

Summary updated

Abstract

Research has repeatedly demonstrated that intermediate hidden states extracted from large language models and speech audio models predict measured brain response to natural language stimuli. Yet, very little is known about the representation properties that enable this high prediction performance. Why is it the intermediate layers, and not the output layers, that are most effective for this unique and highly general transfer task? We give evidence that the correspondence between speech and language models and the brain derives from shared meaning abstraction and not their next-word prediction properties. In particular, models construct higher-order linguistic features in their middle layers, cued by a peak in the layerwise intrinsic dimension, a measure of feature complexity. We show that a layer's intrinsic dimension strongly predicts how well it explains fMRI and ECoG signals; that the relation between intrinsic dimension and brain predictivity arises over model pre-training; and finetuning models to better predict the brain causally increases both representations' intrinsic dimension and their semantic content. Results suggest that semantic richness, high intrinsic dimension, and brain predictivity mirror each other, and that the key driver of model-brain similarity is rich meaning abstraction of the inputs, where language modeling is a task sufficiently complex (but perhaps not the only) to require it.
BibTeX
@inproceedings{cheng2026abstraction,
  author = {Emily Cheng and Aditya R. Vaidya and Richard J. Antonello},
  title = {{Abstraction Induces the Brain Alignment of Language and Speech Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=n5Ds4qbtjM},
}
Figure