TL;DR
Why do the intermediate layers of large language and speech models predict human brain activity better than their final output layers? This paper demonstrates that model-brain alignment is driven by meaning abstraction—quantified by a peak in the representation's intrinsic dimension—rather than the next-word prediction objective itself.
What Problem Does This Paper Address?
A fascinating discovery in computational neuroscience is that the hidden states of deep language models (LLMs) and speech models can linearly predict neural responses (measured via fMRI or ECoG) when humans listen to natural language. However, an unexplained anomaly persists: the intermediate layers of these networks consistently outperform the final output layers at predicting brain activity.
This raises a fundamental question about why models and brains align. Is it because they share a similar predictive coding objective (minimizing next-word surprisal), or is it a byproduct of convergent evolution toward shared abstractions of meaning? This paper addresses this gap by investigating the geometric and semantic properties of hidden layers across various language and audio foundation models.
Methods
The authors employ a multi-pronged approach combining neural encoding, information theory, and representation geometry:
- Brain Encoding Models: They train ridge regression models to map hidden states from LLMs and speech models to fMRI and ECoG brain signals.
- Intrinsic Dimension ($I_d$): To measure feature complexity without assuming linearity, they use a state-of-the-art nonlinear estimator called GRIDE (Generalized Ratios Intrinsic Dimension Estimator) on generic, in-distribution datasets.
- Layerwise Probing: They use linear probes (SentEval for text, GloVe/Mel-filterbanks for audio) to evaluate whether a layer contains low-level surface features or high-level semantic information.
- Layerwise Surprisal: They use the TunedLens technique to map intermediate layers directly to the vocabulary space, calculating next-token prediction error at each stage of processing.
- Causal Manipulation (Braintuning): They finetune a speech model directly on fMRI data to observe how optimizing for brain predictivity alters the model's internal geometry and semantic content.
Datasets
The study utilizes several open-source datasets across modalities:
- fMRI: Data from 3 subjects listening to ~20 hours of English podcast stories (LeBel et al., 2023).
- ECoG: High-gamma frequency band signals from 9 subjects listening to a 30-minute podcast (Zada et al., 2025).
- Model Training/Evaluation: The Pile (for LLMs), LibriSpeech (for speech models), and SentEval (for linguistic probing tasks).
Key Insights and Ideas
- The Two-Phase Processing Hypothesis: Language models operate in two distinct phases across their layers. First, they build up complex, higher-order linguistic features from the input, causing the intrinsic dimension ($I_d$) to peak in the middle layers. Second, they compress and refine this representation to fit the specific output distribution required for next-token prediction.
- Prediction Narrows Representations: While final layers are optimal for minimizing next-token surprisal, this objective actually forces a dimensionality bottleneck that strips away broader semantic abstractions, making output layers less similar to the human brain.
- Abstraction, Not Just Dimensionality: High dimensionality alone does not guarantee brain alignment. Testing a baseline of Random Fourier Features (RFF) showed that while increasing dimensionality improves brain predictivity slightly, it plateaus at a very low level compared to actual language models. The dimensionality must represent structured linguistic abstraction.
Key Results
- $I_d$ Strongly Tracks Brain Predictivity: Across virtually all tested models (OPT, Pythia, WavLM, Whisper), the layers with the highest intrinsic dimension were almost perfectly co-located with the layers that best predicted fMRI and ECoG signals. The global Spearman correlation between normalized $I_d$ and encoding performance was $\rho = 0.76$ for fMRI and $\rho = 0.43$ for ECoG.
- Surprisal Fails to Explain Layerwise Alignment: Next-token surprisal decreases monotonically across layers, meaning the final layer is always the best predictor of the next word. Yet, brain predictivity peaks much earlier (between 40-70% of processing). In mixed-effects modeling, $I_d$ completely explained away the effect of surprisal for fMRI data.
- Causal Evidence via Braintuning: Directly finetuning WavLM to better predict fMRI responses causally increased both its internal intrinsic dimension and its performance on semantic probing tasks, confirming a tight, non-trivial link between semantic richness, geometry, and brain alignment.
Limitations and Future Work
The authors note that Intrinsic Dimension ($I_d$) is a coarse-grained, dataset-level metric. While it successfully quantifies the number of degrees of freedom (or feature complexity) within a layer, it does not explicitly reveal what those features are (e.g., specific syntactic structures vs. semantic concepts).
Future work is needed to explicitly map these geometric degrees of freedom to interpretable linguistic properties and to fully untangle the co-dependencies between intrinsic dimensionality and neural encoding performance.
Summary updated