Large language models do not reach an answer in one step. Each layer refines the guess of the layer before it. Recent neuroscience work shows that this layer-by-layer refinement lines up with the order in which the human brain processes a heard word. Shallow layers best predict brain activity at short delays after the word, and deep layers best predict activity hundreds of milliseconds later. This note proposes using that correspondence as a clock for social bias. For any sentence where a model must choose between a stereotypical and a counter-stereotypical reading, we can ask at what depth the model commits, and whether the human brain shows a matching delay at the matching latency.
The opportunity is that bias research measures almost everything at the output. Benchmarks such as BBQ, StereoSet and CrowS-Pairs score the final prediction. Brain-alignment research, meanwhile, has used only neutral narratives. Neither field can say whether a model's stereotype preference is a fast, shallow association that later evidence must override, or whether it is something the last layers add on top of an otherwise even-handed representation. These two possibilities call for different fixes. The brain comparison lets us test whether either pattern has a human counterpart. Without that comparison, calling a shallow preference "fast" or "automatic" would be only a figure of speech.
Core Claim
We claim that stereotype resolution has a measurable timing in both models and brains. We further claim that this timing is a separate quantity from surprise, meaning how unexpected a word is. Concretely, we expect three things. Stereotype-favoring preferences appear at shallow relative depth. Counter-stereotypical evidence becomes the model's stable preference at a later depth than stereotype-consistent evidence, even when the two are equally surprising to the model. And the brain shows a parallel extra delay for stereotype-incongruent words in naturalistic speech, again beyond what surprisal alone explains. A secondary hypothesis is that part of what the bias literature calls amplification sits in the final layers. Those same layers are known to drift away from human processing.
The closest work has each half of this but never both. The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models records electrodes placed directly on the cortex (ECoG) in nine patients listening to a podcast. It maps GPT-2 XL's 48 layers onto a roughly 500 ms window, with a layer-to-lag correlation of r = 0.93 in the temporal pole and 0.85 in the inferior frontal gyrus. It also shows that surprising words delay the peaks of early layers. That surprisal-delay analysis is the method we extend, and also the confound we must beat. Scaling and context steer LLMs along the same computational path as the human brain shows the same ordering in magnetoencephalography (MEG), a non-invasive recording of the brain's magnetic fields. That study uses 22 models on 10 hours of audiobook per participant and reports a temporal score of r = 0.99 for Llama 3-8B. It supplies the scoring method and the model set.
Large Language Models Are Human-Like Internally decodes intermediate layers with the logit lens and the tuned lens. The tuned lens is a learned per-layer linear map into the vocabulary. With it, the authors show that middle layers fit human reading times and the N400 brain response better than the final layer, and that the final layer is over-optimized for next-token prediction. That finding motivates our secondary hypothesis. All three studies use neutral stimuli.
On the bias side, FairBelief - Assessing Harmful Beliefs in Language Models shows that harmful completions sit at the top of the output distribution, and BBQ: A hand-built bias benchmark for question answering finds a 3.4-point accuracy penalty when the facts contradict a stereotype. Both read only the final layer, so neither can say where the preference forms or where the override fails.
Research Program
The model-side measurement is a per-item commitment depth. For a candidate pair, such as a stereotypical versus an anti-stereotypical answer, we compute a margin at each layer $l$ of $L$ through the tuned lens:
$$m_l = \log p_l(\text{stereo}) - \log p_l(\text{anti}),$$
The commitment depth $d^*$ is the smallest $l/L$ such that $m_k$ has the same sign as $m_L$ for every $k \ge l$.
We require stability from layer $l$ onward, rather than a single first crossing, because of LLM Layers Immediately Correct Each Other. That paper shows adjacent layers routinely cancel each other's updates, with a mean cosine similarity of about −0.2 between successive updates. A first-crossing definition would therefore track noise. That paper acts as a constraint on our design. We also use its sublayer attribution to ask whether the corrective feed-forward blocks push the margin toward or away from the stereotype.
We compute the margin curve and $d^*$ on several item types. These are BBQ ambiguous items, BBQ disambiguated items split into stereotype-consistent and counter-stereotypical, StereoSet intrasentence triplets, and CrowS-Pairs. The models are open causal models spanning the size range used by Raugel et al., from the GPT-2 family to Llama 3-8B. We use a tuned lens because the final-layer drift reported in MulCogBench makes the raw logit lens unreliable near the top of the network. The raw logit lens is reported as a robustness check.
The key analysis is a surprisal-matched contrast. We pair counter-stereotypical and stereotype-consistent items whose final-layer surprisal on the answer differs by less than a fixed bin width. We then test whether $d^*$ still differs between them.
The brain-side measurement uses the Armeni MEG dataset (A 10-hour within-participant magnetoencephalography narrative dataset). This dataset is about 10 hours of Sherlock Holmes audiobook per participant, with word-aligned timings. Its many gendered referents, occupations and class-marked roles provide naturally occurring tests of whether a word fits the stereotype its context sets up. We label each candidate word as stereotype-congruent, incongruent or neutral using human raters. The raters judge the word in context, which keeps the label independent of any model. We fit the Goldstein-style analysis separately for each class: a ridge regression from each layer's embedding to the MEG signal at each delay. We then estimate the shift in each layer's peak delay for incongruent versus congruent words, including surprisal as a covariate and also matching on surprisal.
If access allows, the ECoG podcast data serve as a replication with sharper anatomical localization. This step is conditional, because 30 minutes of podcast may contain too few incongruent events. A count of events before any model fitting, together with a power estimate, decides whether this replication is attempted.
Baselines
The naive reference point is final-layer scoring, which is how every bias benchmark in the file is reported. For calibration, StereoSet reports a language-modeling score of 88.3, a stereotype score of 60.1 and an iCAT of 73.0 for GPT-2 large. iCAT is StereoSet's combined measure of language competence and neutrality, where 100 is ideal. BBQ reports that about 77% of errors on ambiguous items are stereotype-aligned, and a 3.4-point penalty on counter-stereotypical disambiguated items. We rerun these final-layer scores on our own models and splits so that every comparison shares data. The layer-wise analysis adds information only if the stereotype margin's depth profile reveals structure the final score hides. The clearest case would be a counter-stereotypical answer that leads at intermediate depth and is reversed late. We found no reported result on Lacuna for layer-wise stereotype scores on any of these benchmarks.
The strongest existing competitor on the brain question is the surprisal-split peak-delay analysis of Goldstein et al. That analysis already predicts delays for unexpected words. It is run on exactly our words, with the same ridge regression, the same cross-validation folds and the same delay grid. Our account beats it only if stereotype congruence explains additional variance in peak delay over surprisal. Specifically, the incongruence coefficient must have a cross-validated confidence interval that excludes zero in the inferior frontal gyrus and temporal pole, or in the corresponding MEG sensors.
On the model side, the strongest competitor is Kuribayashi et al.'s best-layer tuned-lens procedure, which picks the most human-like layer on held-out data. Run as a debiasing readout with the same validation budget, it tests whether a principled depth choice adds anything over simply searching layers.
The ablation that isolates our key ingredient replaces stereotype labels with labels shuffled within surprisal bins, on both the model and brain sides. If the delay survives shuffling, it is not about stereotypes at all.
Why not just…?
Why not just call it surprisal? Counter-stereotypical words are usually less predictable, and Goldstein et al. already showed that unpredictable words shift peak delays. This is the most serious threat, and the proposal is built around it. Surprisal matching on the model side, surprisal as a covariate on the brain side, and the shuffled-label ablation decide the question. If the effect disappears under matching, we will report that stereotype timing is reducible to surprisal. That would be a useful negative result, because it would mean bias research can borrow the full surprisal toolkit.
Why not just measure layer-wise bias in the model and skip the brain? The model-only analysis is in fact the first deliverable, and it stands on its own. But it cannot support the interpretive claim that a shallow stereotype preference behaves like a fast association and a late override behaves like slow integration. That claim rests on the layer-to-time mapping, and it is testable only by checking whether the brain shows the delay at the latency the mapping predicts. Without the brain data, "early" means only "low layer index."
Why not just correlate brain-alignment scores with bias scores across model sizes? Both quantities grow with capability. StereoSet reports a Spearman correlation of 0.87 between language-modeling score and stereotype score, and Raugel et al. report r = 0.87 between temporal score and log model size. A raw cross-model correlation is therefore almost guaranteed and means nothing on its own. We report it only as a partial correlation that controls for language-modeling score. Even then we treat it as secondary to the item-level contrasts, which hold the model fixed.
Why not just debias at the output with prompts or decoding? FairBelief places the harmful completions at the top of the distribution, where truncating the tail does not reach them. Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach shows refusal falling from 32.8% to 29.4% as task complexity rises. Our readout experiment checks directly whether answering from the commitment-depth layer beats a system-prompt baseline. Under equal tuning, it should lower ambiguous-context bias on BBQ without costing disambiguated accuracy.
Decisive Checks
The first decisive check is the surprisal-matched model contrast. Counter-stereotypical disambiguated BBQ items should show a later commitment depth than consistent ones in at least the larger models, with item-level bootstrap intervals that exclude zero. If they do not, the depth-delay hypothesis fails before any brain data are touched.
The second is the MEG test. The incongruence coefficient on peak delay must survive the surprisal covariate in higher-order language regions and be absent in early auditory regions. Goldstein et al. found no layer-to-lag mapping in the middle superior temporal gyrus, so an effect there would point to an acoustic artifact rather than integration. The relative depth of the model's delay should also map, through the fitted layer-to-lag line, onto roughly the latency of the brain's delay.
The third tests whether amplification is a final-layer phenomenon. If the stereotype margin is smaller in the human-like middle layers than at the output, the bias analogue of Kuribayashi et al.'s final-layer over-optimization holds. In that case, reading answers from the commitment-depth layer should beat the best-layer search and the shuffled ablation on BBQ ambiguous bias at equal disambiguated accuracy.
References
- The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
- Scaling and context steer LLMs along the same computational path as the human brain
- Large Language Models are Human-Like Internally
- FairBelief - Assessing Harmful Beliefs in Language Models
- BBQ: A hand-built bias benchmark for question answering
- LLM Layers Immediately Correct Each Other
- MulCogBench: a multi-modal cognitive benchmark dataset for evaluating Chinese and English computational language models
- A 10-hour within-participant magnetoencephalography narrative dataset to test models of language comprehension
- StereoSet: Measuring stereotypical bias in pretrained language models
- Uncovering Stereotypes in Large Language Models: A Task Complexity-based Approach