TL;DR
Does training on the abstract logic of computer code make a language model "brain-like"? This paper finds that the answer is largely no. While code-trained models achieve positive brain alignment scores, these scores are mostly a byproduct of the transformer architecture itself; only training on natural language (English) produces the specific internal representational geometry that truly mirrors human neural activity.
What Problem Does This Paper Address?
Large Language Models (LLMs) are surprisingly good at predicting human brain activity during language processing. A lingering question in the field is why. Is it because they learn the nuances of human language, or is it because they learn "abstract sequential and compositional structure"?
Because programming languages are highly structured and logical, some researchers hypothesized that code-trained models might align with the brain just as well as language-trained ones. If true, this would suggest that "intelligence" or "abstraction" is the driver of alignment, rather than language specifically. This paper tests that hypothesis by comparing models trained on everything from high-level Python to raw machine code against models trained on simple English stories.
Methods
The author trained 25 GPT-2-small models from scratch, using a shared tokenizer and training recipe across five different corpora. To ensure a fair comparison, they used:
- Encoding Analysis: Fitting a ridge regression to predict fMRI/ECoG responses from model activations.
- Representational Geometry: Using techniques like Centered Kernel Alignment (CKA) to see how reproducible internal states are across different training runs, and Representational Similarity Analysis (RSA) to compare the "dissimilarity matrices" of the model vs. the brain without fitting any parameters.
- Ablations and Interventions: "Transplanting" blocks from language models into code models and retraining only specific parts (like embeddings) to see which components carry the "brain-like" signal.
Datasets
- Training Corpora:
- TinyStories: Simple, synthetic English prose.
- Python: High-level source code.
- C functions, x86-64 assembly, and Machine code (hex): Three versions of the same underlying programs at different levels of abstraction.
- Brain Benchmarks:
- Pereira-243/384: fMRI responses to sentences.
- Tuckute2024: Language-network fMRI responses.
- Blank2014: fMRI responses to stories.
- Fedorenko2016: ECoG (electrodes on the brain) responses to words in sentences.
Key Insights and Ideas
- The "Random" Baseline is High: A key finding is that a randomly initialized transformer already gets a "positive" brain score. This is due to the architecture (attention, positional encoding) and the tokenizer. Therefore, a model's success should be measured by how much it improves over a random baseline, not its absolute score.
- No "Abstraction Ladder": The researchers expected a hierarchy where Python > C > Assembly > Binary in terms of brain alignment. Instead, they found no such ordering. Code models generally performed similarly to each other and stayed near the random-weight level.
- Geometry vs. Accuracy: A model can be "reproducible" (getting the same internal states across different seeds) without being "brain-like." For example, binary-trained models were highly reproducible but occupied only about three effective dimensions, whereas the English-trained models expanded to about 26 dimensions.
- The Embedding Bottleneck: Interestingly, updating just the embedding matrix of a code model with English data improved its brain alignment more than further training on its native code. This suggests the "interface" between the tokens and the transformer blocks is a major factor in alignment.
Key Results
On the primary benchmarks (Pereira-243 and Tuckute2024), the English-trained TinyStories models significantly outperformed all code models.
- Encoding Advantage: TinyStories exceeded the code mean by $\Delta r = 0.053$ and $0.059$. As seen in Figure 2, this advantage persists across almost the entire depth of the model.
- Fit-Free RSA: In tests that don't allow for a "fitted" readout, TinyStories' correspondence with neural data peaked at $\rho = 0.14$, which is double the level of code and random-weight models.
- Benchmark Specificity: The "language advantage" wasn't universal; it disappeared on the Blank2014 and Fedorenko2016 benchmarks, highlighting that "brain-likeness" depends heavily on the specific dataset and neural recording method used.
Limitations and Future Work
- Architecture: The study was limited to GPT-2-small. Larger models or different architectures (like Mamba or RWKV) might show different dynamics.
- Corpus Diversity: TinyStories is a very specific, simplified version of English. Future work could explore whether more complex or diverse natural language data further widens the gap.
- The "Stitch" Problem: The block transplant experiments were done without a "learned stitch" (a transformation layer between blocks), which might have masked some potential for code-to-language transfer.
- Domain Specificity: The study compares models against the human language network. It does not address how models might align with the brain's "Multiple Demand" network, which is known to be active during code comprehension in humans.
Summary updated