Abstract
State-of-the-art natural language processing (NLP) models, while effective, often operate as "black boxes" with opaque internal representations. This paper proposes a novel framework that uses human brain activity as a gold standard for language comprehension to interpret and improve these models. By aligning the internal embeddings of models like BERT and Transformer-XL with fMRI and MEG recordings of humans reading natural text, the authors identify layers that capture specific linguistic functions. Furthermore, they demonstrate that modifying model architectures to better align with brain-like processing—specifically using uniform attention in early layers—significantly improves performance on complex syntactic tasks.
Introduction
The rapid advancement of deep learning has led to NLP models that excel at a wide range of tasks. However, understanding how these models represent language remains a challenge. Traditional probing methods rely on artificial tasks, which may not reflect true comprehension. This research bridges cognitive neuroscience and AI by leveraging the human brain's proven capacity for language understanding to evaluate and refine machine representations.
Problem Statement
Despite their success, deep NLP models lack transparency. We do not fully understand the nuances of their internal embeddings. The authors argue that if a model's representations can accurately predict brain activity in regions associated with specific linguistic functions, we can gain insight into what information the model is processing.
Methodology: Aligning Models to Brain Activity
The researchers developed an alignment approach to map NLP model representations to human brain activity. This process involves acquiring brain data (fMRI/MEG) and model features for the same natural text, then training encoding models to predict brain activity from model layers.
- Data Acquisition: fMRI and MEG recordings were collected from subjects reading Chapter 9 of Harry Potter and the Sorcerer’s Stone.
- Feature Extraction: The same text was processed through ELMo, BERT, USE, and Transformer-XL, with representations extracted from every layer.
- Encoding Models: A linear regression model was trained to map model layer representations to brain activity.
- Significance Testing: A classification task determined if the model was predictive of brain activity, identifying regions predicted by specific layers.
Interpreting Representations with Brain Priors
To interpret the results, the authors used established neuroscience "priors"—specific brain regions known to handle either isolated words (Group 1) or long-range context (Group 2). By seeing which layers predict activity in which regions, they can "decompose" complex embeddings into constituent linguistic parts.
Key Insights and Results
- The Middle Layer Peak: In transformer models, middle layers (e.g., BERT Layer 7) are the most "brain-like," balancing contextual integration with generalizability.
- Transformer-XL and Long-Range Context: Transformer-XL uniquely maintains brain-prediction performance as context length increases, validating its recurrence mechanism for long-range dependencies.
- Uniform Attention in Early Layers: BERT’s early layers were found to be "over-specialized." Replacing learned attention with uniform attention in layers 1-6 made the model more brain-aligned and improved syntactic accuracy in 8 out of 13 tasks.
- Temporal Dynamics: MEG analysis confirmed that ELMo embeddings capture word-length information at 100ms and part-of-speech information at 200ms, mirroring the temporal progression of human brain activity.
Conclusion
This work demonstrates that the human brain can serve as a powerful tool for interpreting and improving AI. By aligning machine representations with biological ones, researchers can identify architectural inefficiencies and develop more robust, "brain-like" language models. Future research aims to integrate brain data directly into the training process of NLP models.
Summary updated