From Language to Language-ish: How Brain-Like is an LSTM's Representation of Nonsensical Language Stimuli?

Maryam Hashemzadeh, Greta Kaufeld, Martha White, Andrea E. Martin, Alona Fyshe

Introduction

Recent advancements in natural language processing have shown that the internal states of Large Language Models (LLMs) correlate with human brain activity during language comprehension. However, it remains unclear whether this alignment is driven by shared semantic understanding or by the underlying structural (syntactic) processing. This study investigates this distinction by testing how LSTMs represent "out-of-distribution" (OOD) stimuli, such as nonsensical "Jabberwocky" sentences, and comparing these representations to human EEG data.

Methods

The researchers utilized a decoding model approach to map neural signals to model states. The experimental pipeline involved three primary stimuli conditions:

  1. Sentence: Well-formed Dutch sentences.
  2. Jabberwocky: Sentences composed of pseudo-words that maintain correct Dutch grammar and syntax.
  3. Word-list: Randomly ordered words devoid of grammatical structure.
Experimental pipeline for decoding EEG data into LSTM representations.

A character-level LSTM, pre-trained on 1 million Dutch Wikipedia sentences, was used to extract internal vector representations ($g(S)$). A character-level architecture was chosen specifically for its ability to process out-of-vocabulary pseudo-words by analyzing sub-word structures. Brain activity was recorded via EEG from 27 native Dutch speakers while they listened to these stimuli. A Ridge Regression decoder was then trained to map the EEG signals ($R$) to the LSTM’s hidden layers. Performance was evaluated using a "2 vs. 2" classification test, determining if the decoder could correctly match brain recordings with their corresponding model representations.

Results and Insights

The study found that the "brain-like" quality of LSTMs is significantly driven by structural representations. Decoding accuracy was highest for normal sentences (~0.60) and remained robust for Jabberwocky stimuli (~0.57), whereas performance for random word lists did not significantly exceed chance levels.

2 vs. 2 decoding accuracy across different layers of the character-level LSTM for the three stimuli conditions.

As shown in Figure 2, the middle layer (LSTM2) consistently provided the strongest correlation with brain activity across both structured conditions. Probing tasks confirmed that this layer acts as a "multitasker," encoding both complex syntactic and semantic information. Notably, a decoder trained on EEG data from normal sentences could successfully predict LSTM representations for Jabberwocky stimuli with an accuracy of 0.578. This cross-condition generalization suggests that the syntactic signatures in both the human brain and the LSTM are consistent enough to be recognized even when semantic content is absent.

Discussion

These findings validate Jabberwocky as an effective "syntax-only" control for comparative studies between humans and machines. Both systems appear to utilize similar structural processing machinery when encountering language-like stimuli.

However, some limitations remain. The authors noted that LSTMs might partially decode word positions rather than pure syntax, although probing tasks suggest syntax is the primary driver. Furthermore, while these results hold for Dutch—a language with transparent orthography—future research is needed to determine if these patterns generalize to languages with more complex orthographies (like English) or different structural paradigms (like Chinese). Additionally, investigating whether Transformer-based models exhibit similar "Jabberwocky" correlations would clarify if these findings are specific to recurrent architectures or a general property of modern language models.

Summary updated

BibTeX
@inproceedings{hashemzadeh2020from,
  author = {Maryam Hashemzadeh and Greta Kaufeld and Martha White and Andrea E. Martin and Alona Fyshe},
  title = {{From Language to Language-ish: How Brain-Like is an LSTM's Representation of Nonsensical Language Stimuli?}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.57},
}
Figure