Information gain modulates brain activity evoked by reading

Lauri Kangassalo, Michiel Spapé, Niklas Ravaja, Tuukka RuotsaloScientific 2020 · DOI

Abstract

The human brain processes language to optimize efficient communication. While lower-level features like word length and frequency are known to affect neural responses, the impact of discourse-level context remains less understood. This study proposes an information-theoretic model to explain cognitive resource allocation during reading. Using EEG data from participants reading Wikipedia entries, we demonstrate that "Information Gain" (IG)—a measure of word specificity relative to a topic—modulates brain activity. High IG words elicit a sustained positive shift in event-related potentials (ERPs). Furthermore, a machine learning classifier successfully predicted IG from unseen EEG data, suggesting that biological information processing seeks to maximize performance under capacity constraints.

Introduction

How does the brain navigate the "information economy" of natural reading? Traditional psycholinguistics has focused on how we react to word-level features or grammatical violations. However, reading is fundamentally about extracting information within a broader context. This research, published in Scientific Reports, investigates how the brain dynamically allocates processing power based on a word's contribution to the overall topic—a phenomenon termed "Information Gain."

Quantifying Informativeness via Information Theory

To study this, researchers utilized Information Gain (IG) to mathematically define a word's value. IG measures the reduction in uncertainty about a document's topic provided by a specific word.

Distribution of words by Information Gain (IG). High IG words (red) are topic-specific (e.g., "tiber", "hallucinations"), while low IG words (blue) are common or less specific (e.g., "the", "is").

In a large Wikipedia corpus, words like "magma" or "tectonic" provide high IG in the context of a "Volcanoes" article because they point to specific topics. In contrast, words like "usually" or "the" provide little information about the overarching subject. This metric allows researchers to test the "efficient coding principle": the hypothesis that biological systems evolve to maximize information throughput while minimizing wasted effort on redundant data.

Experimental Design

The study involved 17 participants reading sentences from 16 diverse Wikipedia documents (e.g., "Cats," "Rome," "Volcanoes") presented via Rapid Serial Visual Presentation (RSVP). To isolate the effect of Information Gain, the team used Linear Mixed Models (LMMs) to control for confounding variables such as word length and frequency, which are often correlated with informativeness.

Results: The Neural Signature of Information Gain

The primary finding was a significant modulation of the brain's electrical activity by Information Gain. High-information words triggered a sustained "slow positive shift" in the EEG signal, beginning around 250ms and lasting until 700ms post-stimulus.

Event-related potentials (ERPs) for high vs. low information gain words. High IG words (red) trigger a sustained positive shift compared to low IG words (blue).

This "Early Positive Shift" (EPS) suggests that the brain prioritizes contextually rich words. Because the effect starts as early as 250ms—prior to full semantic integration—it implies a top-down mechanism where the brain uses preceding context to "pre-calculate" which words merit more cognitive resources. Statistical analysis confirmed that IG significantly improved the model's ability to explain EEG variance (chi-squared = 5.98, p < 0.05), with the most pronounced impact observed on the N400 component (chi-squared = 7.73, p < 0.05), typically associated with meaning processing.

Machine Learning and Predictive Power

To validate these findings, the researchers employed a Regularized Linear Discriminant Analysis (LDA) classifier. The goal was "reverse inference": predicting whether a participant was reading a high- or low-information word based solely on a single second of EEG data. The classifier achieved an average AUC (Area Under the Curve) of 0.641 (p < 0.01). While seemingly modest, this result is highly significant given the inherent noise in single-trial EEG data and demonstrates that informativeness is encoded reliably enough to be decoded by machine learning.

Discussion and Future Directions

The results support the view of the brain as a biological information processor that optimizes performance by divesting resources from redundant words. However, the study has limitations. The RSVP method (one word at a time) differs from natural reading, where eye movements allow us to skip uninformative words. Future research using eye-tracking or fMRI could provide better spatial localization of these effects.

Ultimately, this work paves the way for non-intrusive technologies that can monitor perceived text informativeness in real-time, potentially leading to advanced brain-computer interfaces for personalized learning and information filtering.

Summary updated

Abstract

The human brain processes language to optimise efficient communication. Studies have shown extensive evidence that the brain's response to language is affected both by lower-level features, such as word-length and frequency, and syntactic and semantic violations within sentences. However, our understanding on cognitive processes at discourse level remains limited: How does the relationship between words and the wider topic one is reading about affect language processing? We propose an information theoretic model to explain cognitive resourcing. In a study in which participants read sentences from Wikipedia entries, we show information gain, an information theoretic measure that quantifies the specificity of a word given its topic context, modulates word-synchronised brain activity in the EEG. Words with high information gain amplified a slow positive shift in the event related potential. To show that the effect persists for individual and unseen brain responses, we furthermore show that a classifier trained on EEG data can successfully predict information gain from previously unseen EEG. The findings suggest that biological information processing seeks to maximise performance subject to constraints on information capacity.
BibTeX
@inproceedings{kangassalo2020information,
  author = {Lauri Kangassalo and Michiel Spapé and Niklas Ravaja and Tuukka Ruotsalo},
  title = {{Information gain modulates brain activity evoked by reading}},
  booktitle = {Scientific},
  year = {2020},
  doi = {10.1038/s41598-020-63828-5},
}
Figure