Abstract
The human brain processes language to optimize efficient communication. While lower-level features like word length and frequency are known to affect neural responses, the impact of discourse-level context remains less understood. This study proposes an information-theoretic model to explain cognitive resource allocation during reading. Using EEG data from participants reading Wikipedia entries, we demonstrate that "Information Gain" (IG)—a measure of word specificity relative to a topic—modulates brain activity. High IG words elicit a sustained positive shift in event-related potentials (ERPs). Furthermore, a machine learning classifier successfully predicted IG from unseen EEG data, suggesting that biological information processing seeks to maximize performance under capacity constraints.
Introduction
How does the brain navigate the "information economy" of natural reading? Traditional psycholinguistics has focused on how we react to word-level features or grammatical violations. However, reading is fundamentally about extracting information within a broader context. This research, published in Scientific Reports, investigates how the brain dynamically allocates processing power based on a word's contribution to the overall topic—a phenomenon termed "Information Gain."
Quantifying Informativeness via Information Theory
To study this, researchers utilized Information Gain (IG) to mathematically define a word's value. IG measures the reduction in uncertainty about a document's topic provided by a specific word.
In a large Wikipedia corpus, words like "magma" or "tectonic" provide high IG in the context of a "Volcanoes" article because they point to specific topics. In contrast, words like "usually" or "the" provide little information about the overarching subject. This metric allows researchers to test the "efficient coding principle": the hypothesis that biological systems evolve to maximize information throughput while minimizing wasted effort on redundant data.
Experimental Design
The study involved 17 participants reading sentences from 16 diverse Wikipedia documents (e.g., "Cats," "Rome," "Volcanoes") presented via Rapid Serial Visual Presentation (RSVP). To isolate the effect of Information Gain, the team used Linear Mixed Models (LMMs) to control for confounding variables such as word length and frequency, which are often correlated with informativeness.
Results: The Neural Signature of Information Gain
The primary finding was a significant modulation of the brain's electrical activity by Information Gain. High-information words triggered a sustained "slow positive shift" in the EEG signal, beginning around 250ms and lasting until 700ms post-stimulus.
This "Early Positive Shift" (EPS) suggests that the brain prioritizes contextually rich words. Because the effect starts as early as 250ms—prior to full semantic integration—it implies a top-down mechanism where the brain uses preceding context to "pre-calculate" which words merit more cognitive resources. Statistical analysis confirmed that IG significantly improved the model's ability to explain EEG variance (chi-squared = 5.98, p < 0.05), with the most pronounced impact observed on the N400 component (chi-squared = 7.73, p < 0.05), typically associated with meaning processing.
Machine Learning and Predictive Power
To validate these findings, the researchers employed a Regularized Linear Discriminant Analysis (LDA) classifier. The goal was "reverse inference": predicting whether a participant was reading a high- or low-information word based solely on a single second of EEG data. The classifier achieved an average AUC (Area Under the Curve) of 0.641 (p < 0.01). While seemingly modest, this result is highly significant given the inherent noise in single-trial EEG data and demonstrates that informativeness is encoded reliably enough to be decoded by machine learning.
Discussion and Future Directions
The results support the view of the brain as a biological information processor that optimizes performance by divesting resources from redundant words. However, the study has limitations. The RSVP method (one word at a time) differs from natural reading, where eye movements allow us to skip uninformative words. Future research using eye-tracking or fMRI could provide better spatial localization of these effects.
Ultimately, this work paves the way for non-intrusive technologies that can monitor perceived text informativeness in real-time, potentially leading to advanced brain-computer interfaces for personalized learning and information filtering.
Summary updated