Modeling Bottom-up Information Quality during Language Processing

Cui Ding, Yunju Yin, Lena A. Jäger, Ethan Wilcox2025 · DOI

Abstract

While psycholinguistic research has extensively explored how "top-down" contextual expectations influence reading, the formal quantification of "bottom-up" visual information quality has remained elusive. This paper introduces a formal, information-theoretic framework to quantify how the quality of orthographic input affects cognitive effort. By modeling reading as a Bayesian sequential sampling process, the authors demonstrate that reading time is inversely proportional to the mutual information (MI) between visual signals and word identity. Using multimodal large language models (MLLMs) to estimate information content, the study identifies a significant "top-half" information advantage in both English and Chinese scripts and characterizes the non-linear relationship between visual degradation and processing speed.

Introduction

It is a common experience that reading a smudged or partially obscured text requires more effort and time than reading clear print. However, traditional models of reading have lacked a rigorous mathematical framework to explain exactly how much specific visual degradations increase cognitive load. Most existing research focuses on top-down factors—how our internal vocabulary and the preceding context allow us to predict upcoming words.

This paper shifts the focus to the physical reality of processing visual signals (orthography). The authors aim to bridge the gap between abstract noisy-channel communication models and the behavioral reality of human reading by providing a quantitative metric for "information quality."

A Bayesian Framework for Reading Effort

The researchers propose that word recognition is a sequential sampling process. In this model, a reader takes successive "samples" of visual evidence until their uncertainty (entropy) regarding the word's identity falls below a specific threshold.

Mathematically, the authors derive that the number of samples required—and by extension, the reading time—is inversely proportional to the Mutual Information (MI) between the visual input and the word identity. This provides a formal link between the clarity of a visual signal and the time spent processing it. To calculate MI for degraded or partially obscured words, the study utilized three distinct estimators:

  1. A Bayesian Baseline: Utilizing structural similarity (SSIM) metrics.
  2. Qwen2.5-VL-7B: A state-of-the-art multimodal large language model (MLLM), tested in both zero-shot and fine-tuned capacities.
  3. TransOCR: A custom-trained Transformer-based Optical Character Recognition model.

Experimental Methodology

To validate the model against human behavior, the authors employed Mouse Tracking for Reading (MoTR). This web-based paradigm requires users to reveal text by moving a mouse cursor, serving as a high-resolution proxy for eye-tracking.

The study tested three primary visual conditions:

  • Unoccluded: The full word is visible.
  • Upper-half visible: Only the top 50% of the word/character is shown.
  • Lower-half visible: Only the bottom 50% of the word/character is shown.

Stimuli were drawn from the OneStopQA dataset (English Guardian news articles) and a newly introduced Chinese OneStopQA benchmark, consisting of Mandarin translations of the same articles. Lexical frequency data from SUBTLEXus (English) and the Modern Chinese Character Database (Chinese) were used to control for word familiarity.

Key Insights: The "Top-Half" Advantage

The study provides quantitative evidence for a long-standing psycholinguistic theory: the upper half of a word or character contains more information than the lower half.

Cross-Linguistic Differences

While both English and Chinese exhibit a "top-half" advantage, the effect is significantly more pronounced in English. This suggests that alphabetic scripts rely heavily on distinctive features located in the upper portions of letters, such as ascenders (e.g., the tops of 'd', 'h', 'k'). In contrast, Chinese logographs, which are structured within a more balanced two-dimensional square frame, show a more even distribution of information, though the top-heavy bias remains present.

Non-linear Processing Effort

The relationship between information loss and reading time is non-linear. The model and behavioral data show that small degradations in a high-quality signal lead to disproportionately large increases in reading time. Conversely, once a signal is already heavily degraded, further loss of information has a diminishing impact on processing speed, as the reader is already operating at a maximum level of uncertainty.

Results and Quantitative Costs

The behavioral experiments confirmed the predicted hierarchy of reading speeds: Full < Upper < Lower.

By correlating MI with reading times, the authors estimated the specific "cost" of information loss. In English, every "nat" (a natural unit of information) lost resulted in a slowdown of 25–66 ms. In Chinese, the cost was notably lower, ranging from 5–10 ms per nat. Furthermore, the fine-tuned MLLMs (Qwen2.5-VL and TransOCR) achieved over 90% accuracy on full words and maintained robust performance under occlusion, closely mirroring the patterns of human error and delay.

Limitations and Future Directions

While the framework provides a powerful tool for quantifying bottom-up processing, it currently operates under several simplifications:

  • Linearity: The model assumes a word-by-word process and does not yet account for regressions (looking back) or parafoveal preview (processing upcoming words in the periphery).
  • Sample Independence: The model treats visual samples as independent, whereas the human visual system likely integrates information across samples in a more complex, dependent manner.
  • Contextual Interaction: Future work is needed to explore how strong top-down contextual expectations might mitigate the need for high-quality visual input, potentially lowering the "cost" per nat when a word is highly predictable.

By formalizing the role of visual quality, this research provides a new foundation for understanding the interplay between the physical signal and cognitive effort in language processing.

Summary updated

Abstract

Contemporary theories model language processing as integrating both top-down expectations and bottom-up inputs.One major prediction of such models is that the quality of the bottom-up inputs modulates ease of processing-noisy inputs should lead to difficult and effortful comprehension.We test this prediction in the domain of reading.First, we propose an information-theoretic operationalization for the "quality" of bottom-up information as the mutual information (MI) between visual information and word identity.We formalize this prediction in a mathematical model of reading as a Bayesian update.Second, we test our operationalization by comparing participants' reading times in conditions where words' information quality has been reduced, either by occluding their top or bottom half, with full words.We collect data in English and Chinese.We then use multimodal language models to estimate the mutual information between visual inputs and words.We use these data to estimate the specific effect of reduced information quality on reading times.Finally, we compare how information is distributed across visual forms.In English and Chinese, the upper half contains more information about word identity than the lower half.However, the asymmetry is more pronounced in English, a pattern which is reflected in the reading times.
BibTeX
@misc{ding2025modeling,
  author = {Cui Ding and Yunju Yin and Lena A. Jäger and Ethan Wilcox},
  title = {{Modeling Bottom-up Information Quality during Language Processing}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.592},
}
Figure