TEMPERATURE-SCALING SURPRISAL ESTIMATES IMPROVE FIT TO HUMAN READING TIMES – BUT DOES IT DO SO FOR THE “RIGHT REASONS”?

Tong Liu, Iza Škrjanec, Vera Demberg

Abstract

A wide body of evidence shows that human language processing difficulty is predicted by surprisal—a word's negative log probability in context. However, as Large Language Models (LLMs) grow in size and achieve lower perplexity, their ability to predict human reading times (RTs) paradoxically decreases. This "scaling paradox" suggests a mismatch between machine-estimated probabilities and human cognitive expectations. In this paper, we investigate whether temperature scaling can bridge this gap. We find that applying a temperature parameter ($T \approx 2.5$) to soften LLM predictions significantly improves their fit to human reading times (up to 89% improvement in $\Delta$LLH). However, our analysis reveals that this improvement is primarily driven by how models handle subword tokenization, suggesting the benefit may be more of a mathematical artifact than a reflection of human cognitive processing.

Introduction

In psycholinguistics, the surprisal of a word—its negative log probability given its context—is a robust predictor of processing difficulty, typically measured via reading times. Intuitively, a better language model should provide better estimates of surprisal. Yet, recent research has uncovered a scaling paradox: as models become more accurate at predicting text (lower perplexity), their correlation with human reading behavior often degrades.

One prominent hypothesis is that LLMs are "overconfident." Trained on massive datasets, they may assign much higher probabilities to rare words than a human would. This study tests whether re-calibrating these models using temperature scaling can improve their psycholinguistic predictive power.

Methodology

We employ temperature scaling, a post-processing technique that modifies the "logits" (raw scores) before the final softmax layer by a parameter $T$. A $T > 1$ flattens the distribution, making the model less certain and increasing the surprisal of high-probability words.

We evaluated the GPT-2 family (Small, Medium, Large, and XL) across three major reading time corpora:

  1. Dundee Corpus: Eye-tracking data from newspaper editorials.
  2. Natural Stories Corpus: Self-paced reading times for narrative texts.
  3. Brown Corpus: Self-paced reading times for diverse English prose.

Predictive power was quantified using delta log-likelihood ($\Delta$LLH), representing the improvement in model fit when surprisal is added to a baseline of simple predictors like word length and frequency.

Results

The Universal Optimal Temperature

Across all models and corpora, we observed a consistent peak in predictive power at a temperature of approximately $T \approx 2.5$. As shown in Figure 1, this softening of the probability distribution leads to a dramatic increase in $\Delta$LLH compared to the default $T=1$.

Predictive power (ΔLLH) as a function of scaled temperature T for GPT-2 models across three corpora.

Applying the optimal temperature improved the predictive power by 23-43% on the Dundee corpus and a staggering 60-89% on the Natural Stories corpus. Interestingly, larger models (XL) did not necessarily require different temperatures than smaller ones, but they often showed higher absolute improvements.

The Subword "Smoking Gun"

While the fit improvements are massive, the "right reasons" for this improvement remain questionable. Our analysis shows that the benefit of temperature scaling is almost entirely concentrated in words composed of multiple subword tokens.

Comparison of ΔLLH for single vs. multiple token words across datasets, showing the effect is driven by multi-token words.

As illustrated in Figure 2, the $\Delta$LLH for single-token words (dotted lines) often peaks early or even declines with higher temperatures, whereas the fit for multiple-token words (dashed lines) continues to improve or remains high. This suggests that temperature scaling might be "fixing" an inherent flaw in how we aggregate subword probabilities (summing the surprisal of parts) rather than uncovering a deep truth about human cognition.

Discussion

Our findings present a nuanced view of LLM calibration. While larger LLMs are actually better calibrated to objective word probabilities than smaller ones, they remain "overconfident" relative to human expectations. However, the fact that the benefit is tied to tokenization suggests that our current methods of calculating surprisal from subword-based models are problematic.

Words split into many tokens (often rare words or named entities) have their surprisal values pushed into a more cognitively plausible range by temperature scaling. This implies that the scaling is compensating for the mathematical artifact of summing surprisals across sub-lexical units, which may not be how humans process complex words.

Conclusion

Temperature scaling is a powerful tool for improving the fit of LLM surprisal to human reading times, consistently peaking at $T \approx 2.5$. However, the discovery that this effect is driven by multi-token words suggests that researchers should be cautious in interpreting these results as a direct reflection of human "overconfidence." Future work should investigate whether these effects persist in whole-word models or if alternative methods of token-to-word surprisal aggregation can resolve the scaling paradox without manual temperature tuning.

Summary updated

Abstract

A wide body of evidence shows that human language processing difficulty is predicted by the information-theoretic measure surprisal, a word’s negative log probability in context. However, it is still unclear how to best estimate these probabilities needed for predicting human processing difficulty -- while a long-standing belief held that models with lower perplexity would provide more accurate estimates of word predictability, and therefore lead to better reading time predictions, recent work has shown that for very large models, psycholinguistic predictive power decreases. One reason could lie in the fact that language models might be more confident of their predictions than humans, because they have had exposure to several magnitudes more data. In this paper, we test what effect temperature-scaling of large language model (LLM) predictions has on surprisal estimates and their predictive power of reading times of English texts. Firstly, we show that calibration of large language models typically improves with model size, i.e., poorer calibration cannot account for poorer fit to reading times. Secondly, we find that temperature-scaling probabilities lead to a systematically better fit to reading times (up to 89\% improvement in delta log likelihood), across several reading time corpora. Finally, we show that this improvement in fit is chiefly driven by words that are composed of multiple subword tokens.
BibTeX
@inproceedings{liu2024temperature,
  author = {Tong Liu and Iza Škrjanec and Vera Demberg},
  title = {{TEMPERATURE-SCALING SURPRISAL ESTIMATES IMPROVE FIT TO HUMAN READING TIMES – BUT DOES IT DO SO FOR THE “RIGHT REASONS”?}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.519},
}
Figure