Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers

James A. Michaelov, Benjamin K. Bergen

Abstract

How well do language models deal with quantification? In this study, we focus on ‘few’-type quantifiers, as in ‘few children like toys’, which might pose a particular challenge for language models because the sentence components without the quantifier are likely to co-occur, and ‘few’-type quantifiers are rare. We present 960 English sentence stimuli from two human neurolinguistic experiments to 22 autoregressive transformer models of differing sizes. Not only do all the models perform poorly on ‘few’-type quantifiers, but overall the larger the model, the worse its performance. This inverse scaling is consistent with previous work suggesting that larger models increasingly reflect online rather than offline human processing, and we argue that the decreasing performance of larger models may challenge uses of language models as the basis for natural language systems.

Introduction

Quantifiers are small words that carry massive weight. Compare "Most sharks are dangerous" to "Few sharks are dangerous." The logic of the sentence flips entirely based on one word. The authors investigate whether Large Language Models (LLMs) actually "understand" these logical constraints or if they are simply playing a game of statistical word association. Specifically, they look at few-type quantifiers (e.g., few, rarely, almost no). These are tricky because they are relatively rare in training data, and the words that follow them often contradict the most common statistical patterns. For instance, in the sentence "Few squirrels gather ____," a model might be tempted to predict "nuts" because squirrels and nuts are so frequently linked, even though the quantifier "few" makes that statement factually unlikely. The core question is one of scaling: as models get bigger and "smarter," do they get better at handling this logic, or does their increased sensitivity to common patterns make them even more prone to error?

Methodology

The researchers adopted a methodology from psycholinguistics, specifically following the work of Ettinger (2020). They tested 22 different autoregressive transformer models, including the GPT-2, GPT-3, GPT-Neo, OPT, and InstructGPT series, ranging from 117 million to 175 billion parameters. The models were presented with 960 sentence stimuli derived from human neurolinguistic studies (Urbach and Kutas, 2010). These stimuli used a 2x2 design: Quantifier Type ("Most-type" vs. "Few-type") and Typicality (Typical vs. Atypical completions). The researchers measured the surprisal (the negative log probability) of the final word. A "correct" prediction meant the model assigned a higher probability to the logically appropriate word. They also measured sensitivity: how much the model's prediction changed when the quantifier was swapped.

Key Results

The study found a striking divergence in performance based on quantifier type. Most-type accuracy was excellent across the board, generally increasing with model size from 85% to 97%. In contrast, few-type accuracy was dismal, with most models scoring below 20%. Crucially, as models grew larger, this accuracy consistently decreased, marking a clear instance of inverse scaling.

Figure 1: Performance metrics across model series and sizes, showing the divergence between most-type and few-type accuracy.

Discussion: The Typicality Trap and Inverse Scaling

Larger models are more "well-read," meaning they have a stronger grasp of common associations. Ironically, this makes them worse at few-type quantifiers. They become so convinced that "squirrels gather nuts" that they ignore the "few" at the start of the sentence. This "typicality trap" leads to the observed inverse scaling.

Figure 4: Detailed view of inverse scaling in few-type accuracy as model size increases.

This phenomenon mirrors human "online" processing. In humans, the N400 brain response shows that our brains initially struggle with "few-type" quantifiers in real-time (online) before we eventually "correct" our understanding (offline). LLMs seem to be stuck in that initial, association-heavy "online" phase. The authors argue this is a sign of "outer misalignment": models are getting better at their training objective (predicting the next word based on statistics), but that objective is leading them away from the desired behavior of logical reasoning.

Conclusion

The findings suggest that standard scaling and even fine-tuning on human feedback (InstructGPT) may not be enough to fix deep-seated statistical biases regarding rare logical constructs. While some research suggests inverse scaling might eventually "flip" at extreme scales (e.g., PaLM 540B), the current results highlight a significant challenge for building safer, more reliable natural language systems that require robust logical reasoning.

Summary updated

Abstract

How well do language models deal with quantification? In this study, we focus on ‘few’-type quantifiers, as in ‘few children like toys’, which might pose a particular challenge for language models because the sentence components with out the quantifier are likely to co-occur, and ‘few’-type quantifiers are rare. We present 960 English sentence stimuli from two human neurolinguistic experiments to 22 autoregressive transformer models of differing sizes. Not only do all the models perform poorly on ‘few’-type quantifiers, but overall the larger the model, the worse its performance. This inverse scaling is consistent with previous work suggesting that larger models increasingly reflect online rather than offline human processing, and we argue that the decreasing performance of larger models may challenge uses of language models as the basis for natural language systems.
BibTeX
@inproceedings{michaelov2023rarely,
  author = {James A. Michaelov and Benjamin K. Bergen},
  title = {{Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=WoGFPOm0yl},
}
Figure