Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Charles Yu, Ryan Sie, Nico Tedeschi, Leon Bergen

Introduction

Neural language models (LMs) like BERT and GPT-2 exhibit impressive grammatical capabilities. However, their performance is not uniform; they often judge some sentences as more grammatical than others, even when the underlying rules are identical. This study investigates whether this variation is systematic and if it stems from word frequency in training data—a common assumption in human language acquisition.

Methodology

The researchers utilized a minimal pair methodology, comparing model probabilities for grammatically correct versus incorrect sentence pairs (e.g., "The cat walks" vs. "*The cat walk").

The evaluation focused on:

  1. Subject-Verb Agreement (SVA) and Reflexive Anaphora (RA).
  2. Noun-Centric Scoring: Performance was calculated for individual nouns across 500 templates.
  3. Few-Shot Learning: Models were fine-tuned on five example sentences containing novel "nonsense" words (e.g., wug) to assess how quickly grammatical properties are acquired.

The study analyzed pre-trained versions of Transformer-XL, GPT-2, and BERT, with frequency counts derived from their respective training corpora (WikiText-103, WebText, etc.).

The Frequency Paradox

The most significant finding is the near-total lack of correlation between a noun's frequency in the training corpus and the model's grammatical performance with that noun. Whether a noun appears 100 or 1,000,000 times, the model's ability to handle its agreement rules remains essentially unchanged.

Model performance shows almost zero correlation with word frequency across Transformer-XL, BERT, and GPT-2.

Statistically, frequency explains less than 0.1% of the variation in performance. This suggests that the "more data is better" mantra does not apply to the mastery of specific lexical grammatical properties once a basic threshold is met.

Systematic Difficulty Across Architectures

Interestingly, noun performance is highly correlated across different tasks and model architectures. If GPT-2 struggles with the grammar of a specific noun, BERT and Transformer-XL are likely to struggle as well.

Pairwise comparisons show that grammatical performance on specific tasks is highly correlated between different model architectures.

This consistency suggests that certain nouns possess inherent properties—perhaps semantic or structural—that make them "grammatically slippery" for current neural network architectures, regardless of the training objective or data scale.

Learnability and Few-Shot Performance

Despite these persistent "blind spots," the study found that grammar is remarkably "cheap" to learn. In few-shot experiments, models reached performance levels on novel words similar to real-world items after seeing just five examples. Furthermore, models demonstrated "semantic bootstrapping," inferring a noun's number (singular/plural) from semantic context (e.g., "The wug worked all alone") rather than just explicit grammatical markers.

Conclusion

This research challenges the assumption that exposure frequency is the primary driver of grammatical mastery in LMs. While models exhibit systematic difficulties with certain words, these properties are easily learned from minimal data. The central mystery remains: why do models continue to struggle with certain high-frequency words in their original training sets if the rules are so easily acquired? Future work is needed to identify the true source of this performance variation.

Summary updated

Abstract

Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models' accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun's performance on grammatical tasks. Finally, we find that a novel noun's grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.
BibTeX
@inproceedings{yu2020word,
  author = {Charles Yu and Ryan Sie and Nico Tedeschi and Leon Bergen},
  title = {{Word Frequency Does Not Predict Grammatical Knowledge in Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=RjyNBLXLf},
}
Figure