Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Charles Yu, Ryan Sie, Nico Tedeschi, Leon Bergen

Abstract

Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models' accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun's performance on grammatical tasks. Finally, we find that a novel noun's grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.

Introduction

Neural language models (LMs) have achieved remarkable proficiency in capturing the nuances of human grammar. However, their performance is not uniform. They frequently exhibit "lexical variation," where a model might correctly handle a grammatical task like subject-verb agreement for one noun (e.g., "the cat walks") but fail for another (e.g., "the attorney walks").

A long-standing intuition in the field, the Frequency Hypothesis, posits that this variation is driven by exposure: models should perform better on words they have encountered more frequently during training. This study rigorously tests this hypothesis across multiple architectures and grammatical tasks.

Methodology

The researchers utilized a minimal pair methodology to evaluate three prominent models: Transformer-XL, GPT-2, and BERT. A minimal pair consists of two sentences differing only by a single grammatical feature (e.g., "The cat walks" vs. "*The cat walk"). A model "passes" if it assigns a higher probability to the grammatically correct sentence.

The study calculated performance scores for hundreds of animate nouns across 10 distinct tasks, including:

  • Subject-Verb Agreement (SVA): Tested in simple contexts, across relative clauses, and across prepositional phrases.
  • Reflexive Anaphora (RA): Ensuring reflexive pronouns (e.g., "himself", "themselves") agree with their antecedents.

To measure exposure, the authors calculated word frequencies from the models' respective training corpora: WikiText-103 for Transformer-XL and BERT, and OpenWebText for GPT-2.

The Frequency Paradox

The most significant finding of this research is the near-total lack of correlation between a noun's frequency in the training data and the model's grammatical accuracy regarding that noun. Across four orders of magnitude of frequency, the correlation coefficient ($r$) remained near zero.

Figure 4: Model performance on grammatical tasks plotted against training corpus frequency for Transformer-XL, BERT, and GPT-2. The flat regression lines indicate that frequency explains less than 0.1% of the variance in performance.

This "Frequency Paradox" suggests that even for words seen millions of times, models do not reach a ceiling of grammatical perfection. Conversely, words seen only a hundred times are handled with similar average accuracy.

Systematic Variation Across Models

While frequency does not explain the variation, the variation itself is not random. The study found that noun performance is highly correlated across different tasks and even across different model architectures. If a noun is "difficult" for GPT-2 in a subject-verb agreement task, it is likely also difficult for BERT and Transformer-XL.

Figure 3: Pairwise comparisons of model performance across various grammatical tasks. The strong positive correlations suggest that different language models share systematic "blind spots" for specific nouns.

Principal Component Analysis (PCA) revealed that a single factor explains nearly half of the variance in task performance, indicating that "grammatical knowledge" for a specific noun is a stable, underlying trait shared by these models. In fact, 24 out of 30 comparisons between different LMs showed significant positive correlations in how they ranked noun difficulty.

Few-Shot Learning and the Mystery of Excess Variation

To further investigate the acquisition of grammar, the researchers introduced "novel" words into the models' vocabularies. They found that models could learn a new word's grammatical "number" (singular vs. plural) from as few as five examples. This learning occurred not only through direct syntactic cues but also through semantic context (e.g., "The wug worked all alone" implying singularity).

This efficiency deepens the paradox: if grammatical rules are so easily learned from a handful of examples, high-frequency nouns should theoretically exhibit near-perfect performance. The persistence of "hard" nouns despite massive exposure remains an unexplained mystery in neural linguistic representation, potentially pointing toward issues like catastrophic forgetting or inherent architectural limitations.

Conclusion

This work challenges the assumption that data volume is the primary driver of grammatical mastery in language models. The discovery of frequency-independent, cross-model systematic variation suggests that some nouns possess inherent properties that make them difficult for current neural architectures to process grammatically. Future research is needed to identify these properties and determine if this phenomenon extends to other parts of speech or more complex linguistic structures.

Summary updated

Abstract

Neural language models learn, to varying degrees of accuracy, the grammatical properties of natural languages. In this work, we investigate whether there are systematic sources of variation in the language models' accuracy. Focusing on subject-verb agreement and reflexive anaphora, we find that certain nouns are systematically understood better than others, an effect which is robust across grammatical tasks and different language models. Surprisingly, we find that across four orders of magnitude, corpus frequency is unrelated to a noun's performance on grammatical tasks. Finally, we find that a novel noun's grammatical properties can be few-shot learned from various types of training data. The results present a paradox: there should be less variation in grammatical performance than is actually observed.
BibTeX
@inproceedings{yu2020word,
  author = {Charles Yu and Ryan Sie and Nico Tedeschi and Leon Bergen},
  title = {{Word Frequency Does Not Predict Grammatical Knowledge in Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=RjyNBLXLf},
}
Figure