Abstract
Jason Wei, Dan Garrette, Tal Linzen, Ellie Pavlick. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.
2021.
Introduction
A central debate in artificial intelligence centers on whether neural networks are "stochastic parrots" that memorize statistical patterns or "symbolic reasoners" capable of learning abstract rules. Subject-Verb Agreement (SVA) serves as a primary test for this capability. For example, in the sentence "The gorp adds nothing," a speaker knows "adds" is correct because "gorp" is singular, regardless of whether they have encountered the word "gorp" before. While BERT has shown proficiency in SVA, it was previously unclear if this success was due to abstract rule acquisition or the memorization of specific noun-verb pairs encountered during extensive pre-training.
Methodology
To investigate the relationship between training frequency and rule learning, the researchers conducted a controlled study:
- Training from Scratch: The team trained multiple BERT-Base models from scratch using a 2.3-billion-token version of English Wikipedia.
- Frequency Manipulation: They identified 60 "Verbs of Interest" (VOIs) and systematically altered their absolute frequency (total occurrences) and relative frequency (the ratio between singular and plural forms) in the training data.
- Evaluation Stimuli: Models were tested on 23,298 natural Wikipedia sentences and 7.5 million "nonce" sentences—grammatically correct but semantically nonsensical strings (e.g., "The sheepdog of basic needs providers review a damaging effect"). Nonce sentences ensure the model relies on syntax rather than semantic heuristics.
- Diagnostic Probing: Probing classifiers were used to determine if the model correctly identified the grammatical number of the subject and verb independently of the final prediction.
Key Results
The study suggests that BERT is a "Symbolic Learner with Noisy Observations." It successfully learns abstract rules but struggles to apply them when lexical features are poorly represented due to low frequency.
Generalization and Frequency Thresholds
BERT generalizes remarkably well to noun-verb pairs that never appeared together in the training data, achieving an 8.8% error rate on unseen pairs compared to 3.3% for seen pairs. However, performance is highly dependent on frequency. The model typically requires seeing a verb between 10 and 100 times to reliably infer its singular or plural features. Below this threshold, error rates rise as the model fails to categorize the word's grammatical number.
The Impact of Skew
Relative frequency, or "skew," creates significant bias. When one verb form (e.g., "adds") is much more frequent than its counterpart ("add"), the model develops a strong prior. In nonce sentences where the competing verb form was 16x more frequent than the target, the error rate reached 62.5%, as the model effectively "guesses" the more common word form.
Discussion and Insights
Probing analysis reveals that most errors on rare words stem from feature misclassification—the model fails to identify the word's number—rather than a failure to apply the SVA rule. This supports the hypothesis that the model possesses the rule (Subject Number = Verb Number) but lacks clear lexical representations for infrequent words.
Limitations and Future Work
This research focuses exclusively on Subject-Verb Agreement; other syntactic rules may show different sensitivities to frequency. Additionally, the study does not fully distinguish between errors caused by lexical misidentification versus structural parsing failures. A key remaining question is why BERT requires dozens of examples to learn features that humans can often acquire from a single instance. Future work could explore how different architectures or morphological signals might improve this sample efficiency.
Summary updated