Abstract
Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules. We investigate this question using the case study of BERT's performance on English subject-verb agreement. Unlike prior work, we train multiple instances of BERT from scratch, allowing us to perform a series of controlled interventions at pre-training time. We show that BERT often generalizes well to subject-verb pairs that never occurred in training, suggesting a degree of rule-governed behavior. We also find, however, that performance is heavily influenced by word frequency, with experiments showing that both the absolute frequency of a verb form, as well as the frequency relative to the alternate inflection, are causally implicated in the predictions BERT makes at inference time. Closer analysis of these frequency effects reveals that BERT's behavior is consistent with a system that correctly applies the SVA rule in general but struggles to overcome strong training priors and to estimate agreement features (singular vs. plural) on infrequent lexical items.
Introduction
A central debate in natural language processing (NLP) is whether neural language models (LMs) learn abstract, symbolic rules or merely act as sophisticated lookup tables memorizing word co-occurrences. While models like BERT excel at syntactic tasks such as subject-verb agreement (SVA), their training on massive, static corpora makes it difficult to determine if they are applying learned rules or simply recalling specific noun-verb pairs seen during pre-training. This study addresses this by pre-training BERT models from scratch on manipulated datasets to isolate the causal effects of word frequency on rule learning.
Methodology
To investigate how frequency impacts rule learning, we evaluate three hypotheses:
- Idealized Symbolic Learner: Applies rules over abstract categories without frequency effects.
- Item-Specific Learner: Memorizes co-occurrences and fails on unseen pairs.
- Symbolic Learner with Noisy Observations: Learns rules but errs when mapping specific words to categories (e.g., misidentifying the number of a rare word).
We pre-trained multiple BERT models, manipulating the absolute frequency (total occurrences) and relative frequency (ratio of singular to plural forms) of 60 "Verbs of Interest" (VOIs). Evaluation was conducted on a natural Wikipedia dataset and a novel nonce sentence dataset. Nonce sentences (e.g., "The cities of chaperones play a long history") are grammatically correct but semantically meaningless, ensuring the model relies on syntax rather than semantic context.
Results and Discussion
Our findings suggest that BERT is not merely a memorization engine. It generalizes remarkably well to subject-verb pairs that never appeared together in the training data. As shown below, error rates for unseen pairs (frequency 0) are only marginally higher than for frequently seen pairs, strongly supporting the rule-governed behavior hypothesis over simple item-specific memorization.
However, this rule-governed behavior is significantly modulated by frequency. Absolute frequency is critical for feature learning; probing classifiers indicate that SVA failures on rare words often stem from a failure to encode the word's "number" feature (singular vs. plural) rather than a failure of the SVA rule itself.
Furthermore, relative frequency acts as a powerful prior that can override rule application. When one verb form is significantly more frequent than its counterpart in training, the model tends to default to the more frequent form regardless of the syntactic requirement. This effect is most pronounced in nonce sentences where semantic cues are absent.
These results align best with the "Noisy Symbolic Learner" hypothesis. BERT learns abstract SVA rules, but its lexical parsing is noisy and highly susceptible to frequency imbalances in the training data.
Conclusion
BERT demonstrates a capacity for abstract syntactic reasoning, yet its performance is constrained by lexical frequency. While it generalizes to novel pairs, its ability to correctly identify grammatical features is bottlenecked by absolute frequency, and its rule application can be overridden by strong statistical priors from relative frequency. Future research should explore how model architecture or scale might improve the sample efficiency of feature learning and the robustness of rule application against frequency biases.
Summary updated