Introduction
Transformer language models (TLMs) have demonstrated remarkable capabilities across various natural language processing tasks. However, they frequently struggle with negation, often failing to grasp simple logical contradictions. Traditional NLP research typically treats negation as a monolithic logical operator. In contrast, developmental psychology suggests that human children acquire negation skills in a hierarchical fashion, mastering specific functional categories like "rejection" or "existence" at different stages. This study investigates whether TLMs mirror this developmental hierarchy or if they treat negation as a uniform linguistic phenomenon.
Methods and Data
The researchers framed negation reasoning as a Natural Language Inference (NLI) task. In this setup, a model must determine if a premise sentence entails a hypothesis sentence. To analyze negation developmentally, the authors constructed the Developmental Negation Corpus by identifying seven distinct categories within the SNLI and MNLI datasets using a rule-based parser.
The categories include:
- Possession (PO): "You don't have the right temperatures."
- Existence (EX): "There is no link."
- Labeling (L): "It is not orders."
- Prohibition (PR): "Don't run into the..."
- Inability (I): "I could not pick out..."
- Epistemic (EP): "I don't know why."
- Rejection (R): "I didn't want to be overheard."
Due to the rarity of certain categories like Labeling and Prohibition, WordNet-based data augmentation was employed to ensure a minimum of 1,500 examples per category. The study evaluated standard models like BERT and RoBERTa alongside "child-scale" models like BabyBERTa, which are trained on significantly smaller, child-directed datasets.
Key Results
The experimental results reveal that negation is not a monolithic skill for TLMs. Performance varied significantly across categories, with models generally excelling at Labeling (L) and Possession (PO) while struggling with Rejection (R) and Existence (EX).
Learning Curves and Model Scale
As shown in the learning curves, standard models reached high accuracy quickly on simpler categories but plateaued or fluctuated on more complex ones. Notably, BabyBERTa—the model trained on the least amount of data—showed a unique advantage in the Rejection (R) category. This aligns with human developmental patterns, where rejection is among the first forms of negation mastered by children, suggesting that the nature of training data (child-directed vs. general web text) is a critical factor in how models process negation.
Transferability and Sequential Learning
Further experiments into transfer learning indicated that negation skills are largely "orthogonal." Training a model on one category (e.g., Inability) rarely improved performance on another (e.g., Epistemic). This suggests that models are learning specific syntactic patterns rather than a generalized logic of negation. Sequential training also failed to produce a "eureka" moment; instead, models often exhibited catastrophic forgetting, where performance on previously learned categories dropped as new ones were introduced.
Discussion and Future Work
The findings suggest that current TLMs do not possess a unified understanding of negation. Instead, they acquire fragmented, category-specific linguistic patterns. While smaller models trained on child-directed data show some developmental alignment, the lack of transfer between categories remains a significant hurdle for achieving human-like logical reasoning.
Future research should focus on refining rule-based parsers to capture more nuanced negation structures and exploring mixed-category training regimes to better simulate natural linguistic environments.
Summary updated