Probing Linguistic Systematicity

Emily Goodwin, Koustuv Sinha, Timothy J. O'Donnell

Abstract

Recently, there has been much interest in the question of whether deep natural language understanding models exhibit systematicity; generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear. There is accumulating evidence that neural models often generalize non-systematically. We examined the notion of systematicity from a linguistic perspective, defining a set of probes and a set of metrics to measure systematic behaviour. We also identified ways in which network architectures can generalize non-systematically, and discuss why such forms of generalization may be unsatisfying. As a case study, we performed a series of experiments in the setting of natural language inference (NLI), demonstrating that some NLU systems achieve high overall performance despite being non-systematic.

Introduction

While modern Natural Language Understanding (NLU) models achieve high accuracy on standard benchmarks, a critical question remains: do these models truly understand the underlying rules of logic, or are they merely sophisticated pattern-matchers? This paper investigates systematicity—the ability of a model to apply word meanings consistently across novel contexts. In linguistics, systematicity is a core component of compositionality: if a model understands "John loves the girl," it should inherently understand "The girl loves John" by virtue of knowing the individual words and grammatical rules.

Methodology: Probing with Natural Logic

To rigorously test for systematicity, the authors employ Natural Logic, a framework that maps linguistic relationships to set-theoretic operations. This allows for precise definitions of entailment, contradiction, and equivalence.

Natural logic relations and their set-theoretic definitions used for probing.

The study utilizes an artificial language with a fixed template (e.g., Quantifier + Modifier + Noun + Modifier + Negation + Verb) to isolate logical reasoning from other linguistic complexities. Three specific "Jabberwocky" probes were designed:

  1. Known Word Perturbation: Swapping "closed-class" words (e.g., changing all to some) in sentences the model previously handled correctly.
  2. Identical Open-class Words: Testing pairs with identical nouns and verbs (e.g., "All blickets wug" vs. "Some blickets don't wug") to see if the model recognizes logical contradictions regardless of the specific vocabulary.
  3. Consistency Probe: Reversing the premise and hypothesis to check if the model maintains logical consistency (e.g., if A entails B, then B must reverse-entail A).

Four architectures were evaluated: Bidirectional GRU (BGRU), Infersent, a Self-Attentive encoder (SATT), and a Hierarchical Convolutional Network (CONV).

Key Findings

The Jabberwocky Reveal

Models that achieved ~95% accuracy on standard holdout sets (known words in new combinations) saw their performance plummet when faced with the Jabberwocky probes. For instance, on the "Identical Open-class Words" probe, the BGRU's accuracy on negation dropped to approximately 55%. Negation proved to be a significant weak point; for the SATT model, accuracy on negation-related probes was as low as ~23%. This suggests that models rely on brittle, context-dependent heuristics rather than robust abstract rules.

Contextual Sensitivity and Variance

A striking finding was the high variance in performance across different "blocks" of lexical items. A model might correctly process the logic for one set of novel words (e.g., "blickets") but fail for another (e.g., "toves"). This indicates that models learn "contextually conditioned" meanings, where the interpretation of a logical operator like "all" changes based on the adjacent noun.

Standard deviation of model performance across different lexical contexts (blocks) for small and large training sets.

Failure of Scale

Increasing the training data by nearly tenfold (the "large" condition) did not resolve these issues. As shown in the variance plots, models continued to exhibit high standard deviations across contexts, failing to achieve the level of systematicity seen in the training distribution.

Conclusion

The results suggest that standard neural architectures lack the necessary inductive biases to learn true linguistic systematicity. Even top-performing models often achieve success through local heuristics that do not generalize to novel, logically identical scenarios. Future research should explore architectural modifications that encourage models to treat logical operators as truly abstract, context-independent units.

Summary updated

Abstract

Recently, there has been much interest in the question of whether deep natural language understanding models exhibit systematicity; generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear. There is accumulating evidence that neural models often generalize non-systematically. We examined the notion of systematicity from a linguistic perspective, defining a set of probes and a set of metrics to measure systematic behaviour. We also identified ways in which network architectures can generalize non-systematically, and discuss why such forms of generalization may be unsatisfying. As a case study, we performed a series of experiments in the setting of natural language inference (NLI), demonstrating that some NLU systems achieve high overall performance despite being non-systematic.
BibTeX
@inproceedings{goodwin2020probing,
  author = {Emily Goodwin and Koustuv Sinha and Timothy J. O'Donnell},
  title = {{Probing Linguistic Systematicity}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  url = {https://openreview.net/forum?id=usVPYZYIWqK},
}
Figure