Abstract
Recently, there has been much interest in the question of whether deep natural language understanding (NLU) models exhibit systematicity, generalizing such that units like words make consistent contributions to the meaning of the sentences in which they appear. There is accumulating evidence that neural models do not learn systematically. We examine the notion of systematicity from a linguistic perspective, defining a set of probing tasks and a set of metrics to measure systematic behaviour. We also identify ways in which network architectures can generalize non-systematically, and discuss why such forms of generalization may be unsatisfying. As a case study, we perform a series of experiments in the setting of natural language inference (NLI). We provide evidence that current state-of-the-art NLU systems do not generalize systematically, despite overall high performance.
Introduction
In human language, the ability to understand a sentence like "John loves the girl" implies an automatic understanding of "The girl loves John." This property, known as systematicity, allows us to compose meanings from individual parts in a consistent, rule-based manner. While modern deep learning models achieve near-human performance on standard benchmarks, there is a growing suspicion that they rely on statistical shortcuts rather than a true grasp of linguistic logic. This paper investigates whether high-performing Natural Language Inference (NLI) models are truly systematic or if they are merely "cheating" using local heuristics.
Methodology: The Jabberwocky Probe
To test for systematicity, the authors employ a "Jabberwocky" probing strategy. They created an artificial language with a strict grammar, distinguishing between:
- Closed-class words: Logical operators like all, some, no, and not, which should maintain a fixed, systematic meaning across all contexts.
- Open-class words: Nouns and verbs, which are replaced with novel, nonsense words (e.g., "blickets," "wug") that the model has never encountered during training.
If a model has truly learned the systematic meaning of "all," its performance should remain stable regardless of whether it is paired with "pigs" or "blickets." The study evaluates four architectures: BGRU, Infersent (INFS), Self-Attentive (SATT), and a Hierarchical ConvNet (CONV), across two data scales (Small and Large).
Results: High Accuracy, Low Understanding
The results reveal a stark contrast between benchmark performance and systematic understanding. While all models achieved over 93% accuracy on standard holdout tests with known words, their performance plummeted and became highly volatile when faced with Jabberwocky probes.
As shown in Figure 1, the "logic" applied by these models is heavily dependent on the specific nonsense words used. The high variance in accuracy across different "blocks" of nonsense words suggests that models assign different "meanings" to logical operators based on the surrounding context—a clear violation of systematicity.
The Failure of Attention and the Limits of Scale
One of the most significant findings is the failure of the Self-Attentive (SATT) model. On probes requiring the model to rely solely on logical operators (where premise and hypothesis use identical open-class words), SATT's performance dropped to approximately 23.71%. This suggests that attention-based models may rely heavily on word-overlap heuristics that fail when logical structures are shuffled.
Furthermore, increasing the dataset size from "Small" to "Large" did little to alleviate these issues. As seen in Figure 5, while raw accuracy might improve slightly, the models remain brittle and sensitive to context. Negation, in particular, remained the most difficult logical operator for all models to handle systematically.
Conclusion
The evidence suggests that current state-of-the-art NLU systems lack the necessary inductive bias to treat language as a compositional, symbolic system. High accuracy on standard benchmarks does not guarantee a robust understanding of logic. To build truly human-like NLU, future research must focus on engineering architectures that treat closed-class logical units as invariant constants across all linguistic contexts.
Summary updated