Schrödinger's tree - On syntax and neural language models

Artur Kulmizev, Joakim Nivre

TL;DR

This paper is a critical "state of the union" for research into how neural language models (LMs) handle syntax. The authors argue that our current understanding is a paradox—a "Schrödinger’s Tree"—where syntax appears simultaneously present and absent because researchers often conflate surface-level patterns (like word order) with abstract hierarchical structure.

What Problem Does This Paper Address?

Since the "deep learning revolution" of the mid-2010s, NLP has moved away from hand-coded linguistic features toward massive, opaque "black box" models like BERT and GPT. While these models are incredibly fluent, we don't fully understand how they process language.

Researchers have spent years trying to determine if these models "know" syntax (the hierarchical rules of language). However, the results are often contradictory: some studies show models are syntactic geniuses, while others show they can solve complex tasks even when the word order is completely scrambled. Kulmizev and Nivre address this lack of conceptual clarity, providing a framework to resolve these conflicting findings.

Methods

The paper is a meta-analysis and position paper. It categorizes the existing body of research into three dominant paradigms:

  1. Targeted Syntactic Evaluation (TSE): Testing model behavior using "minimal pairs" (e.g., checking if a model prefers "The keys... are" over "The keys... is").
  2. Probing: Attempting to "extract" syntactic trees directly from the model's internal mathematical representations (hidden states).
  3. NLU Evaluation: Assessing how much a model relies on syntax to solve downstream tasks like the GLUE benchmark, often by corrupting the input (shuffling words) to see if performance drops.

The authors introduce a vital conceptual distinction: Coding Properties (surface-level cues like word order, morphology, and function words) vs. Syntactic Structure (the abstract, hierarchical relationship between words).

Datasets

The authors review studies that utilize several major benchmarks and datasets:

  • GLUE / SuperGLUE: Standard benchmarks for general natural language understanding.
  • BLiMP (Benchmark of Linguistic Minimal Pairs): A large-scale dataset for TSE.
  • SyntaxGym: A platform for targeted evaluation of language models.
  • Universal Dependencies (UD) & Surface-Syntactic UD (SUD): Treebanks used for probing internal representations to see which "style" of syntactic tree a model prefers.

Key Insights and Ideas

  • Coding Properties $\neq$ Syntax: High performance on a subject-verb agreement task doesn't mean a model has "learned grammar." It might just be sensitive to a specific "coding property." In English, word order is a dominant coding property; in Finnish, it's morphological case.
  • The NLU Paradox: Models often retain 70-90% of their performance on NLU tasks even when the input is a "word salad" (shuffled tokens). This suggests that many current benchmarks can be solved using lexical heuristics (keyword spotting) rather than true syntactic reasoning.
  • Formalism Matters: When "probing" a model for trees, the results change depending on the linguistic theory used to define the tree (e.g., UD vs. SUD). The model's "internal tree" might look different depending on the "lens" the researcher uses to look at it.
  • The Three Research Questions: The authors argue researchers often confuse three distinct questions:
  • Does the model learn syntax? (Observation)
    1. Can the model learn syntax? (Capacity)
    2. Does the model need syntax to solve the task? (Necessity)

Key Results

The paper synthesizes several striking findings from the literature:

  • Robustness to Corruption: Studies (like Sinha et al. 2021) show that models pre-trained on scrambled text and fine-tuned on normal text suffer negligible performance loss on GLUE, calling into question how much "syntax" is actually required for modern NLU.
  • Probing Success: Conversely, structural probes (like Hewitt & Manning 2019) can successfully recover complex dependency trees from BERT's embeddings, suggesting the information is there, even if the model doesn't always use it for the final task.
  • The "Schrödinger" Effect: Syntax is "alive" when we look for it with probes, but "dead" when we test if the model actually needs it to solve downstream tasks.

Limitations and Future Work

  • English-Centricity: Most of the reviewed research focuses on English. The authors urge for more typologically diverse studies, as different languages rely on different coding properties (like inflectional morphology) to signal syntax.
  • The Trap of Aggregate Metrics: Relying on a single "accuracy" score (like a BLiMP score) hides crucial variations. A model might be great at simple agreement but fail at complex nested dependencies.
  • Call for "Faithful" Explanations: Future work needs to move beyond just seeing if syntax is present and start proving that the model's behavior is actually caused by that syntactic knowledge.

Summary updated

Abstract

In the last half-decade, the field of natural language processing (NLP) has undergone two major transitions: the switch to neural networks as the primary modeling paradigm and the homogenization of the training regime (pre-train, then fine-tune). Amidst this process, language models have emerged as NLP's workhorse, displaying increasingly fluent generation capabilities and proving to be an indispensable means of knowledge transfer downstream. Due to the otherwise opaque, black-box nature of such models, researchers have employed aspects of linguistic theory in order to characterize their behavior. Questions central to syntax -- the study of the hierarchical structure of language -- have factored heavily into such work, shedding invaluable insights about models' inherent biases and their ability to make human-like generalizations. In this paper, we attempt to take stock of this growing body of literature. In doing so, we observe a lack of clarity across numerous dimensions, which influences the hypotheses that researchers form, as well as the conclusions they draw from their findings. To remedy this, we urge researchers make careful considerations when investigating coding properties, selecting representations, and evaluating via downstream tasks. Furthermore, we outline the implications of the different types of research questions exhibited in studies on syntax, as well as the inherent pitfalls of aggregate metrics. Ultimately, we hope that our discussion adds nuance to the prospect of studying language models and paves the way for a less monolithic perspective on syntax in this context.
BibTeX
@article{kulmizev2022schrodinger,
  author = {Artur Kulmizev and Joakim Nivre},
  title = {{Schrödinger's tree - On syntax and neural language models}},
  journal = {Frontiers Artif. Intell.},
  year = {2022},
  doi = {10.3389/frai.2022.796788},
}

Loading related content...

Figure