When a large language model (LLM) correctly identifies that "the keys to the cabinet are on the table," does it understand the hierarchical rule of subject-verb agreement, or is it simply noticing that "keys" and "are" frequently appear near each other? In natural language, syntax (structure) and semantics (meaning) are deeply intertwined, making it difficult to tell if a model is a sophisticated grammarian or merely a statistical pattern matcher relying on "semantic shortcuts."
To solve this, researchers use syntactic isolation: a suite of techniques designed to decouple the structural rules of language from the meanings of individual words. By stripping away the "common sense" of a sentence, we can see if the underlying grammatical machinery still functions.
The Nonce Sentence and Jabberwocky Probing
The most common way to isolate syntax is to present models with "nonce" sentences—strings that are grammatically perfect but semantically nonsensical. This approach traces back to Noam Chomsky’s famous example, "Colorless green ideas sleep furiously."
Gulordava et al. pioneered this in neural networks by replacing content words in natural sentences with random words that share the same morphological features (e.g., replacing one plural noun with another). If a model can still predict the correct verb agreement in these nonsensical strings, it suggests the model has learned abstract hierarchical structures rather than just memorizing word pairs.
A more extreme version of this is Jabberwocky probing. As Maudslay et al. describe, this involves replacing content words with phonologically plausible pseudowords (like "the slithy toves did gyre"). This method ensures that the model cannot rely on any pre-existing lexical associations. Their findings suggest that many "syntactic probes" used to evaluate LLMs are actually "cheating" by using semantic cues; when tested on Jabberwocky sentences, the performance of these probes often drops significantly.
Identifying Lexical Biases
Why does isolation matter? Because models often default to "lexical heuristics"—simple rules of thumb based on specific words. Lasri demonstrated that BERT’s apparent mastery of agreement is brittle; when tested on nonce sentences where semantic cues are removed, the model’s performance collapses, revealing a heavy reliance on local word patterns.
Similarly, Wei et al. found that even when models learn abstract rules, they are heavily biased by word frequency. If a model sees a frequent word form, it may default to that form regardless of what the grammar requires. By using semantically empty sentences, researchers can force the model to rely exclusively on learned grammatical rules rather than these associative memories.
Probing the "Shape" of Internal Representations
Syntactic isolation isn't just for testing accuracy; it helps us see how models organize information internally. Arps et al. used nonce datasets to prove that Transformers like RoBERTa encode full constituency parse trees in their hidden layers. Because these trees can be extracted even from nonsensical sentences, the authors argue that this structural knowledge is a distinct, independent component of the model's representation.
This geometric perspective is further explored by Viswanathan et al., who used a block-shuffling algorithm to progressively disrupt syntax and semantics. They found that the "geometric complexity" of a model's internal tokens changes based on linguistic meaning, independent of raw statistical frequency.
Learning from a Single Example
Isolation also allows researchers to study how models acquire new linguistic rules. Thrush et al. adapted psycholinguistic "novel word learning" experiments to BERT. By masking all content words except for a brand-new, "novel" verb, they isolated the model's ability to generalize syntactic rules (like the dative shift) from a single exposure. This proves that the model's architecture is capable of "one-shot" syntactic generalization, even if it usually relies on the crutch of familiar semantics.
Beyond Text: Human and Machine Comparisons
The drive to isolate syntax extends to cognitive science. Hashemzadeh et al. found that the internal representations of LSTMs correlate with human brain activity (EEG) even when both are processing Jabberwocky sentences. This suggests that the "structural engine" of neural networks may share fundamental similarities with the human brain's own mechanisms for processing language without meaning.
By stripping away the "what" of a sentence, syntactic isolation reveals the "how"—the underlying structural skeleton that allows both humans and machines to navigate the infinite complexity of language.
Key Foundational Work
The VerbCorner Project: Toward an Empirically-Based Semantic Decomposition of Verbs (2013) — contributed 1 concept elements to this direction.
Computational Modelling of Structural Priming in Dialogue (2006) — contributed 1 concept elements to this direction.