When we read or listen to a sentence, words arrive one after another in a linear stream. Yet, human language is fundamentally non-linear. Beneath the surface sequence lies a geometric architecture: words group into phrases, and phrases nest inside one another to form hierarchical trees.
A central debate spanning computational linguistics, cognitive science, and neuroscience asks: Does the mind, or a neural network, process language by tracking sequential transitions, or by constructing abstract hierarchical trees?
Understanding why hierarchical representations matter—and how researchers prove their existence—requires looking at where linear models fail, how the human brain responds to structure, and how artificial networks internalize grammar.
The Failure of Linear Proximity
To understand why language requires a hierarchical representation rather than a flat sequence, consider the phenomenon of agreement attraction errors. In the sentence, "The girl near the boys likes climbing," the main verb likes must agree with the singular subject girl. However, the plural noun boys sits immediately next to the verb in the linear sequence.
If the brain processed language purely as a sequence of local word-to-word transitions, the proximity of boys would frequently trick it into using the plural verb like. In Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models, Christos-Nikolaos Zacharopoulos et al. demonstrate that while humans maintain high structural accuracy by relying on the sentence's underlying tree geometry, standard language models often collapse to near coin-flip performance when distracting nouns are linearly close. This reveals that human language navigation relies on structural depth rather than surface proximity.
Cognitive and Neurobiological Evidence for Trees
How do we know the human cognitive system actively builds these abstract trees? Evidence comes from both behavioral patterns and direct brain imaging.
Behavioral Echoes and Expectations
One powerful window into the mind's machinery is structural priming—the unconscious tendency to reuse grammatical patterns we have recently heard or spoken. In A Review of the Syntactic Priming—A Research Method in Sentence Production, Luyao Chen et al. synthesize decades of research showing that priming effects diminish or vanish when the underlying syntactic hierarchy differs, even if the surface word-category sequence remains identical. This priming is remarkably abstract; David Reitter et al. show in Priming Effects in Combinatory Categorial Grammar that priming operates across different levels of the hierarchy, allowing a single word's grammatical type to prime an entire phrase.
Furthermore, when humans reconstruct degraded or noisy speech, their internal expectations are guided by these abstract geometries. Stephan C. Meylan and Sathvik Nair demonstrate in Evaluating models of robust word recognition with serial reproduction that human linguistic "priors" are best modeled by hierarchical phrase grammars rather than flat word-to-word transitions.
Mapping the Brain's Geometry
Neuroimaging allows us to see these structural representations in real time. By converting complex syntactic trees into multi-dimensional embeddings, Aniketh Janardhan Reddy and Leila Wehbe in Can fMRI reveal the representation of syntactic structure in the brain? show that fMRI scans capture the subtle geometry of sentence trees distributed across the language network, independent of cognitive parsing effort.
This neural tracking is highly specialized. SUBBA REDDY OOTA et al. show in How does the brain process syntactic structure while listening? that specific regions like the temporal lobe and angular gyrus rely on distinct syntactic formalisms (such as constituency and dependency trees) to explain brain activity beyond what semantics alone can account for. This tracking requires genuine structural assembly: Amelia Burroughs and Nina Kazanina use EEG frequency tagging in Grammatical category and the neural processing of phrases to prove that the brain's response reflects abstract phrase construction rather than the mere repetition of word categories.
How Neural Networks Internalize Hierarchy
Given that human language relies on hierarchical trees, do artificial neural networks naturally learn this structure from raw text, or do they require explicit structural constraints?
Emergent vs. Explicit Structure
Standard sequential models like LSTMs do not have built-in tree structures, yet they can perform surprisingly well on syntax tasks. To explain why, Yair Lakretz et al. peeked inside these models in The emergence of number and syntax units in LSTM language models and discovered that they spontaneously develop specialized "syntax units." These internal neurons act as gates that track long-distance grammatical dependencies.
However, relying on emergent properties has limits. Adhiguna Kuncoro et al. demonstrate in LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better that explicitly incorporating a top-down tree construction order provides a superior inductive bias, significantly outperforming purely sequential models on complex grammar tasks.
Tracking the Derivation
To see exactly how modern architectures build these representations, Taiga Someya et al. introduced a method called Derivational Probing in Derivational Probing: Unveiling the Layer-wise Derivation of Syntactic Structures in Neural Language Models. They found a stark architectural divide: masked language models like BERT construct syntax from the bottom up (assembling local phrases before global structure), whereas autoregressive models like GPT-2 develop local and global structures in parallel.
When models are designed to induce their own trees without supervision—such as the Tree Transformer evaluated by Lingling Zhou et al. in Tree Transformer's Disambiguation Ability of Prepositional Phrase Attachment and Garden Path Effects—they successfully mimic human-like surprise when encountering linguistic traps like garden-path sentences, even if their absolute parsing accuracy remains lower than supervised models.
Aligning Models to Human Brain Activity
The ultimate validation of a model's internal representation is how well it mirrors human neurophysiology.
The foundation of this comparison rests on "surprisal"—a measure of how unexpected a word is given its context. Stefan L. Frank et al. established in Word surprisal predicts N400 amplitude during reading that computational word surprisal directly predicts the amplitude of the brain's electrical N400 response, providing a metric to test competing linguistic frameworks.
[Input Text Stream] ---> [Sequential Model] ---------> Weak Alignment with EEG
---> [Hierarchical Parser] ------> Strong Alignment with EEG (P600/N400)
When models are explicitly structured around hierarchy, their alignment with the brain improves dramatically. John T. Hale et al. demonstrated in Finding syntax in human encephalography with beam search that Recurrent Neural Network Grammars (RNNGs)—which use hierarchical parsing—can successfully predict specific neurophysiological markers of syntax, such as the P600 response.
Crucially, this alignment cannot be achieved simply by scaling up flat data. In Text Genre and Training Data Size in Human-like Parsing, John T. Hale et al. show that scaling training data only improves a model's alignment with human EEG if the model architecture explicitly incorporates hierarchical phrase structures. This structural advantage extends to memory retrieval; Shinnosuke Isono et al. show in If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation? that hierarchical syntactic structures predict human reading speeds significantly better than flat token sequences, indicating that the brain's internal memory retrieval mechanism is optimized for grammatical geometry.
The Universality of Structural Priors
If hierarchical representations are essential for language, where do they come from? Classic linguistic theory suggests humans possess a language-specific cognitive engine dedicated to recursion. However, recent computational work challenges this boundaries.
In Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models, Isabel Papadimitriou demonstrated that neural models can learn abstract structural rules from non-linguistic hierarchical data—such as MIDI music or Java code—and successfully transfer that knowledge to improve their natural language processing. This suggests that the preference for hierarchy may stem from a generalized structural capacity rather than a language-specific faculty.
Furthermore, this complex machinery may not need to be fully formed at birth. Marten van Schijndel and Micha Elsner argue in Bootstrapping into Filler-Gap: An Acquisition Story that language learners can bootstrap their way into understanding complex long-distance dependencies using simpler, non-hierarchical mechanisms before full syntactic tree representations mature.
Ultimately, whether looking at the firing of neurons in the temporal lobe or the activation of attention heads in a Transformer, the evidence converges: language is not merely a sequence of words, but a dynamic geometry of nested trees. Models that embrace this hierarchy achieve a deeper, more human-like mastery of text.
Key Foundational Work
Sequential vs. Hierarchical Syntactic Models of Human Incremental Sentence Processing (2012) — contributed 1 concept elements to this direction.
Priming Effects in Combinatory Categorial Grammar (2006) — contributed 1 concept elements to this direction.