When we evaluate how difficult a sentence is to read or process, our first instinct is often to look at surface-level statistics: how many words it contains, or how rare those words are. However, surface metrics fail to capture the true architecture of language. Consider two sentences of identical length: one might be a straightforward sequence of independent clauses, while the other might contain a tangled web of nested relative clauses and distant word relationships.
To truly quantify linguistic difficulty, researchers look beyond surface statistics to the geometric and topological properties of a sentence's grammatical structure. By parsing sentences into hierarchical trees—such as constituency or dependency parses—we can measure the physical layout of grammar. This structural approach explains not only why certain texts strain human working memory, but also why they present distinct computational challenges for artificial neural networks.
The Dimensions of Structural Complexity
Structural complexity can be broadly mapped along two geometric axes within a syntactic parse tree: the vertical axis of hierarchical depth, and the horizontal axis of linear dependency distance.
Vertical Complexity: Hierarchical Depth and Embedding
The vertical axis measures how deeply clauses and phrases are nested within one another. When a sentence branches into multiple subordinate clauses, it increases the "embeddedness" of the text.
[Main Clause]
└── [Subordinate Clause]
└── [Relative Clause]
Schwarm et al. and Mac Namee et al. demonstrated that maximum parse tree height and clausal embeddedness serve as highly robust indicators of reading difficulty, significantly outperforming traditional readability formulas like Flesch-Kincaid. Each additional layer of nesting requires a reader to maintain an unresolved grammatical context in memory. Liu et al. similarly leveraged the maximum depth of child phrases to estimate the cognitive processing load for language learners, showing that hierarchical depth tracks comprehension difficulty more accurately than simple vocabulary-based metrics.
Horizontal Complexity: Dependency Distance and Locality
The horizontal axis measures the linear distance between words that share a grammatical relationship (a head and its dependent). For example, in the phrase "The cat that was black sat," the subject ("cat") and the verb ("sat") are separated by an intervening relative clause.
According to Dependency Locality Theory, human language processing is constrained by working memory; holding a head word in mind while waiting for its dependent incurs a measurable cognitive cost. Juzek et al. showed that scientific English has evolved over 300 years to systematically minimize this dependency distance, streamlining text for expert communication. When dependencies span long distances or cross one another—creating "non-projective" structures—the processing effort spikes. Kunilovskaya et al. utilized these linear distances and non-projective edge crossings to detect "translationese," demonstrating that the subtle cognitive friction of the translation process leaves a distinct structural fingerprint in dependency trees.
Phrasal Density vs. Clausal Nesting
Complexity is not monolithic; authors can complicate sentences in different ways. Ridley et al. formalized this by distinguishing between phrasal density (the number of modifiers packed into a single noun phrase) and clausal complexity (the hierarchical depth of dependency trees). A sentence can be dense without being deeply nested, or deeply nested without containing dense noun phrases. This distinction allows researchers to profile stylistic variations across different linguistic backgrounds and academic domains.
Why It Works: The Cognitive and Computational Mirror
Why do these graph-theoretic metrics align so well with actual difficulty? The answer lies in a shared constraint between human cognitive architectures and computational models: both must process hierarchical information sequentially.
The Human Processing Cost
Human eye-tracking data confirms that our brains track these structural metrics in real time. Sarti et al. and Brunato et al. demonstrated that while human readers might perceive a sentence as difficult due to surface features (such as heavy punctuation or numerical values), eye-tracking data reveals that long-distance dependency links are the true drivers of actual processing delays.
Recognizing this, Reichart et al. proposed using the depth of nested grammatical structures as a proxy for annotator fatigue. By filtering out sentences with high memory loads from annotation pipelines, they balanced model performance with human sustainability.
The Machine Processing Mirror
This structural friction is mirrored inside neural language models. The authors of Higher-order Comparisons of Sentence Encoder Representations discovered that sentences with high syntactic depth cause the greatest geometric divergence—or "disagreement"—between the internal layers of models like BERT.
Simple Sentence ──> [Layer 1] ──(High Agreement)──> [Layer 12]
Complex Sentence ──> [Layer 1] ──(Divergence)─────> [Layer 12]
This structural complexity is so fundamental to sequence processing that it transcends natural language. In computer science, the structural complexity of source code is measured via Abstract Syntax Tree (AST) depth. Beau et al. showed that AST depth captures the true difficulty of a coding task for large language models far better than code length or external dependencies. Remarkably, pre-training language models on datasets with high structural nesting (like deep ASTs in code) fundamentally unlocks downstream logical inference and instruction-following capabilities in natural language, as demonstrated by Uchiyama et al..
Applications Across Domains
Because structural metrics capture the underlying cognitive load of language, they serve as powerful tools across a wide range of practical applications:
- Dialogue and Interaction: In conversation, speakers dynamically adjust their structural complexity to align with one another over time (Xu et al.). When conversations become syntactically complex, the high integration cost triggers explicit grounding signals (like verbal handshakes) to ensure comprehension, particularly when visual cues are absent (Tietze et al.).
- Clinical Diagnostics: Automated syntactic complexity measurement is proving to be a vital non-invasive biomarker in healthcare. Lukic et al. demonstrated that tracking specific markers of hierarchical complexity—such as the production of subordinating conjunctions in connected speech—allows machine learning models to accurately differentiate between variants of Primary Progressive Aphasia, mapping directly to localized patterns of brain atrophy.
- Parser Optimization: The computational cost of parsing a sentence is fundamentally driven by the discrepancy between its surface word order and its underlying functional dependency structure. By quantifying this topological mismatch, parsers can better constrain their search spaces and improve accuracy across languages with highly flexible word orders (Gerdes et al.; Havelka).
By shifting our perspective from what words are used to how those words are structurally anchored, syntactic complexity measurement provides a universal framework for understanding how both human minds and machines navigate the architecture of expression.
Key Foundational Work
Computational Analysis of Predicational Structures in English (1980) — contributed 2 concept elements to this direction.
A Computational Theory of Processing Overload and Garden-Path Effects (1990) — contributed 1 concept elements to this direction.