Fusing Constituency and Dependency Structures

188 concepts138 papers1 proposal

To understand a sentence, humans naturally track two distinct structural dimensions: how words group together into hierarchical phrases, and how individual words functionally relate to one another. In computational linguistics, these two dimensions are captured by different formalisms. Constituency parsing models the sentence as a nested hierarchy of phrasal blocks (e.g., noun phrases, verb phrases), while dependency parsing maps directed, functional links between head words and their modifiers (e.g., subject, object).

Historically, natural language processing (NLP) models have treated these two representations as mutually exclusive choices. A researcher would select one grammar type based on the task at hand, discarding the alternative view. However, a growing body of research demonstrates that relying on a single syntactic perspective creates a structural bottleneck. By fusing constituency and dependency representations into a unified, multi-view framework, models can leverage the complementary strengths of both to resolve deep structural ambiguities.

The Complementary Nature of the Two Maps

Why does combining these two formalisms work? The core intuition lies in their orthogonal strengths. Constituency syntax excels at boundary detection—defining the exact spans of text that form logical units. Dependency syntax excels at relational mapping—identifying long-range functional connections between words, regardless of how far apart they sit in a sentence.

Consider a task like Semantic Role Labeling (SRL), which aims to discover "who did what to whom." As Fei et al. observe, an SRL model must solve two sub-problems: identifying the boundaries of semantic arguments and assigning their specific roles relative to a predicate. Constituency trees provide the clean phrasal boundaries needed to isolate arguments, while dependency trees provide the direct word-to-word links needed to assign roles. Forcing a model to use only one representation means it must either guess boundaries from sparse dependency links or infer functional relations from complex phrasal hierarchies.

Constituency (Phrasal Spans):
[The quick brown fox] [jumped over the lazy dog]
       └───── Noun Phrase ─────┘ └────────── Verb Phrase ──────────┘

Dependency (Functional Relations):
       ┌─────── subject ──────┐
[The] [quick] [brown] [fox] [jumped] [over] [the] [lazy] [dog]
                       └─────── modifier ───────┘

This synergy extends across many structural tasks. In information extraction and sentiment analysis, Fei et al. note that constituency structures prevent models from breaking up multi-word entities, while dependency structures allow opinions to link to targets across long, clause-heavy sentences. By integrating both views, the models establish a mutual fault-tolerance mechanism: if an automated parser introduces an error in the dependency graph, the constituency tree can often constrain the search space and prevent downstream failures, and vice versa.

Architectures for Hybrid Syntactic Fusion

To exploit these complementary signals, researchers have developed several distinct strategies to merge heterogeneous structures into neural networks.

Graph-Level Fusion

One of the most direct ways to combine these views is to construct a unified graph where words, phrases, and structural relations coexist. Xu et al. introduced a Graph-to-Sequence framework that builds a single syntactic graph containing linear word order, dependency edges, and constituency hierarchies. By running Graph Neural Networks (GNNs) over this hybrid topology, the model learns representations that simultaneously account for local spans and global relationships. Similarly, Pang et al. use hybrid graph convolutional networks to define a "structural scope," filtering out irrelevant words in aspect-level sentiment analysis by intersecting phrasal boundaries with dependency paths.

Joint and Dual-Tree Networks

Rather than merging the graphs upfront, another approach processes the two structures in parallel, allowing them to cross-pollinate during representation learning. Meng et al. propose a Multi-Task Dual-Tree Network where separate encoders ingest constituency and dependency trees, using cross-attention layers to align phrasal boundaries with relational links. In parsing itself, Strzyz demonstrates that framing parsing as sequence labeling across both representations simultaneously improves performance, as the phrase-grouping logic of constituency parsing inherently guides word-to-word dependency decisions.

Heterogeneous Tree Distillation

While multi-view models achieve high accuracy, running multiple external parsers during inference is computationally expensive and prone to latency bottlenecks. To circumvent this, Fei et al. introduced a "Mimic and Conquer" strategy using Knowledge Distillation (KD).

During training, a sequential student model (such as an LSTM) is trained to simultaneously mimic the hidden structural distributions of two separate "teacher" models—one trained on constituency trees and one on dependency trees. This allows the student model to internalize both phrasal and relational biases within its standard sequential layers. During inference, the external parsers are discarded entirely, giving the model the speed of a simple sequence model but the accuracy of a hybrid syntactic architecture.

Extending the Paradigm to Semantics

The success of hybrid syntactic modeling has led researchers to extend the paradigm beyond pure grammar, fusing formal syntax with semantic structures like Semantic Role Labeling (SRL) graphs and abstract word clusters.

For instance, Marcheggiani and Titov demonstrated that stacking syntactic dependency biases with explicit semantic predicate-argument structures inside GCNs provides non-redundant benefits for machine translation. While syntax enforces grammatical constraints, the semantic layer prevents the model from misattributing actions to the wrong participants during translation.

Furthermore, the field is actively exploring how these explicit, symbolic trees interact with the implicit syntax learned by large pre-trained language models. As Liu points out, a major frontier involves unifying the soft, statistical attention patterns inherent in Transformer heads with the rigid, formal constraints of dependency and constituency trees, creating a more robust foundation for deep linguistic reasoning.

Key Foundational Work

A Machine Translation System From Japanese Into English - Another Perspective of MT Systems - (1980) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hybrid_syntactic_representation_cccf7d59,
  title = {hybrid syntactic representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hybrid-syntactic-representation/rdr_cccf7d59aaa347f4b1814ada9f886eee},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_hybrid_syntactic_representation_cccf7d59_v1,
  title = {hybrid syntactic representation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hybrid-syntactic-representation/rdr_cccf7d59aaa347f4b1814ada9f886eee/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake