Parsing natural language involves a fundamental tension between structural complexity and categorical precision. To determine the meaning of a sentence, a model must solve two problems simultaneously: identifying the hierarchical skeleton (who is connected to whom) and assigning labels to those connections (what is the nature of the relationship).
When these tasks are performed jointly, the search space grows exponentially. A parser must consider every possible tree structure multiplied by every possible combination of labels. Research across dependency, constituency, and discourse parsing suggests that the most efficient and often most accurate way to navigate this space is to decouple these two concerns into a sequential pipeline.
The Structural Skeleton as a Constraint
The core intuition behind two-stage parsing is that the "where" of a syntactic relationship is often easier to determine than the "what." By first identifying the unlabeled topology of a sentence, a system can establish a structural scaffold that drastically narrows the possibilities for subsequent labeling.
McDonald et al. et al. demonstrated this by separating the search for a Maximum Spanning Tree (structure) from the assignment of syntactic labels. This decoupling allows the labeler to leverage the full context of the predicted tree—information that would be computationally impossible to include if the labels were being decided while the tree was still being built. Similarly, Stern et al. et al. found that scoring labels and spans independently in neural constituency parsing simplifies the architecture without sacrificing accuracy, treating them as distinct classification tasks.
This modularity is particularly vital for unsupervised grammar induction. Klein and Manning argued that treating "constituent-ness" as a binary property allows models to focus on structural coherence before categorizing phrases. Reichart et al. et al. extended this by showing that separating hierarchical bracketing from label assignment nearly doubled the performance of unsupervised systems, as it prevents the model from getting lost in the high dimensionality of simultaneous structural and categorical search.
Divide and Conquer for Long Sequences
The benefits of two-stage parsing become even more pronounced as sentences grow in length or complexity. In long sequences, the number of potential dependencies increases quadratically, often leading to a "drop-off" in accuracy for traditional single-pass parsers.
To combat this, many researchers employ a "divide and conquer" strategy. Ohno et al. et al. tackled the winding nature of spoken monologues by first resolving internal dependencies within small segments and then establishing links between those segments to form a global structure. For Chinese, which often features long "run-on" sentences, Hu et al. et al. and Tang et al. et al. used punctuation as natural split points to decompose the search space, preventing the exponential growth of structural ambiguity.
This hierarchical approach is also the standard for discourse parsing. Feng and Hirst et al. and Joty et al. et al. decompose document-level analysis into intra-sentential parsing (local trees) and multi-sentential parsing (global structure). This allows the model to apply different features and complexity levels to local grammatical relations versus global rhetorical organization.
Bypassing Computational Complexity
Decoupling is not just an accuracy play; it is a necessity for using expressive but expensive formalisms. Lexicalized Tree Adjoining Grammars (LTAGs), for instance, are linguistically powerful but traditionally require $O(n^7)$ time to parse. Corro et al. et al. bypassed this by first generating an unlabeled dependency tree and then applying a labeling algorithm to transform it into an LTAG tree in linear time.
Similarly, Harbusch proposed separating the context-free "backbone" of a grammar from its complex adjoining operations. By filtering out invalid structures using efficient algorithms in the first stage, the system only applies computationally expensive operations to a small set of viable candidates.
The Trade-off of Independence
While decoupling simplifies the pipeline, it introduces a risk: the "labeling" stage is entirely dependent on the "structural" stage being correct. Teng et al. et al. noted that while separating these tasks simplifies the model, it may prevent the system from exploiting mutual dependencies—where knowing a label might actually help clarify the structure.
However, the recurring success of these two-stage systems suggests that the computational and representational gains of modularity generally outweigh the risks of error propagation. By treating structure and labeling as sequential problems, researchers can build parsers that are both linguistically rich and computationally viable.