Introduction: Bridging Dependency and Constituency Parsing
In natural language processing, sentence structure is typically represented through either dependency parsing (word-to-word relationships) or constituency parsing (hierarchical phrase groupings). While both offer valuable syntactic information, they are often treated as distinct tasks requiring different modeling approaches.
This paper explores "spinal trees"—a hybrid representation where every word is associated with a "spine" (a vertical sequence of phrase labels) and dependencies connect these spines. Historically, parsing these structures required complex, hand-engineered models. The authors propose a neural transition-based system that simplifies this process using Stack-LSTMs.
Methodology: Transition-Based Spinal Parsing
The proposed system extends the arc-standard transition system and the Stack-LSTM architecture. The core innovation is the introduction of a new transition operation:
- The
node(n)Operation: Unlike standard dependency parsers that only useshift,left-arc, andright-arc, this model addsnode(n). This allows the parser to incrementally build a word's spine by adding non-terminal nodes (e.g., NP, VP) to the stack. - Dual Composition: The model employs two types of composition functions. Arc Composition merges head and dependent representations when a dependency is formed, while Node Composition recursively merges existing spine representations with new phrase labels.
- Head-Driven Construction: The tree is built in a head-outwards order, ensuring that phrase internal structures are established relative to their central words.
Key Insights: The Power of LSTMs
A significant finding of this research is that Stack-LSTMs appear to be "head-agnostic." Traditionally, choosing the "head" of a phrase required sophisticated linguistic rules (e.g., Stanford Dependencies). However, the authors found that using simple structural rules—such as always selecting the leftmost word as the head—yielded comparable or even superior performance. This suggests that the LSTM's internal states are robust enough to induce useful syntactic representations regardless of the specific linguistic theory used for training.
Furthermore, the incremental building of spines (step-by-step rather than atomic assignment) allows the parser to utilize partial structural information during dependency decisions. Ablation studies confirmed that "node composition" is essential, as its removal led to a significant drop in F1 scores.
Experimental Results
The model was evaluated on the Penn Treebank (PTB). It achieved highly competitive results, performing on par with dedicated dependency parsers while simultaneously providing constituency information.
As shown in the results table, the model maintains consistent performance (F1 ~90.5, UAS ~93.1) across various head-selection strategies. This transition-based approach also offers high efficiency, operating in linear time relative to the number of nodes.
Limitations and Future Directions
While successful, the current implementation has areas for improvement:
- Dependency Labels: The model currently predicts structure but not dependency labels (e.g., subject vs. object).
- Search Strategy: The parser uses a greedy search; implementing beam search could further improve accuracy.
- Unary Constraints: A hard limit was placed on consecutive
node(n)operations to prevent infinite loops during training, suggesting a need for more elegant handling of unary productions.
In conclusion, this work demonstrates that a neural transition-based parser can effectively unify dependency and constituency parsing through spinal trees, leveraging the representational power of Stack-LSTMs to overcome the need for complex, hand-coded linguistic rules.
Summary updated