Abstract
Most current dependency parsers presuppose that input words have been morphologically disambiguated using a part-of-speech tagger before parsing begins. We present a transition-based system for joint part-of-speech tagging and labeled dependency parsing with non-projective trees. Experimental evaluation on Chinese, Czech, English and German shows consistent improvements in both tagging and parsing accuracy when compared to a pipeline system, which lead to improved state-of-the-art results for all languages.
Introduction
In traditional Natural Language Processing, tasks are often handled as a pipeline: first, a part-of-speech (POS) tagger labels each word (e.g., identifying "book" as a noun or a verb), and then a parser uses those tags to build a tree of grammatical dependencies.
The fundamental problem with this approach is error propagation. If the tagger makes a mistake, the parser—which treats those tags as ground truth—is almost certain to fail. This is particularly problematic in "richly inflected" languages like Czech or German where word forms are highly ambiguous and morphology and syntax are deeply intertwined. This paper introduces a transition-based system that jointly performs POS tagging and labeled dependency parsing for both projective and non-projective trees, allowing the model to resolve ambiguities that a standalone tagger might miss.
Methodology
The system builds upon a transition-based parsing framework, where a configuration (representing the current state of the stack and buffer) is transformed into a parse tree through a sequence of actions.
Key Innovations
- Joint Transitions: The standard
SHIFTtransition is replaced withSHIFT_p. When a word is moved from the buffer to the stack, the system simultaneously assigns it a POS tag $p$. - Non-Projectivity: By including the
SWAPtransition, the system handles non-projective dependencies (where arcs cross), which are common in many languages but difficult for simpler transition-based parsers. - Beam Search & Global Learning: Instead of making greedy decisions, the system uses beam search to track multiple hypotheses. It employs a structured perceptron with a passive-aggressive update rule to learn weights globally.
- Hash Kernel: To handle the massive feature space created by joint modeling without excessive memory usage, the authors use a hash kernel to map features to weights, providing a significant speedup.
Key Insights: Syntactic Context for Tagging
The core advantage of joint modeling is that the parser's structural knowledge helps resolve POS ambiguity. For example, if the parser sees a word that could be a noun or a verb, but the surrounding structure strongly suggests a predicate is needed, the joint model can correctly tag it as a verb.
As shown in the error analysis for German (Figure 6), the joint model significantly reduces confusion between similar forms, such as infinitive (VVINF) and finite verbs (VVFIN), by leveraging the syntactic requirement for a predicate.
Results
The system was evaluated on several major benchmarks, including the CoNLL 2009 Shared Task (Chinese, Czech, English, and German) and the Penn Treebank (WSJ).
The joint system consistently outperformed pipeline baselines across all tested languages:
- Accuracy: The model achieved state-of-the-art results, reaching 93.38% Unlabeled Attachment Score (UAS) for English (WSJ).
- Tagging Gains: POS tagging accuracy improved significantly; in Chinese, POS accuracy rose by 0.60% compared to the baseline in development settings.
- Core Functions: Error analysis showed the biggest improvements were in identifying core grammatical functions like subjects and objects.
- Efficiency: Despite the complexity of joint modeling, the system remained practical, parsing at a rate of approximately 0.4 seconds per sentence on standard hardware.
Conclusion
While joint modeling was expected to primarily benefit morphologically rich languages like Czech, the authors found it also improved results for "configurational" languages like English and Chinese. This suggests that POS ambiguity is a universal bottleneck in parsing. Future work may explore more fine-grained morphological joint models to handle features like gender, number, or case in even more complex languages.
Summary updated