Introduction: The Challenge of Error Propagation
Pipelined syntactic parsers are a staple of natural language processing, but they suffer from a critical vulnerability: error propagation. In a typical pipeline, a Part-of-Speech (POS) tagger first identifies word categories, and a parser then builds a grammatical structure based on those tags. If the tagger makes a single mistake, the parser is forced to build its structure on a flawed foundation, leading to significant drops in overall accuracy.
While integrated models that perform both tasks simultaneously exist, they are often computationally expensive and difficult to adapt to new domains. This research explores a middle ground: keeping the flexibility of a pipelined system while mitigating error propagation through "ambiguous tagging."
Methods: From Single to Ambiguous Tagging
The authors propose allowing the POS tagger to output multiple weighted candidates rather than a single "best" guess. They evaluated three primary strategies:
single: The traditional approach, outputting only the most probable tag.multi: Outputting a set of candidate tags, treating them as equally likely.prob: Outputting a set of candidate tags, each weighted by its probability.
To maintain efficiency, a threshold parameter ($\theta$) is used to prune tags with low relative probability, preventing the parser from being overwhelmed by unlikely possibilities. These probabilities are then integrated into a Head-Driven Phrase Structure Grammar (HPSG) parser.
Key Insights: The Role of Probabilities
A critical finding was that simply providing more options (multi) is insufficient and can even degrade performance if the parser cannot distinguish between likely and unlikely tags. The prob strategy, which provides the parser with the confidence level for each tag, was the clear winner. This approach allows the parser to weigh different structural interpretations against the likelihood of the underlying POS tags.
Key Results: Accuracy and Domain Portability
The researchers tested their approach on the Penn Treebank (general news) and the GENIA corpus (biomedical abstracts).
General Accuracy
On the standard Penn Treebank test set, the ambiguous prob tagger improved the parser's F1 score from 84.10 to 85.06. As shown in the performance trends, the prob strategy consistently outperforms the single baseline across various training data sizes, narrowing the gap toward "gold" (perfect) tagging performance.
Domain Portability
The most significant impact was seen in domain adaptation. Moving a parser from news text to specialized biomedical text is notoriously difficult. By simply swapping in an ambiguous POS tagger trained on medical data (prob-BIO), the parser's F1 score on the GENIA corpus jumped from 76.66 to 82.02. This result was remarkably close to the "gold" performance (within 0.36 points), suggesting that ambiguous tagging effectively solves the POS bottleneck for domain portability without requiring a complete retraining of the complex parser.
Conclusion
Ambiguous POS tagging offers a powerful, computationally efficient way to improve the robustness of syntactic parsers. By passing weighted probabilities rather than rigid labels, the system preserves the flexibility of pipelined architectures while achieving accuracy and portability levels previously reserved for more complex, integrated models. Future work may focus on incorporating sequential dependencies between tags to further refine the probability distributions provided to the parser.
Summary updated