Abstract
Anders Søgaard. Proceedings of the ACL 2010 Conference Short Papers.
2010.
Semi-supervised learning for Part-of-Speech (POS) tagging has historically struggled to consistently outperform supervised baselines due to the complexity of sequence models. This paper introduces a simplified approach that reframes sequence labeling as a classification task through "stacking." By combining this framework with a novel "tri-training with disagreement" algorithm, the author achieves a 4.21% error reduction on the Wall Street Journal corpus. Notably, the disagreement-based variant proves exceptionally efficient, requiring significantly less unlabeled data than standard tri-training.
Introduction
Part-of-Speech (POS) tagging is a fundamental task in Natural Language Processing. While supervised models trained on large corpora like the Wall Street Journal (WSJ) achieve high accuracy, the cost of manual annotation remains a barrier. Semi-supervised learning—leveraging small amounts of labeled data alongside large unlabeled datasets—offers a potential solution. However, complex sequence models like HMMs or CRFs often suffer from error propagation when trained on noisy, automatically labeled data, sometimes performing worse than their supervised counterparts. This work seeks a more robust, simpler method to harness unlabeled data without the negative results that have plagued the field for decades.
Methodology
The proposed system rests on two primary innovations:
1. Stacking: Tagging as Classification
To avoid the pitfalls of semi-supervised sequence modeling, the author employs "stacking." The process involves running a supervised tagger (SVMTool) and an unsupervised clustering algorithm (Unsupos) on the text. The outputs (predicted tags and cluster IDs) are then used as features for a second-stage meta-classifier. This transformation reduces the problem from sequence prediction to a standard classification task, which is more amenable to robust semi-supervised techniques.
2. Tri-training with Disagreement
Tri-training typically involves three classifiers ($c_1, c_2, c_3$). In standard tri-training, if $c_2$ and $c_3$ agree on a label for an unlabeled instance, that instance is added to the training set for $c_1$. The author introduces a "disagreement" constraint: an instance is only added to $c_1$’s training set if $c_2$ and $c_3$ agree and $c_1$ currently predicts a different label. To ensure the necessary diversity among classifiers, they are trained on different bootstrap samples of the labeled data. This approach focuses the learning process on instances where the model is likely to improve.
Results and Discussion
The effectiveness of these methods was evaluated using the WSJ portion of the Penn Treebank for labeled data and the Brown corpus for unlabeled data.
As shown in Table 4, the "tri-training with disagreement" (tri-disagr) method achieved the highest accuracy, resulting in a 4.21% error reduction over the SVMTool baseline. The system improved baseline accuracy from 97.15% to 97.27%. The approach also proved effective for a MaxEnt tagger, demonstrating its generalizability across different base learners.
Efficiency of Disagreement
A key finding of the study is the extreme efficiency of the disagreement-based selection compared to standard tri-training.
While standard tri-training incorporated over 1.1 million tokens from the unlabeled corpus, the disagreement variant achieved comparable or better performance by adding only 173 tokens on average (Table 3). This suggests that the vast majority of unlabeled data is redundant in a semi-supervised context and that targeting model weaknesses is a far more efficient strategy.
Conclusion
This paper demonstrates that semi-supervised POS tagging can be made both simple and effective by treating it as a classification problem. The "tri-training with disagreement" method provides a significant boost in accuracy while being remarkably data-efficient. This "off-the-shelf" framework makes it easier to incorporate other types of unsupervised features, such as word embeddings or gazetteers, into the tagging process in the future.
Summary updated