Abstract
Statistical parsing typically requires large amounts of manually annotated data, which is expensive and time-consuming to produce. This paper explores the application of co-training—a semi-supervised learning algorithm—to the task of statistical parsing. By leveraging a small labeled dataset alongside a larger unlabeled corpus and decomposing the parsing task using Lexicalized Tree Adjoining Grammar (LTAG), the proposed method achieves significant performance gains, demonstrating the viability of semi-supervised approaches for complex structural tasks.
The Challenge of Statistical Parsing
Parsing, the process of determining the grammatical structure of a sentence, is a fundamental task in Natural Language Processing (NLP). Traditional state-of-the-art statistical parsers are heavily dependent on large-scale treebanks, such as the Penn Treebank, which consist of tens of thousands of sentences manually annotated with complex hierarchical structures.
The manual creation of these treebanks is a bottleneck; it requires expert linguistic knowledge and makes adapting parsers to new domains or languages prohibitively expensive. This research addresses a critical question: Can we achieve high-quality parsing performance using significantly less labeled data by exploiting unlabeled text?
Semi-Supervised Learning via Co-Training and LTAG
The core contribution of this work is the adaptation of the co-training algorithm to statistical parsing. Co-training typically requires two independent "views" of the data to allow two models to mutually train each other. While straightforward for classification tasks (e.g., using text vs. hyperlinks for web page categorization), parsing is inherently structural and monolithic.
To create these two views, the author utilizes Lexicalized Tree Adjoining Grammar (LTAG). LTAG allows a sentence's parse to be decomposed into two distinct components:
- SuperTagging (H1): Assigning a local elementary tree to each word in the sentence based on its local context.
- Attachment (H2): Determining how these elementary trees are combined (via substitution or adjunction) to form a complete parse.
The Co-Training Loop
The process begins with a small labeled seed set and a large unlabeled pool. Two models, H1 (the tagger) and H2 (the attacher), are trained on the labeled data. They then iteratively process a cache of unlabeled sentences. Sentences where H1 is highly confident are added to the training set for H2 (with H1's predicted labels), and vice versa. This allows the models to "teach" each other by exploiting the mutual constraint that a valid parse must satisfy both local tagging and global attachment rules.
Experimental Setup and Results
The method was evaluated using the standard Penn Treebank Wall Street Journal (WSJ) corpus. The labeled training set consisted of only ~9,600 sentences (less than 25% of the full training set), while ~30,000 sentences were used as unlabeled data.
The results demonstrate a substantial improvement over the supervised baseline:
| Model | Precision | Recall |
|---|---|---|
| Baseline (Labeled Only) | 72.23% | 69.12% |
| Co-Training (Labeled + Unlabeled) | 80.02% | 79.64% |
By incorporating unlabeled data, the parser's accuracy increased by approximately 8-10 percentage points. This significant jump highlights the effectiveness of co-training in reducing the reliance on expensive manual annotations.
Key Insights and Future Directions
A primary insight of this research is that co-training can scale from simple classification to complex, recursive structural labels. The "check-and-balance" provided by the two views—local tagging and global attachment—serves as a powerful filter for selecting high-quality training examples from unlabeled data.
However, several areas for future work remain. While the performance gain is impressive, it still trails behind fully supervised models trained on the complete 40,000-sentence corpus. Additionally, the current method relies on a "tree dictionary" extracted from the data; future iterations could explore co-training without such prior knowledge. Integrating active learning, where the system identifies the most informative (rather than just the most confident) examples for human labeling, could further optimize the training process.
Summary updated