Multiple TreeBanks Integration for Chinese Phrase Structure Grammar Parsing Using Bagging

Meishan Zhang, Wanxiang Che, Ting Liu

Introduction

Phrase Structure Grammar (PSG) parsing is a fundamental task in Natural Language Processing (NLP), focused on determining the hierarchical constituent structure of sentences. Traditionally, statistical parsers are trained on a single, manually annotated corpus (TreeBank). However, these single-model systems often suffer from high variance and are constrained by the specific annotation style and limited size of their training data.

This paper proposes a robust framework to enhance Chinese PSG parsing by leveraging bagging (bootstrap aggregating) and integrating multiple TreeBanks with disparate annotation standards. By combining sub-models trained on different data samples and different corpora—specifically the Tsinghua Chinese Treebank (TCT) and the Penn Chinese Treebank (CTB)—through a CKY reparsing algorithm, the authors demonstrate a significant improvement in parsing accuracy.

Methodology

The proposed system architecture is built upon three primary technical pillars designed to handle data diversity and structural consistency.

1. Bagging with the Berkeley Parser

The researchers utilize the Berkeley parser, a state-of-the-art un-lexicalized PCFG parser, as the base engine. To reduce model variance, they generate $k$ sub-models by repeatedly sampling the training corpus. Each bootstrap sample is approximately 64.3% of the original corpus size. This ensemble approach ensures that the final system is less sensitive to the specificities of a single training set.

2. Cross-TreeBank Integration and Transformation

A major challenge in Chinese NLP is the lack of a single unified annotation standard. TCT and CTB use different tag sets and structural conventions. To utilize the larger CTB7.0 corpus alongside TCT, the authors developed a Transformation Model. This model maps CTB-style grammars into TCT-style grammars, allowing the ensemble to benefit from the additional linguistic information in CTB without introducing structural contradictions.

The system architecture showing the training and testing pipelines for integrating TCT and CTB data.

3. CKY Reparsing for Model Combination

Rather than using simple majority voting to combine the outputs of the $k$ sub-models, the system employs a CKY reparsing algorithm. This method treats the constituents suggested by the sub-models as weighted candidates. The algorithm then finds the optimal tree structure that maximizes the total weight of the rules while adhering to formal grammar constraints, ensuring the final output is a valid phrase structure tree.

Experimental Setup and Datasets

The study primarily utilizes the TCT (Tsinghua Chinese Treebank), which served as the core corpus for the CIPS-Bakeoff2012 Task3. To enhance performance, the CTB7.0 (Penn Chinese Treebank) was integrated as a secondary, larger corpus. The datasets were partitioned into training, development, and final test sets to rigorously evaluate the impact of bagging and cross-corpus integration.

Results and Discussion

The experimental results reveal a clear "staircase" of improvement. The baseline system, a single Berkeley parser trained on TCT, achieved an F1 score of 81.33% on development data. Implementing bagging on TCT alone (Bbag) increased this to 84.41%. The full system (Cbag), which integrated CTB data through the transformation model, reached a peak F1 score of 85.03%.

The Impact of Ensemble Size

A key insight from the study is the relationship between the number of sub-models ($k$) and parsing performance. As shown in the performance curve, the F1 score increases steadily as $k$ grows, eventually plateauing around $k=15$. This confirms that while bagging is typically associated with simple classification, it is highly effective for complex structural tasks like parsing.

Performance increments as the bagging number (k) increases.

On the official test data, the final Cbag system achieved an F1 score of 81.16%, outperforming the best single-model closed systems by nearly 4%. This result underscores the value of combining diverse data sources and using ensemble methods to overcome the limitations of individual datasets.

Conclusion

This research demonstrates that bagging and cross-TreeBank integration can significantly advance the state-of-the-art in Chinese PSG parsing. By bridging annotation gaps through a transformation model and ensuring structural integrity via CKY reparsing, the authors provide a scalable method for improving parser robustness. Future work will explore more sophisticated statistical transformation models to further refine the integration of disparate linguistic resources.

Summary updated

BibTeX
@inproceedings{zhang2012multiple,
  author = {Meishan Zhang and Wanxiang Che and Ting Liu},
  title = {{Multiple TreeBanks Integration for Chinese Phrase Structure Grammar Parsing Using Bagging}},
  booktitle = {CIPS-SIGHAN},
  year = {2012},
}
Figure