Automatic Adaptation of Annotation Standards for Dependency Parsing ? Using Projected Treebank as Source Corpus

Wenbin Jiang, Qun Liu

Abstract

Dependency parsing is a fundamental task in Natural Language Processing, yet its progress is often hindered by the high cost of manual annotation and the lack of unified standards across different treebanks. This paper proposes a novel framework to adapt annotation standards by leveraging bilingual corpora. By projecting English syntactic structures onto Chinese translations through an error-tolerant alignment matrix, the authors generate a source treebank that guides the training of a target parser. Experimental results demonstrate significant accuracy improvements, particularly in low-resource scenarios.

Introduction

Identifying the grammatical structure of sentences—known as dependency parsing—is essential for downstream NLP applications. However, the field faces a significant challenge: annotation standards. Different treebanks often employ varying rules for labeling relationships, making it difficult to combine datasets. While resource-rich languages like English benefit from extensive labeled data, most other languages suffer from a scarcity of hand-annotated corpora. This research addresses this bottleneck by automatically generating source data from bilingual text and adapting it to a target standard.

Methodology

The authors introduce a two-step framework for cross-lingual annotation adaptation.

1. Error-Tolerant Tree Projection

Traditional methods for projecting trees between languages often rely on direct 1-to-1 word mapping, which is sensitive to non-literal translations. To mitigate this, the authors utilize an alignment matrix—a probabilistic representation of word correspondences. A dynamic programming algorithm is then employed to identify the Chinese dependency tree that maximizes consistency with the English source tree across all potential alignments, making the process robust to translation noise.

2. Annotation Adaptation via Guide Features

Rather than directly merging the projected data with the target corpus, the authors use the projected data to train a source parser. During the training of the target parser, the source parser’s predictions serve as "guide features." The target parser learns to weigh these structural hints—categorized as parent-child, child-parent, siblings, or none—against its own high-quality labeled data.

Evaluation and Results

The method was evaluated using a source corpus of 5.6 million Chinese-English sentence pairs and the Penn Chinese Treebank (CTB) versions 1.0 and 5.0 as target corpora.

Performance in Low-Resource Scenarios

The adaptation strategy proves most effective when labeled target data is scarce. As shown in Figure 1, the target parser consistently outperforms the baseline, with the most pronounced gains occurring at lower sentence counts. For instance, with approximately 2,000 sentences, the adaptation provides nearly a 2-point accuracy boost.

Dependency accuracy vs. sentence count of target corpus.

Overall Accuracy Gains

On the CTB 1.0 dataset, the proposed system achieved an accuracy of 83.56%, representing a 1.33% improvement over the baseline parser (82.23%). Notably, even though the source parser itself has low standalone accuracy (53.28%) due to projection noise, its structural hints remain valuable for the target parser.

Performance comparison on CTB 1 and CTB 5 datasets.

Conclusion

This research demonstrates that bilingual corpora can be effectively harnessed to overcome the limitations of manual annotation. By using error-tolerant projection and a guide-feature adaptation mechanism, dependency parsers can achieve higher accuracy with less labeled data. Future work will explore extending this methodology to constituent parsing and refining the handling of complex syntactic structures.

Summary updated

BibTeX
@inproceedings{jiang2009automatic,
  author = {Wenbin Jiang and Qun Liu},
  title = {{Automatic Adaptation of Annotation Standards for Dependency Parsing ? Using Projected Treebank as Source Corpus}},
  booktitle = {IWPT},
  year = {2009},
  url = {https://openreview.net/forum?id=MGMbNATm9zr},
}
Figure