Annotating Topological Fields and Chunks - and Revising POS Tags at the Same Time

Frank Henrik Müller, Tylman UleCOLING 2002

TL;DR

This paper introduces a novel method for automatically annotating German sentences with topological fields and chunks, while simultaneously improving the accuracy of part-of-speech (POS) tags. By integrating a rule-based topological field parser with POS tagging, the system can correct errors in the initial POS tags, leading to more accurate syntactic analysis and overall annotation quality.

What Problem Does This Paper Address?

Accurate syntactic annotation of natural language corpora is crucial for linguistic research and downstream NLP tasks. However, errors in the initial part-of-speech (POS) tagging can propagate and significantly degrade the quality of subsequent annotation levels, such as identifying topological fields and clause boundaries in German. The German sentence structure, with its complex verb placement and embedded clauses, makes accurate annotation particularly challenging. Existing POS taggers, often relying on local context, struggle with ambiguous words and long-distance dependencies, leading to errors that hinder deeper syntactic analysis. This paper addresses the problem of improving overall annotation accuracy by tackling errors at the POS tagging level during the syntactic parsing process.

Methods

The paper proposes a hybrid approach that integrates topological field parsing with POS tag correction. The core idea is to use a rule-based topological field parser, which relies on hand-crafted finite state grammars, to identify sentence structure. This parser operates on POS tags and local context. Crucially, when the parser encounters a potential mismatch or ambiguity, it leverages the ranked POS tag assignments (not just the top choice) to attempt a parsable structure. If successful, the parser promotes a lower-ranked POS tag to the best tag, effectively correcting the initial POS annotation. This "tag correction" component is integrated directly into the parsing cascade, allowing for a more robust and accurate annotation process. The method prioritizes annotating topological fields first, which helps to constrain the search space for subsequent annotations.

Datasets

The paper mentions that the POS tagger used in their system was trained on a corpus of 490,000 tokens, consisting of 315,000 tokens from newspaper texts (primarily "die tageszeitung") and 150,000 tokens from novels. This corpus was compiled for the "Deutsches Referenzkorpus" (DEREKO) project. The system was tested using standard tenfold cross-validation.

Key Insights and Ideas

  • POS Tag Correction as a Byproduct of Parsing: The most significant insight is that syntactic parsing, specifically topological field parsing in German, can be used as a mechanism to identify and correct errors in the initial POS tags. This is a departure from traditional pipelines where POS tagging is a strictly preceding step.
  • Leveraging Ambiguity for Improvement: Instead of treating POS tag ambiguity as a problem to be solved solely by the tagger, the system embraces it. The ranked POS tag assignments provide a rich source of information that the parser can exploit to find the most linguistically plausible structure.
  • Topological Fields as a Structural Scaffold: The topological field model provides a robust framework for German sentence structure. By first annotating these fields, the system creates a strong structural scaffold that guides the correction of lower-level annotations like POS tags.
  • Efficiency of Rule-Based Systems for Specific Tasks: The paper demonstrates that even in an era of machine learning dominance, hand-crafted rule-based systems can be highly effective for specific linguistic phenomena, especially when integrated intelligently with other components.

Key Results

The paper reports significant improvements in tagging accuracy, particularly for verbs.

  • The error rate for the tag VVFIN (finite lexical verb) decreased from 9.95% to 8.07%.
  • For all verb tags combined, the error rate was reduced from 6.28% to 4.78%.
  • The error rate for subordinators (KOUS and KOUI) also decreased, from 3.33% to 1.53% for KOUS and from 7.23% to 5.31% for KOUI.
  • The system achieved a high precision of 96.3% for changes made to the subordinator component.
  • The number of clauses that were unparsable due to incorrect verb tags was reduced by 19%.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the system improves POS tagging accuracy for specific classes of words (verbs, subordinators), it doesn't necessarily address all types of POS tagging errors. The effectiveness relies on the presence of "reliable markers" for topological fields.
  • Future Work: The authors plan to investigate the impact of this POS tag correction on overall parsing accuracy. They also aim to explore how this approach compares to machine learning methods for topological field parsing and to further refine the integration of POS tag correction into the parsing process.

Summary updated

BibTeX
@inproceedings{muller2002annotating,
  author = {Frank Henrik Müller and Tylman Ule},
  title = {{Annotating Topological Fields and Chunks - and Revising POS Tags at the Same Time}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2002},
}
Figure