Automatic Refinement of a POS Tagger Using a Reliable Parser and Plain Text Corpora

Hideki Hirakawa, Kenji Ono, Yumiko YoshimuraCOLING 2000

TL;DR

This paper introduces APRAS, an unsupervised learning system that automatically extracts rules to improve English Part-of-Speech (POS) taggers. By leveraging the existing linguistic knowledge within a machine translation system (tagger and parser), APRAS refines POS disambiguation rules from plain text, leading to improved accuracy and parsing efficiency without requiring manually annotated data.

What Problem Does This Paper Address?

Accurate Part-of-Speech (POS) tagging is a fundamental step in many Natural Language Processing (NLP) tasks. Traditionally, high-accuracy POS taggers are built using large, manually annotated corpora. However, creating such corpora is labor-intensive and expensive, especially for acquiring specialized or domain-specific knowledge. While some methods attempt to learn from plain text, they often have limitations, such as focusing only on unknown words based on their endings or struggling with scalability. This paper addresses the challenge of acquiring robust POS disambiguation rules from readily available plain text corpora, aiming to improve tagger performance without the need for extensive manual annotation.

Methods

The core of the proposed method lies in the APRAS (Automatic POS Rule Acquisition System), which operates in two phases: rule extraction and rule application.

Rule Extraction Phase: 1. POS Tagging and Parsing: A pre-existing, reliable Machine Translation (MT) system's tagger and parser are used. The tagger generates multiple ranked POS sequences for each sentence in a training corpus. The parser then attempts to parse these sequences. 2. Conflicting Sentence Identification: Sentences are categorized into: * Balanced: The top-ranked POS sequence is parsable. * Conflicting: The top-ranked sequence is unparsable, but another sequence in the ranking is parsable. * Unparsable: All sequences are unparsable. APRAS focuses on "conflicting" sentences.

  1. PA Rule Candidate Generation: For conflicting sentences, APRAS identifies "focus words" – words whose POS tag differs between the initial (top-ranked) POS sequence and the most preferable parsable POS sequence (MPP POS sequence). For each focus word, a "PA rule candidate" is generated. This candidate specifies the focus word, its initially tagged POS, its parsable POS, and the surrounding POS context (preceding and succeeding POS tags). The rule essentially states: "Give priority to the parsable POS over the initially tagged POS in this specific context."
  2. Rule Filtering: A statistical filtering mechanism using "adjustment ratios" is employed. These ratios are calculated based on how often a rule candidate leads to a parsable sentence when applied. Rules with high adjustment ratios are considered more valid.

Rule Application Phase: The extracted and filtered PA rules are integrated into the MT system's tagger. These rules act as a refinement mechanism, guiding the tagger to avoid generating less parsable POS sequences, thereby improving the overall parsing accuracy and potentially reducing processing time.

Datasets

  • Training Corpus: English news articles (6,684,848 sentences, 530MB).
  • Test Corpus: A different set of English news articles (146,229 sentences; 2.26M Words).

The paper does not introduce new benchmarks but utilizes existing plain text corpora.

Key Insights and Ideas

  • Leveraging Existing NLP Systems: The most significant insight is the effective use of a pre-existing, well-developed NLP system (specifically, its tagger and parser) as a supervisor for unsupervised rule acquisition. This avoids the need for manual annotation by relying on the system's inherent linguistic knowledge.
  • "Conflicting Sentences" as a Source of Knowledge: Identifying sentences where the initial POS guess is wrong but a correctable alternative exists is a clever way to pinpoint areas where disambiguation rules are needed.
  • PA Rules as Refinements, Not Replacements: The PA rules are designed to adjust the tagger's output in specific contexts, rather than replacing the tagger entirely. This makes the system robust, as incorrect rules are less likely to cause catastrophic failure.
  • Statistical Filtering for Validity: The "adjustment ratio" provides a data-driven, statistical method to assess the reliability of generated PA rules, allowing for the selection of high-quality rules.

Key Results

  • Improved Tagging and Parsing: Applying the extracted PA rules to a test corpus resulted in a 15.5% improvement in tagging and parsing speed and an 8.0% increase in the ratio of successfully parsed sentences.
  • POS Tagging Accuracy: Out of the sentences where PA rules were applied, 78.4% of the changes made in tagging results were an improvement.
  • Rule Extraction Efficiency: The system extracted 17,731 rules from the training corpus, which were then filtered down to 4,494 rules (after removing those with adjustment ratios below 60%).
  • Rule Validity: For rules with adjustment ratios above 60%, 86% were found to be valid, with only 7% being invalid.

Limitations and Future Work

  • Context Window Size: The paper notes that expanding the context window for PA rules could increase accuracy but decrease frequency, making statistical filtering difficult. The current context size is set to two words.
  • Invalid Rules: While the system is robust to invalid rules, the paper acknowledges that some invalid rules were due to parsing errors stemming from insufficient parsing rules or issues with compound words and distinct header expressions.
  • Future Work:
    • Expand the size of training corpora to acquire more PA rules.
  • Investigate the use of invalid PA rules with low adjustment ratios, potentially as "prohibition rules."
    • Further research into distinguishing compound words from non-compound words and handling context window variations.
  • Generalize specific rules to reflect maximum distinctions of individual examples for greater effectiveness.

Summary updated

BibTeX
@inproceedings{hirakawa2000automatic,
  author = {Hideki Hirakawa and Kenji Ono and Yumiko Yoshimura},
  title = {{Automatic Refinement of a POS Tagger Using a Reliable Parser and Plain Text Corpora}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2000},
}
Figure