Adaptive Transformation-Based Learning for Improving Dictionary Tagging.

Burcu Karagol-Ayan, David Doermann, Amy WeinbergACL 2006

TL;DR

This paper introduces an adaptive technique using Transformation-Based Learning (TBL) to automatically parse and tag dictionary entries, extracting lexicographic components like headwords and translations. By applying TBL to both font style and tagging, the system significantly improves accuracy, even with minimal user-provided training data, making it highly effective for low-density languages.

What Problem Does This Paper Address?

Electronic dictionaries are invaluable resources for Natural Language Processing (NLP). However, for languages with limited digital resources (low-density languages), printed bilingual dictionaries are often the primary source of lexical information. Extracting this structured information from printed dictionaries automatically is challenging. While rule-based systems can leverage dictionary layout and formatting, they struggle with noise from OCR errors, inconsistent formatting, and the ambiguity of certain tokens (e.g., "a" can be a part-of-speech tag or part of a translation). This paper addresses the need for a robust and efficient method to automatically parse and tag dictionary entries, improving accuracy with minimal human effort, especially for languages where large parallel corpora are unavailable.

Methods

The core of the proposed system is an adaptive Transformation-Based Learning (TBL) approach applied as a post-processor. TBL is a machine learning method that learns a sequence of rules (transformations) to correct errors in an initial annotation. The authors apply TBL in two key stages:

  1. Improving Font Style Accuracy: Initially, document image analysis provides font style information. TBL is used to refine this font style assignment, as accurate font styles are crucial for identifying lexicographic components.
  2. Improving Tagging Accuracy: After font styles are refined (or using the initial ones), TBL is applied again to correct the assigned tags (e.g., headword, POS, translation).

The system uses a rule-based entry tagger as a baseline, which leverages dictionary structure, font styles, keywords, and separators. TBL then acts as a post-processor to overcome the limitations of this rule-based approach, particularly its inability to handle inter-tag relationships and its susceptibility to noise. The authors explore three configurations: TBL on font style only, TBL on tags only, and TBL on both sequentially. They also investigate the "upper bound" by using manually corrected font styles (ground truth) as input to the rule-based tagger and then applying TBL for tagging.

Datasets

The experiments were conducted on two dictionaries:

  • Cebuano-English dictionary: (Wolff, 1972) - 1163 pages, 4 font styles, 18 tags.
  • Iraqi Arabic-English dictionary: (Woodhead and Beene, 2003) - 507 pages, 3 font styles, 26 tags.

For training and testing TBL, a small subset of pages was used: 8 randomly selected pages for training and 6 for testing. The font style and tags for these pages were manually corrected to serve as ground truth for evaluating TBL's performance.

Key Insights and Ideas

  • Font Style is Crucial: The paper strongly emphasizes that accurate font style identification is a prerequisite for accurate dictionary tagging. Improving font style with TBL significantly boosts overall tagging accuracy.
  • TBL's Adaptability: TBL proves to be highly effective even with extremely limited user-provided training data, making it suitable for low-resource scenarios.
  • Sequential Application is Key: Applying TBL first to improve font style and then to improve tagging yields the best results, outperforming applying TBL only to tags.
  • Phrase Accuracy Gains are Significant: While token accuracy sees substantial improvements, the gains in phrase accuracy (e.g., for definitions or example translations) are even more pronounced.
  • Rule-Based Baseline is Solid but Limited: The rule-based tagger provides a good starting point, but its limitations in handling context and noise are effectively addressed by TBL.

Key Results

The paper reports significant improvements in tagging accuracy:

  • Font Style Accuracy: Increased from 84%-94% to 97%-98% for non-punctuation tokens.
  • Token Tagging Accuracy: Increased from 83%-91% to 93%-94%.
  • Phrase Tagging Accuracy: Increased from 64%-83% to 90%-93%.

When TBL was applied to font style first, it further assisted tagging accuracy by up to 2.62% for tokens and 2.82% for phrases compared to using TBL only for tagging. The system achieved high performance with minimal training data (a few pages), with the learning phase taking less than a minute and application to the whole dictionary less than two minutes.

Limitations and Future Work

  • Error Sources: The authors plan to investigate the sources of errors introduced by TBL and explore methods to avoid them through post-processing or heuristics.
  • Bootstrapping Training Data: They intend to explore increasing the training data size in a bootstrapped manner, where TBL is applied to a few pages, then corrected and used as new training data.
  • Training Data Size: While the paper demonstrates effectiveness with small datasets, further investigation into the optimal training data size for achieving reasonable performance increases is planned.
  • Generalizability: The paper notes that while the method is adaptive, each dictionary requires some manual annotation to define its specific structure and clues, which is a necessary step before TBL can be applied.

Summary updated

BibTeX
@inproceedings{karagolayan2006adaptive,
  author = {Burcu Karagol-Ayan and David Doermann and Amy Weinberg},
  title = {{Adaptive Transformation-Based Learning for Improving Dictionary Tagging.}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
}
Figure