Automated Postediting of Documents

Kevin Knight, Ishwar ChanderAAAI 1994

Abstract

This paper presents a modular framework for enhancing Machine Translation (MT) quality through "automated posteditors"—detachable software components designed to correct specific linguistic errors. By decoupling error correction from the core translation engine, the authors develop specialized modules for tasks such as English article selection ($a, an, the$). Using a data-driven approach with decision tree induction on the Wall Street Journal corpus, the proposed system achieves 78% accuracy in article prediction, approaching human performance levels when restricted to local context.

Introduction

In the early 1990s, Machine Translation and Optical Character Recognition (OCR) systems frequently produced "low-to-medium quality" text that necessitated extensive manual correction. Historically, researchers attempted to resolve these issues by modifying the internal logic of specific MT systems. However, this approach suffered from a lack of portability—improvements for a Japanese-English system could not be easily applied to other language pairs—and failed to address similar errors produced by non-native speakers or OCR processes.

The Modular Postediting Framework

The authors propose a shift toward a modular architecture where automated postediting (PE) modules sit between the MT output and the final human reviewer. This "detachable" design allows for the development of error-specific tools that can be integrated into any text-processing pipeline regardless of the source of the initial draft.

Comparison of traditional human postediting (a) and the proposed automated postediting (PE) workflow (b).

Methodology: Statistical Article Selection

The primary focus of this work is the challenge of English article selection, a significant hurdle for Japanese-to-English translation as Japanese lacks articles entirely. Rather than relying on complex, manually-coded linguistic rules, the authors adopted a data-driven, statistical approach:

  1. Feature Engineering: Over 400,000 core noun phrases were extracted from the Wall Street Journal (WSJ) corpus. These were characterized by over 30,000 features, including lexical context (surrounding words) and abstract properties (part-of-speech tags, mass noun categorization, and superlatives).
  2. Decision Tree Induction: The authors utilized decision tree algorithms to model these features. To manage the massive feature space, specialized trees were built for the 1,600 most frequent "head nouns" (e.g., "president," "stock"), covering approximately 77% of the test set instances.

Experimental Results and Human Baselines

To evaluate the system, the authors established human performance baselines. While humans achieve ~95% accuracy with full document context, their performance drops to 83-88% when limited to the local context (the noun phrase plus a few surrounding words) available to the machine.

Accuracy comparison between human performance and machine baselines across varying levels of context.

The automated system achieved an overall accuracy of 78% on financial texts, significantly outperforming the "always guess the" baseline of 67%. For specific high-frequency nouns, the results were even more robust: 89% for "president" and 94% for "year." On the subset of instances where the system had a specifically trained tree, it reached 81% accuracy, nearing the lower bound of human performance in limited-context scenarios.

Discussion and Limitations

Despite its success, the model faces several limitations. It currently operates on a binary choice between the and a/an, omitting the "zero article" case common in English. Furthermore, the system is "discourse blind," meaning it cannot recognize when a noun should be definite (the) based on its introduction earlier in a document. Future work aims to incorporate broader discourse features and expand the modular postediting logic to other linguistic challenges, such as preposition selection and pluralization.

Summary updated

BibTeX
@inproceedings{knight1994automated,
  author = {Kevin Knight and Ishwar Chander},
  title = {{Automated Postediting of Documents}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}
Figure