Automatic Acquisition of Hierarchical Transduction Models for Machine Translation

Hiyan Alshawi, Srinivas Bangalore, Shona DouglasACL 1998

TL;DR

This paper introduces a fully automatic method for learning hierarchical transduction models for machine translation, directly from bilingual text. By treating translation as a hierarchical process rooted in "head" words, they achieve competitive accuracy without manual linguistic feature engineering, making MT model development more efficient.

What Problem Does This Paper Address?

Building robust machine translation (MT) systems traditionally requires significant human effort to define linguistic rules or features. While automatic methods exist, they often struggle with the inherent hierarchical structure of language and can lead to very large, complex models, especially for languages with significant word order differences. This paper aims to bridge this gap by proposing a method that automatically learns hierarchical translation models, specifically "head transducers," directly from parallel corpora, thereby reducing development effort and potentially improving model efficiency.

Methods

The core of the paper is a two-stage automatic learning process:

  1. Alignment Stage: This stage establishes correspondences between words and subsequences in the source and target languages. It involves:
  2. Alignment Model: A weighted combination of cost functions (correlation and distance measures) is used to score potential word pairings.
  3. Alignment Search: An agenda-based dynamic programming search finds the optimal set of pairings that map the entire source sentence to its target translation.
  4. Transducer Construction Stage: This stage builds hierarchical head transducer models based on the alignments.
  5. Recursive Partitioning: The source and target sentences are recursively partitioned around "head" words. This process ensures that the learned translation structure respects hierarchical dependencies and avoids crossing dependencies in the target.
  6. Hypothesized Transitions: Based on the aligned head words and their dependents, hypothesized state transitions for the head transducer are traced. These transitions capture how source words and their dependent phrases are mapped to target words and their dependent phrases, including word order changes.
  7. Model Aggregation: Counts of these hypothesized transitions are aggregated to train a maximum likelihood head transduction model, learning both the model's structure and its parameters (probabilities for start, transition, and stop events).
  8. Head Selection: A crucial aspect is selecting the "head" word for each partition. The paper explores metrics like distance, word frequency, and word complexity, and finds that using target-language metrics for head selection yields better results.

Datasets

The experiments were conducted using data from the Air Travel Information System (ATIS) corpus, which contains English utterances and their Spanish translations.

  • Training Data: 14,418 English utterances with Spanish translations (totaling 140,788 source words and 167,865 target words).
  • Test Data: 336 held-out English sentences.
  • Reference Translations: Three separate human translations were obtained for the test set to provide a more robust evaluation.

Key Insights and Ideas

  • Hierarchical Structure is Key: The paper strongly advocates for modeling translation as a hierarchical process, rooted in "head" words, rather than a flat sequence-to-sequence transformation. This aligns better with linguistic intuition and leads to more compact and potentially more accurate models.
  • Automatic Acquisition Reduces Effort: The entire process, from alignment to model construction, is fully automatic, significantly reducing the need for manual linguistic expertise. This is a major step towards more scalable MT development.
  • Target-Side Head Selection is Superior: Surprisingly, using metrics to select head words based on the target language's properties (and then aligning them back to the source) proved more effective than using source-side metrics. This suggests that the target language's structure might be a better guide for learning translation mappings.
  • Head Transducers Offer Efficiency: The learned head transducer models are shown to be smaller and require fewer computational resources compared to comparable transfer-based systems, while achieving better accuracy.

Key Results

The developed English-Spanish translation model achieved competitive results:

  • Word Accuracy: Over 75% word accuracy when compared against three reference translations.
  • Specific Results:
    • Against a single held-out human translation, the system achieved 74.1% accuracy on all sentence lengths and 77.3% on sentences up to length 15 (the length used for training).
  • Against the closest of three human translations, the system achieved 78.5% accuracy on sentences of all lengths.
  • The "In Target" head selection method (using target-language metrics) achieved 74.1% word accuracy with 66K parameters, outperforming random head selection (64.7% accuracy, 108K parameters) and source-side head selection (71.4% accuracy, 67K parameters).

Limitations and Future Work

  • Limited Domain: The experiments were conducted on a limited domain (ATIS English-Spanish), and the authors acknowledge the need to test the method on more challenging language pairs with greater word order variation and complex lexical transformations (e.g., English to Japanese).
  • Model Simplicity: The current version of the method is described as simple. The authors plan to increase accuracy by incorporating statistical modeling techniques that have proven successful in speech recognition.
  • Further Exploration of Head Selection: While target-side head selection was found to be better, further research into optimal head selection strategies is implied.

Summary updated

BibTeX
@inproceedings{alshawi1998automatic,
  author = {Hiyan Alshawi and Srinivas Bangalore and Shona Douglas},
  title = {{Automatic Acquisition of Hierarchical Transduction Models for Machine Translation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1998},
}
Figure