Alignment by Bilingual Generation and Monolingual Derivation

Toshiaki Nakazawa, Sadao KurohashiCOLING 2012

TL;DR

This paper tackles the challenge of aligning function words (like prepositions and articles) in distant language pairs, which often lack direct translations and lead to alignment errors. They propose a novel method that leverages monolingual data to "derive" these tricky function words from their content word counterparts, significantly improving alignment and translation quality.

What Problem Does This Paper Address?

Machine translation systems rely heavily on aligning words in parallel corpora to learn translation knowledge. However, aligning words between languages with very different structures, like English and Japanese, is notoriously difficult. A major hurdle is handling "function words" – words that often don't have direct, one-to-one translations in the other language (e.g., English articles like "the" or Japanese particles like "は"). These "unique function words" are frequently misaligned, leading to errors in the learned translation rules. Existing methods often resort to ad-hoc rules or aligning these words to "NULL," which can create gaps and hinder the learning process. This paper aims to provide a more robust and principled way to handle these challenging words.

Methods

The core innovation is a "monolingual derivation" model. Instead of directly aligning unique function words, the proposed system first generates bilingual "treelet pairs" (small, tree-like structures representing phrases or clauses) from parallel data. Then, for each treelet pair, it uses monolingual data to derive unique function words from their associated content words within their own language. This derivation process is guided by "semantic-head dependency trees," which aim to preserve similar dependency relationships across languages. The probabilities for these derivations are learned from large monolingual corpora, which are much easier to obtain than parallel corpora. This approach prevents incorrect alignments of unique function words by treating them as extensions of content words rather than trying to force a direct translation.

Datasets

  • Training and Evaluation: The experiments were conducted on the JST paper abstract corpus, consisting of 996K parallel English-Japanese sentences (24.7M English words, 27.5M Japanese words).
  • Monolingual Data for Derivation Probabilities: English and Japanese Web corpora, each comprising 550M sentences, were used to calculate derivation probabilities.

Key Insights and Ideas

  • Monolingual Data is Key for Function Words: The paper demonstrates that leveraging large monolingual corpora is a powerful way to handle function words that lack direct translations, overcoming the limitations of relying solely on parallel data.
  • Derivation, Not Direct Alignment: Reframing the problem from direct alignment to "derivation" of function words from content words within their own language is a crucial conceptual shift that resolves many alignment issues.
  • Semantic-Head Dependency Trees Improve Alignment: Using semantic-head dependency trees, which better capture cross-lingual semantic relationships compared to syntactic-head trees, is shown to be beneficial for alignment quality.
  • Reducing Gaps is Important: By deriving function words, the model can reduce the "gaps" created by NULL alignments, leading to more coherent dependency structures and better translation.

Key Results

  • The proposed model achieved significant improvements in alignment accuracy (measured by Alignment Error Rate - AER) compared to baseline systems, including GIZA++ and a previous semantic-head dependency tree approach.
  • Specifically, the model achieved a 0.6% and 1.34% absolute AER improvement when evaluating all alignments and core alignments, respectively. This translates to approximately a 10% relative error reduction for core alignments.
  • In translation experiments, the model using all alignments (including derivations) achieved the best BLEU scores for both English-to-Japanese and Japanese-to-English translation, indicating improved translation quality.

Limitations and Future Work

  • Granularity of Derivation Probability: The paper acknowledges that the current method for calculating derivation probabilities (combining lexicalized and POS-based probabilities) is insufficient for some cases, leading to occasional errors. They suggest using word classes with similar derivation distributions as a potential solution.
  • Over-Derivation: The model can sometimes produce too many derivations, which can introduce noise and errors, especially if the parallel data or parsing is noisy.
  • Future Work: The authors propose applying the model to other language pairs, particularly agglutinative languages like Korean, to further validate its effectiveness. They also aim to address the identified limitations regarding derivation granularity and over-derivation.

Summary updated

BibTeX
@inproceedings{nakazawa2012alignment,
  author = {Toshiaki Nakazawa and Sadao Kurohashi},
  title = {{Alignment by Bilingual Generation and Monolingual Derivation}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure