A New Search Approach for Interactive-Predictive Computer-Assisted Translation

Zeinab Vakil, Shahram KhadiviCOLING 2012

TL;DR

This paper introduces a novel search approach for interactive-predictive computer-assisted translation (IPCAT) systems. By combining edit distance with translation cost and incorporating a hybrid back-off model, they significantly improve the performance of IPCAT, making it more efficient and accurate for human translators.

What Problem Does This Paper Address?

While statistical machine translation (SMT) has made great strides, it still falls short of human-level translation quality. Interactive-predictive computer-assisted translation (IPCAT) aims to bridge this gap by having an SMT system suggest completions as a human translator types. However, existing IPCAT systems struggle with offering relevant and accurate suggestions, especially when the user's input deviates from common patterns or when word order differs. This paper addresses the challenge of improving the performance of IPCAT by developing a more sophisticated search mechanism that can better predict and offer useful translation completions.

Methods

The authors propose two main innovations:

  • A New Search Approach: Instead of relying solely on edit distance (which measures character-level differences and can be misled by word reordering), they introduce a combined measure. This new approach weights the edit distance with the "cost of translation," which encompasses both language model and translation model scores. This aims to prioritize hypotheses that are not only similar to the prefix but also have a higher probability of being a correct translation.
  • A Hybrid Back-Off Model: When the primary search mechanism fails to find a suitable suggestion (e.g., when the prefix contains a partial word not present in the search graph), a back-off strategy is employed. The paper proposes a hybrid back-off model that combines the language model with IBM Model 1 (a foundational statistical machine translation model). This allows the system to leverage both the target language's fluency and the source-to-target translation likelihood to suggest completions, offering a more informed fallback than a language model alone.

Datasets

The experiments were conducted on two corpora:

  • Xerox Corpus: An English-German dataset.
  • Verbmobil Corpus: An English-Persian dataset (which was extended from an original English-German corpus).

The paper provides statistics on these corpora, including the number of sentences, running words, and vocabulary size, differentiating between training, development, and evaluation sets.

Key Insights and Ideas

  • Edit distance alone is insufficient: The paper highlights that edit distance, while useful, can be misleading in translation contexts due to word reordering. A hypothesis with a low edit distance might not be semantically or syntactically correct.
  • Translation cost is crucial: Incorporating translation cost (language and translation model scores) alongside edit distance leads to better suggestions, as it prioritizes hypotheses that are more likely to be correct translations.
  • Hybrid back-off offers robustness: A back-off model that combines language modeling with a translation model (like IBM Model 1) provides more reliable suggestions when the primary search fails, especially when dealing with out-of-vocabulary words or unusual phrasing.
  • The power of combined knowledge: The success of both the new search approach and the hybrid back-off model underscores the benefit of integrating multiple sources of information (edit distance, translation cost, language models, translation models) for improved IPCAT performance.

Key Results

The proposed search approach achieved:

  • 2.3% absolute improvement on the Xerox (English-German) corpus.
  • 1.16% absolute improvement on the Verbmobil (English-Persian) corpus.

The hybrid back-off model also showed improvement, with a 0.3% improvement noted on the Verbmobil corpus when using the IBM-1 + LM back-off model compared to just the LM back-off model. The paper presents detailed results in Table 3, showing reductions in KSR (Keystrokes required / Keystrokes needed) and KSMR (KSR + MAR, where MAR is mouse actions) metrics, indicating increased efficiency for human translators.

Limitations and Future Work

  • Reordering of phrases: The paper acknowledges that while the new search method tries to mitigate the issue of word reordering by considering translation cost, it does not explicitly insert reordering operations into the search. This remains a potential area for improvement.
  • Out-of-vocabulary words: The paper notes that when desired translation words are not present in the training corpus, neither the language model nor the IBM-1 model can suggest them. This suggests that improving handling of rare or unseen words could further enhance performance.
  • IBM Model 2: The paper briefly mentions that using IBM Model 2 instead of IBM Model 1 did not yield better results, possibly due to its stricter constraints. Further investigation into different IBM models or other translation models for the back-off strategy could be beneficial.

Summary updated

BibTeX
@inproceedings{vakil2012new,
  author = {Zeinab Vakil and Shahram Khadivi},
  title = {{A New Search Approach for Interactive-Predictive Computer-Assisted Translation}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure