An SMT-driven Authoring Tool

Sriram Venkatapathy, Shachar MirkinCOLING 2012

TL;DR

This paper introduces an SMT-driven authoring tool that helps users compose texts in their native language, aiming for more accurate translations into a target language they don't know. By providing context-aware phrase suggestions based on translatability, the tool aims to reduce post-editing effort and improve translation quality, even for users unfamiliar with the target language.

What Problem Does This Paper Address?

In today's globalized world, many professionals need to communicate in languages other than their native tongue. While machine translation (MT) is useful for understanding foreign texts, current MT technology often produces output that requires significant post-editing to be accurate and fluent. This post-editing process is time-consuming, expensive, and requires fluency in both source and target languages. The paper addresses the challenge of enabling users to compose texts in their native language that are more likely to be translated accurately into an unfamiliar target language, thereby reducing the reliance on extensive post-editing.

Methods

The core of the proposed solution is an interactive authoring tool that leverages Statistical Machine Translation (SMT) models. The tool guides users in composing their text in their native language by offering phrase suggestions. These suggestions are not solely based on linguistic fluency but also on their "translatability" – how well the SMT system can translate them. The tool employs a two-step process:

  1. Interactive Composition: The user types in their native language, and the tool provides suggestions for continuing the text. These suggestions are ranked based on a metric that balances fluency, translatability, and semantic distance from the already typed text.
  2. SMT Translation: Once the source text is composed, it is translated into the target language using the SMT system.

A key innovation is the integration of translatability into the authoring process itself, influencing the source text composition to optimize for downstream translation. The tool uses an inverted index of the phrase table for fast retrieval of suggestions.

Datasets

The paper mentions applying simulations to two datasets for experiments:

  • Technical Manuals Dataset (TM)
  • News Commentary Dataset - WMT2007 (NC)

These datasets appear to be existing corpora used for evaluating the tool's performance.

Key Insights and Ideas

  • Authoring for Translation: The fundamental idea is to shift the focus from simply writing in a native language to writing for translation. By considering the translatability of phrases during composition, the tool aims to proactively improve the quality of the final translation.
  • SMT as a Composition Aid: Instead of just being a translation engine, the SMT model actively participates in the authoring process by suggesting phrases that are likely to yield better translations.
  • Balancing Fluency and Translatability: The proposed ranking metric for suggestions is crucial. It aims to maintain the fluency of the source text while prioritizing phrases that the SMT system can translate more accurately, without deviating too much from the user's intended meaning.
  • Efficiency Gains: The tool not only aims for better translations but also for improved composition speed by allowing users to select phrases rather than typing them out entirely.

Key Results

The paper presents results from simulations, not direct human evaluations.

  • Composition Cost Reduction: Simulations showed that using the tool can reduce word composition, with more suggestions leading to greater savings. The CE (Conditional Entropy) metric was found to be preferable over MTS (Maximum Translation Score) for ranking suggestions, leading to more phrase selections.
  • Potential for Improved Translations: While the simulation methodology for evaluating translation accuracy was limited, some cases showed improved translations as measured by the BLEU score. For instance, replacing "countries" with "states" in a sentence led to a higher BLEU score.

Limitations and Future Work

  • Simulation-Based Evaluation: The primary limitation acknowledged is that the evaluation was performed using a simulator rather than actual human users. This limits the assessment of the tool's full potential, especially regarding actual composition time and post-editing effort.
  • Future Work:
    • Conduct field tests with human users to measure real-world composition time, translation results, and post-editing effort.
  • Investigate translatability and semantic relatedness estimations further.
  • Automatically tune the metric weights used for ranking suggestions.
  • Improve the user interface for better utility, such as merging overlapping phrase suggestions for a more compact list.

Summary updated

BibTeX
@inproceedings{venkatapathy2012smt,
  author = {Sriram Venkatapathy and Shachar Mirkin},
  title = {{An SMT-driven Authoring Tool}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure