User-Friendly Text Prediction For Translators

George F. Foster, Philippe Langlais, Guy LapalmeEMNLP 2002 · OpenReview

Abstract

In the early 2000s, interactive machine translation (IMT) systems like TransType faced a significant "productivity paradox": despite high prediction accuracy, professional translators were often slower when using these tools. This paper, presented at EMNLP 2002, addresses this by shifting the objective from maximizing prediction accuracy to maximizing the user's expected benefit. By integrating a statistical translation model with a realistic model of human typing and reading behavior, the researchers achieved a 10-12% increase in productivity, effectively reversing the previous trend of productivity loss.

The Productivity Paradox in Translation

Early prototypes of interactive machine translation were technically impressive, correctly anticipating up to 70% of the characters a translator intended to type. However, real-world trials revealed a counter-intuitive result: professional translators were 17% slower when using the tool.

The bottleneck was not the model's accuracy, but the cognitive and physical overhead of the interface. Translators had to pause their workflow to read suggestions, evaluate them, and decide whether to accept or reject them. If a suggestion was incorrect or too short to justify the reading time, the interaction became a net loss in efficiency. To solve this, the authors proposed that a system should only offer a prediction when it is likely to save the user time.

Methodology: Maximizing Expected Benefit

The core innovation of this work is a decision-theoretic framework. Instead of always providing a fixed-length suggestion, the system calculates the Expected Benefit (EB) of a prediction and remains silent if the benefit is not positive.

The User Model

The system relies on a sophisticated user model to estimate the utility of a proposal. This model considers two primary factors:

  1. Acceptance Probability: The likelihood that a user will accept a suggestion based on its "gain"—the length of the correct prefix minus the length of the incorrect suffix.
  2. Benefit Function: A metric that quantifies keystrokes saved, accounting for the time cost of reading, the effort of manual typing, and the penalty for erasing errors.
Probability of accepting a suggestion as a function of its gain (correct prefix length minus incorrect suffix length).

As shown in the figure above, users are highly unlikely to accept suggestions with negative or zero gain. The probability of acceptance rises sharply once the gain exceeds a certain threshold, justifying the system's "fewer but better" approach.

The underlying engine uses a log-linear combination of a trigram language model and a Maximum Entropy (MaxEnt) translation model. This setup is optimized for real-time interaction. A Viterbi-style beam search is then employed to find the word sequence (up to 5 words long) that maximizes the expected benefit rather than just the raw probability.

Evaluation and Results

The models were trained and tested using the Canadian Hansard corpus. To evaluate the system, the researchers simulated different types of users—ranging from a "superman" (who always knows if a suggestion is correct) to a "real" user (modeled after data from professional trials).

Percentage of keystrokes saved for different user models across varying maximum proposal lengths (M).

The results demonstrate a significant breakthrough. For "real" users, the system achieved a 10% to 12% reduction in keystrokes. This is a stark contrast to the 17% productivity loss observed in earlier versions of the technology. The data also suggests that performance peaks when the system is allowed to suggest sequences of 3 to 5 words, provided it accounts for the model's decreasing certainty over longer strings.

Conclusion

By prioritizing the user's cognitive load over raw algorithmic accuracy, the authors demonstrated that machine translation can be a genuine aid to human experts. The "benefit-maximizing" framework ensures that the system only intervenes when it provides clear value, transforming a potential distraction into a productivity-enhancing tool. Future work points toward making these models adaptive to individual translator habits and improving real-time alignment between source and target texts.

Summary updated

BibTeX
@inproceedings{foster2002user,
  author = {George F. Foster and Philippe Langlais and Guy Lapalme},
  title = {{User-Friendly Text Prediction For Translators}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2002},
  url = {https://openreview.net/forum?id=KpJGpv7srvQu},
}
Figure