Authors: Jason Naradowsky, Xuan Zhang, Kevin Duh
Venue: Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (2020)
Introduction
Machine translation (MT) systems often struggle with domain adaptation. A model trained on legal text might perform poorly on casual conversation. Traditional online adaptation methods, which update model weights as new data arrives, suffer from "catastrophic forgetting"—improving on new domains at the expense of old ones. Furthermore, real-world feedback is often sparse and noisy, lacking the gold-standard reference translations required for standard training.
This paper proposes a different approach: treating adaptation as a selection problem. Instead of updating a single model, the system maintains a set of specialized, pre-trained MT engines and uses multi-armed bandit algorithms to select the most appropriate one for each sentence based on simple user feedback.
The Bandit Selection Framework
The researchers frame MT system selection as a Multi-Armed Bandit (MAB) problem. In this paradigm, an agent chooses among several "arms" (different MT systems) to maximize a cumulative reward, which represents user satisfaction.
Three primary strategies were evaluated:
- Epsilon-Greedy: Primarily exploits the current best-performing system while occasionally exploring others.
- Upper Confidence Bound (UCB): Prioritizes systems with high potential or those that have been infrequently tested.
- LinUCB (Contextual Bandit): The most sophisticated approach, which utilizes the context of the source sentence (e.g., length, OOV counts, BERT embeddings) to predict the best system for that specific input.
Experimental Setup
The study focused on German-to-English (DE→EN) translation across three distinct domains:
- General-Domain: Mixed OpenSubtitles and WMT data.
- TED: Captions from TED talks.
- WIPO: Technical patent translations.
The "arms" consisted of eight distinct systems, including Neural MT (NMT), Statistical MT (SMT), and domain-specific fine-tuned models. To simulate real-world conditions, human feedback was modeled using "noised" BLEU scores, mimicking various types of user behavior such as harsh critics or low-granularity ratings.
Performance and Learning Dynamics
A key advantage of this selection-based approach is the avoidance of catastrophic forgetting. Because the underlying models remain fixed, the system retains its proficiency in general domains even as it learns to specialize for technical ones.
The contextual bandit (LinUCB) proved particularly effective in mixed-domain scenarios. By leveraging sentence-level features, it learns nuanced policies—for instance, identifying that one system handles short sentences better while another excels at technical jargon.
Key Results
- Superiority over Oracles: In shuffled mixed-domain tests, LinUCB achieved a BLEU score of 46.9. This significantly outperformed the "Best-Arm-Oracle" (the single best system for the entire set), which scored 34.2.
- Rapid Convergence: Learning occurs quickly; cumulative regret drops sharply within the first 50-100 interactions, making the system viable for immediate deployment.
- Robustness: The bandit algorithms remained effective even when faced with inconsistent or granular feedback. BERT embeddings and sentence length were identified as the most effective features for the contextual agent.
Conclusion
By reframing adaptation as a selection task, this work provides a robust solution to the challenges of domain shift and feedback scarcity in machine translation. Future research may explore hybrid models that combine selection with periodic weight updates or investigate the scalability of the approach as the number of available MT engines increases.
Summary updated