Quantifying Information of Tokens for Simple and Flexible Simultaneous Machine Translation

Donghyun Lee, Minkyung Park, Byung-Jun Lee

Introduction

Simultaneous Machine Translation (ST) presents a significant challenge: translating a sentence before the speaker has finished. This creates a fundamental quality-latency trade-off. Waiting for more context improves translation quality but increases delay, while translating early risks errors due to missing information. Traditional approaches often rely on fixed policies like "Wait-k," which lack adaptability, or complex adaptive models that are expensive to train and inflexible to latency adjustments.

This paper introduces the Information Quantifier (IQ), a lightweight module designed to enable standard "offline" translation models to operate simultaneously. By measuring the information content in source and target tokens, IQ makes dynamic decisions on when to wait (READ) and when to translate (WRITE).

Methodology: The Information Quantifier (IQ) Framework

The core innovation is treating "information" as a cumulative scalar value. Instead of training a model to predict actions directly, the researchers developed two small feed-forward neural networks to quantify information.

Architecture of the Information Quantifier (IQ) framework, illustrating how source and target information scores are compared to determine Read/Write actions.

Quantifying Information and Decision Logic

For a given partial source sentence and its current translation, IQ calculates total information scores: $Info^{src}$ and $Info^{trg}$. The decision rule is straightforward:

  • WRITE: If $Info^{src} > Info^{trg} + \epsilon$
  • READ: Otherwise

The parameter $\epsilon$ is a tunable threshold that allows for post-training flexibility. Adjusting $\epsilon$ at inference time enables the system to shift between high-speed (low latency) and high-accuracy (high quality) modes without retraining.

Training with Oracle Supervision

The IQ module is trained using "oracle action sequences"—optimal decision paths generated by comparing an offline model's full-sentence output with its partial-sentence decoding attempts. A "Violation Loss" function penalizes the model when its information scores contradict these oracle actions, ensuring the quantifier learns to accurately reflect the information state.

Experimental Results

The IQ framework was evaluated on the IWSLT14 (German-English) and IWSLT15 (English-Vietnamese) benchmarks.

Translation quality (BLEU) vs. latency (Average Lagging) for IQ and various baselines across four language pairs.

Performance and Efficiency

As shown in the results, IQ consistently outperforms both fixed-policy (Wait-k) and adaptive-policy baselines (SSMT, MMA) across all language pairs. It achieves higher BLEU scores at lower Average Lagging (AL) values. Furthermore, because IQ leverages pre-trained offline models and only requires training two small auxiliary networks, it is significantly more computationally efficient than training full simultaneous models from scratch.

Key Insights and Discussion

The success of IQ stems from its ability to generalize information quantification rather than just memorizing action patterns. This approach is robust to noise and offers unprecedented flexibility through the $\epsilon$ threshold.

However, the method's reliance on oracle sequences presents a limitation. The quality of the IQ module is intrinsically linked to the heuristic used to generate the oracle. Future work may involve using reinforcement learning to refine oracle generation or applying this information-based logic to other streaming tasks like simultaneous speech recognition.

Conclusion

The Information Quantifier provides a simple yet powerful way to adapt high-quality offline translation models for simultaneous tasks. By mathematically grounding the READ/WRITE decision in information theory, it offers a flexible, efficient, and high-performing solution to the quality-latency challenge in ST.

Summary updated

Abstract

Simultaneous Translation (ST) involves translating with only partial source inputs instead of the entire source inputs, a process that can potentially result in translation quality degradation. Previous approaches to balancing translation quality and latency have demonstrated that it is more efficient and effective to leverage an offline model with a reasonable policy. However, using an offline model also leads to a distribution shift since it is not trained with partial source inputs, and it can be improved by training an additional module that informs us when to translate. In this paper, we propose an Information Quantifier (IQ) that models source and target information to determine whether the offline model has sufficient information for translation, trained with oracle action sequences generated from the offline model. IQ, by quantifying information, helps in formulating a suitable policy for Simultaneous Translation that better generalizes and also allows us to control the trade-off between quality and latency naturally. Experiments on various language pairs show that our proposed model outperforms baselines.
BibTeX
@inproceedings{lee2023quantifying,
  author = {Donghyun Lee and Minkyung Park and Byung-Jun Lee},
  title = {{Quantifying Information of Tokens for Simple and Flexible Simultaneous Machine Translation}},
  booktitle = {CoNLL},
  year = {2023},
  doi = {10.18653/v1/2023.conll-1.14},
}
Figure