Abstract
Simultaneous translation is a critical yet challenging NLP task that requires generating translations in real-time as source text arrives. Traditional approaches rely on either rigid fixed-latency policies or complex, unstable reinforcement learning (RL) frameworks. We propose a simplified single-model approach that incorporates a special “delay” token into the target vocabulary. By utilizing a restricted dynamic oracle to guide training through imitation learning, our model achieves flexible, adaptive policies. Experimental results on Chinese-English tasks demonstrate that this method yields superior translation quality (BLEU) and lower latency compared to both fixed and RL-based baselines.
Introduction
Simultaneous translation is often considered a "holy grail" of natural language processing, essential for real-time applications like international conferences. The primary hurdle is the structural divergence between languages, such as word order differences. For instance, translating from a verb-final language (like German) to a verb-initial one (like English) requires the system to decide whether to wait for the verb or attempt a guess.
Current methodologies generally fall into two categories:
- Fixed Policies: Systems like "wait-$k$" wait for a set number of source tokens before alternating between reading and writing. While simple, they lack the flexibility to adapt to varying sentence structures, often leading to premature guesses or unnecessary delays.
- Reinforcement Learning (RL): These models learn adaptive READ/WRITE policies. However, RL is notoriously difficult to train, typically requiring a multi-stage process where a translation model is pre-trained and then frozen while an RL agent is trained on top of it.
This research introduces a "middle way" that combines the simplicity of supervised learning with the flexibility of adaptive policies.
Methodology: Restricted Imitation Learning
The proposed approach transforms a standard Transformer Neural Machine Translation (NMT) model into a simultaneous translator through two key innovations.
The "Delay" Token
A special token, $\langle \epsilon \rangle$, is added to the target vocabulary. During decoding, if the model predicts a standard word, it performs a "WRITE" action. If it predicts $\langle \epsilon \rangle$, it performs a "READ" action, pausing to receive the next source word. This allows the model to manage its own latency dynamically.
Restricted Dynamic Oracle
To train the model, the authors developed a restricted dynamic oracle that provides an "expert" policy for the model to imitate. This oracle defines a valid "latency band" on a grid representing the alignment between source and target words.
The band is constrained by two parameters:
- $\alpha$ (Aggressive Bound): Ensures the model has seen enough source context before committing to a translation, preventing "hallucinations."
- $\beta$ (Conservative Bound): Prevents the model from waiting too long, ensuring the translation remains truly simultaneous.
The training process uses Restricted Imitation Learning, where the model learns to stay within this optimal band. A Sigmoid loss is employed instead of Softmax to allow the model to treat READ and WRITE actions as non-competing labels when both are valid according to the oracle.
Key Insights and Implementation
A significant advantage of this method is its test-time controllability. By adjusting a "temperature" constant ($t$) for the delay token during inference, users can tune the model to be more "eager" (lower latency) or "patient" (higher quality) without retraining.
Additionally, the researchers addressed a noise issue in the Transformer's attention mechanism. They found that retaining all previous delay tokens in the history cluttered the attention layers. Their solution was to remove these tokens from the history but provide the model with an explicit count of previous delays via a separate embedding, maintaining latency awareness without the noise.
Results
The adaptive policy consistently outperformed both fixed wait-$k$ and RL-based models. On the NIST Chinese-English corpus, the model achieved higher BLEU scores at comparable latency levels, measured by Average Lagging (AL).
Qualitative analysis highlights the "smarter waiting" behavior of the adaptive policy. As shown in the comparison below, fixed policies like wait-1 often make incorrect lexical choices because they are forced to translate before the relevant source information is available.
Conclusion and Future Work
By treating the READ action as a vocabulary token and using a restricted oracle for imitation learning, this work simplifies the development of high-performance simultaneous translation systems. Future research directions include dynamically estimating the source-to-target length ratio during inference and integrating the model with speech-to-text systems to handle real-world acoustic features and disfluencies.
Summary updated