Towards Stream Translation: Adaptive Computation Time for Simultaneous Machine Translation

Felix Schneider, Alexander WaibelACL 2020

TL;DR

This paper introduces Adaptive Computation Time (ACT) for simultaneous machine translation (SMT), enabling models to translate continuous text streams without pre-segmentation. By allowing the model to adaptively "ponder" input, they achieve state-of-the-art latency-quality trade-offs and demonstrate the first end-to-end SMT system for unsegmented input.

What Problem Does This Paper Address?

Simultaneous Machine Translation (SMT) aims to translate text as it's being received, balancing the need for speed with translation accuracy. Traditional SMT systems often rely on pre-segmented input (like sentences or utterances) and struggle with real-world scenarios where input is a continuous stream without clear boundaries, such as raw output from Automatic Speech Recognition (ASR). Existing methods either use fixed policies for deciding when to read input and write output, or learned policies that can be numerically unstable or not easily adaptable to modern Transformer architectures. This paper tackles the challenge of building an end-to-end SMT system that can handle unsegmented input streams effectively, improving both latency and quality.

Methods

The core innovation is the application of Adaptive Computation Time (ACT), originally developed for RNNs, to the Transformer architecture for SMT. ACT allows the model to dynamically decide how many "sub-steps" to spend processing each input token before producing an output token. This is achieved by a learned policy that predicts a probability distribution over these sub-steps.

Key methodological contributions include:

  • ACT for Transformer SMT: Adapting ACT to the encoder-decoder attention mechanism in Transformers. Instead of the decoder pondering RNN states, it now ponders encoder states.
  • Monotonicity Constraint: Ensuring that the ACT policy maintains monotonic advancement through the input stream, preventing the model from "going back" in the input.
  • Transformer-XL Integration: Incorporating Transformer-XL's relative attention and memory mechanisms to handle long, continuous streams and simulate a history buffer, crucial for unsegmented input where sentence boundaries are unknown.
  • Ponder Loss: An additional loss function is introduced to incentivize the model to keep its "ponder" times short, further improving latency.

Datasets

  • WMT 2014 German-to-English: Used for segmented translation experiments, comparing against previous state-of-the-art.
  • IWSLT 2020: A larger dataset including document-level data, used for training the stream translation model due to its extensive context.
  • IWSLT tst2010 German-to-English: Used as the test set for stream translation evaluation.

Key Insights and Ideas

  • ACT is a viable alternative to MILk: The paper demonstrates that ACT can achieve state-of-the-art results, offering a more numerically stable approach than Monotonic Infinite Lookback Attention (MILK) when adapted to the Transformer architecture.
  • End-to-end stream translation is possible: This work presents the first end-to-end SMT model capable of handling unsegmented input, a significant step towards real-world applications like translating live speech.
  • Adaptive "pondering" is key: The ability of ACT to dynamically adjust computation time per input token is crucial for balancing latency and translation quality, especially in continuous streams.
  • Transformer-XL features are vital for streams: Relative attention and memory mechanisms are essential for maintaining context and handling long sequences without sentence boundaries.
  • Latency metrics need careful consideration for long streams: Standard latency metrics like Average Lagging (AL) and Differentiable Average Lagging (DAL) may not fully capture the performance on very long, unsegmented sequences, suggesting a need for more robust evaluation.

Key Results

  • Segmented Translation: The ACT+XL model achieved state-of-the-art latency-quality trade-offs on the WMT 2014 German-to-English dataset, outperforming MILK. For example, with a Differentiable Average Lagging (DAL) of around 4, their model achieved a BLEU score of approximately 27.7, while MILK achieved around 27.0 at a similar DAL.
  • Stream Translation: The proposed ACT+XL model demonstrated promising results on the IWSLT dataset, achieving a BLEU score of 26.4 with a DAL of 394. While this represents a drop compared to segmented translation (around 4 BLEU points), it is presented as the first successful end-to-end demonstration of unsegmented stream translation.
  • Parameter Sensitivity: The paper notes that the $\lambda$ parameter (weighting the ponder loss) is sensitive for stream translation, with values below 0.25 leading to poor results due to excessive waiting and skipping.

Limitations and Future Work

  • Performance Gap: The stream translation model still exhibits a performance drop compared to sentence-based models.
  • Latency Metric Limitations: The authors found that standard latency metrics like AL and DAL may not accurately reflect the actual delay in very long, unsegmented sequences.
  • Future Work:
    • Training the entire model end-to-end without multiple pre-training stages.
  • Improving ACT latency decisions by adding more layers to the ACT attention module.
  • Adapting the model for speech translation, which is seen as a critical next step for realistic applications.
  • Investigating more robust latency metrics for long streams.

Summary updated

BibTeX
@inproceedings{schneider2020towards,
  author = {Felix Schneider and Alexander Waibel},
  title = {{Towards Stream Translation: Adaptive Computation Time for Simultaneous Machine Translation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure