XMU’s Simultaneous Translation System at NAACL 2021

Shuangtao Li, Jinming Hu, Boli Wang, Xiaodong Shi, Yidong ChenNAACL 2021

TL;DR

This paper presents XMU's systems for Chinese-to-English simultaneous translation, tackling both text-to-text and speech-to-text scenarios. Their core innovation lies in an adaptive segmentation policy that mimics human interpreters, allowing for controlled latency without sacrificing translation quality by intelligently deciding when to translate segments.

What Problem Does This Paper Address?

Simultaneous translation, the real-time rendering of spoken or written content into another language, is a highly sought-after capability for international communication. However, achieving this in machine translation is challenging due to the need to balance translation quality with minimal latency. Traditional approaches often require waiting for complete sentences, leading to significant delays. This paper addresses the challenge of enabling near-instantaneous translation by developing systems that can translate segments of text or speech as they become available, much like a human interpreter.

Methods

The paper details two systems, both built upon the Transformer architecture.

  • Text-to-Text System: This system employs an "adaptive segmentation policy" inspired by human interpreters. It uses a Meaningful Unit (MU) classifier to identify segments of the source text that have a clear and definite meaning and are unlikely to be significantly altered by subsequent words. Once an MU is identified, a machine translation (MT) model (based on FAIRSEQ) translates it using forced decoding, prepending the previously translated text. The key to controlling latency is the MU classifier's ability to predict when a segment is an MU, allowing for translation before the entire sentence is received. The size of the "future words" considered by the MU classifier and a confidence threshold are parameters that directly influence latency and quality.
  • Speech-to-Text System: This system is a pipeline that leverages the text-to-text system for its machine translation component. It consists of three stages:
  • Speech Recognition: Utilizes Baidu's real-time speech recognition API. Notably, the ASR output segments sentences but does not include punctuation.
  • Punctuation Restoration: A BERT-based sequence labeling model is trained to add punctuation (comma, period, question mark, or no punctuation) to the ASR output, as punctuation is crucial for the MT model.
  • Streaming Machine Translation: The output from the punctuation restoration stage is fed into the text-to-text simultaneous translation system described above.

Datasets

The systems were trained and evaluated using the following datasets:

  • CWMT19: A large dataset of 9.1 million parallel sentence pairs.
  • Baidu Speech Translation Corpus: Used for fine-tuning the text-to-text model and as the development set for evaluating both systems.
  • Simultaneous Translation Corpus: Provided by the workshop organizer, containing 39,000 parallel sentence pairs.

Key Insights and Ideas

  • Mimicking Human Interpreters: The core idea of using an MU classifier to segment text mirrors how human interpreters operate, breaking down information into translatable chunks. This adaptive approach is key to achieving simultaneous translation.
  • Controllable Latency: The MU classification threshold (t) and the number of future words considered (m) offer direct control over the trade-off between translation latency and quality. A higher threshold leads to longer waits but potentially better translation as segments are more reliably "meaningful."
  • Pipeline Efficiency: The speech-to-text system effectively reuses the robust text-to-text translation component, demonstrating the modularity and power of the adaptive segmentation approach.
  • Punctuation's Importance: The speech-to-text system highlights the critical role of punctuation, even in ASR output, for effective downstream machine translation.

Key Results

The paper presents results in terms of translation quality (BLEU) and latency (Consecutive Wait - CW).

  • Text-to-Text: Figure 1 shows a clear trade-off: increasing the MU threshold (t) leads to higher BLEU scores (better quality) at the cost of increased CW (higher latency). The system can achieve a balance, as demonstrated by the curves.
  • Speech-to-Text: Figure 2 illustrates similar trends for the speech-to-text system. However, the BLEU scores are significantly lower than in the text-to-text track, primarily due to a high character error rate (35.3%) from the ASR component, which propagates errors to the MT stage.

Limitations and Future Work

  • ASR Performance: The speech-to-text system's performance is significantly hampered by the ASR component's errors, especially on challenging speech data.
  • MU Classification Improvement: The authors suggest improving the cooperation between ASR and MT, and enhancing the performance of the MU classification itself as future work. This could involve developing more sophisticated MU detection mechanisms or better integrating ASR confidence scores into the segmentation process.

Summary updated

BibTeX
@inproceedings{li2021xmus,
  author = {Shuangtao Li and Jinming Hu and Boli Wang and Xiaodong Shi and Yidong Chen},
  title = {{XMU’s Simultaneous Translation System at NAACL 2021}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure