BIT’s system for AutoSimulTrans2021

Mengge Liu, Shuoying Chen, Minqin Li, Zhipeng Wang, Yuhang GuoNAACL 2021

TL;DR

This paper introduces a Chinese-English simultaneous translation system that tackles the trade-off between translation quality and latency. By segmenting input sentences and employing a multi-stage fine-tuning approach, their system achieves better performance than the baseline, demonstrating the effectiveness of their segmentation strategy.

What Problem Does This Paper Address?

Simultaneous translation, crucial for applications like live subtitling, faces a fundamental challenge: balancing translation speed (low latency) with accuracy. Traditional machine translation models wait for the entire sentence to be spoken before translating, leading to unacceptable delays. However, translating incomplete sentences introduces grammatical and semantic ambiguities, degrading quality. Furthermore, languages like Chinese and English have different word orders, adding complexity to reordering during translation. This paper aims to bridge this gap by developing a system that can translate quickly without sacrificing too much accuracy.

Methods

The core of their approach is a segmentation strategy. Instead of processing full sentences, they break down the incoming Chinese sentence into smaller, grammatically and semantically coherent sub-sentences. This allows translation to begin earlier.

The system comprises two main components:

  1. Sentence Boundary Detector: This acts as a text classifier, trained on a pre-trained Chinese BERT model. It identifies optimal points to segment the streaming input text into meaningful sub-sentences. They achieved a 92.5% accuracy in this task.
  2. Translation Module: This uses a Transformer-big model, implemented with the tensor2tensor framework. To optimize for the simultaneous translation task, they employ a multi-stage fine-tuning process:
  3. Pre-training: The model is initially trained on a large-scale Chinese-English corpus (CWMT19).
  4. Domain Fine-tuning: The model is further fine-tuned on a speech transcription corpus to adapt to the domain of the target test set, which differs from standard text.
  5. Sentence Length Fine-tuning: Crucially, the model is fine-tuned on sub-sentence pairs. This addresses the mismatch between full-sentence training and sub-sentence decoding during simultaneous translation. They explored different sub-sentence corpora, including those filtered for noise.

Sentence segmentation itself is guided by word alignment. They use the fast_align tool to obtain word alignments between Chinese and English sentence pairs. This alignment matrix is then used to identify segmentation points where there's no cross-word alignment between potential sub-sentences, indicating they can be translated independently.

Datasets

  • CWMT19: A standard Chinese-English text corpus used for initial pre-training.
  • BSTC: A corpus containing Chinese speech transcriptions and English translations, used for domain fine-tuning. They specifically used the Chinese and English text portions.
  • Organizer-provided corpus: Used for evaluating the segmentation strategy and for fine-tuning. This includes both full-sentence and sub-sentence data derived from golden transcriptions and ASR (Automatic Speech Recognition) outputs.

Key Insights and Ideas

  • Segmentation is Key: The paper strongly advocates for sentence segmentation as a viable strategy to reduce latency in simultaneous translation, showing it can outperform simpler "wait-k" models.
  • Multi-stage Fine-tuning is Crucial: Simply pre-training on a large corpus isn't enough. Adapting to the specific domain (speech) and the nature of the input (sub-sentences) through targeted fine-tuning significantly improves performance.
  • Noise in Sub-sentence Corpora is Problematic: Directly using all sub-sentences derived from word alignment can negatively impact performance. Filtering these sub-sentences to remove noise is essential for better results.
  • Word Alignment as a Segmentation Guide: Leveraging word alignment to identify independent sub-sentence pairs is an effective way to construct the training data for sub-sentence translation.

Key Results

The paper reports experimental results on a development dataset. While specific BLEU scores are presented in Table 3, the key takeaway is that their best-performing segmentation model (sub-sentence fine-tuned with filtered golden data) achieved better results than the baseline "wait-k" models provided by the organizer. Specifically, their domain fine-tuned model achieved an AL (latency metric) of 7.467 and a BLEU score of 19.45. Their best sub-sentence fine-tuned model (filtered golden) had an AL of 7.795 and a BLEU of 16.67, which, while slightly higher latency, still shows competitive performance. The paper states their final system "performed better than the baseline system."

Limitations and Future Work

  • Noise in Segmentation: The authors acknowledge that even after filtering, sub-sentence corpora can still contain noise, which can negatively impact translation quality. They note that shorter sub-sentences are more sensitive to errors.
  • Difficulty with Free Translation: They highlight cases where direct translation of a sub-sentence is difficult because the original full sentence's meaning was achieved through free translation or fluency considerations that are lost when segments are isolated.
  • Future Work: Not explicitly detailed, but the findings suggest further research into more robust segmentation techniques, better noise reduction strategies for sub-sentence corpora, and methods to handle context across segments would be beneficial.

Summary updated

BibTeX
@inproceedings{liu2021bits,
  author = {Mengge Liu and Shuoying Chen and Minqin Li and Zhipeng Wang and Yuhang Guo},
  title = {{BIT’s system for AutoSimulTrans2021}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure