BIT’s System for Multilingual Track

Zhipeng Wang, Yuhang Guo, Shuoying ChenACL 2023

TL;DR

This paper presents a straightforward yet effective end-to-end multilingual speech translation system for the IWSLT 2023 challenge. By combining convolutional neural networks (CNNs) for local feature extraction with a Transformer for global context, and crucially, pre-training the encoder with a speech recognition task, the system achieves competitive results across 10 target languages with a relatively simple architecture.

What Problem Does This Paper Address?

The paper tackles the challenge of multilingual speech translation (MST), specifically the "one-to-many" scenario where a single system translates speech from one source language (English in this case) into multiple target languages. While bilingual speech translation has seen significant progress, building a single model for multiple languages offers advantages like parameter efficiency and knowledge transfer from high-resource to low-resource languages. The authors aim to develop a robust and efficient MST system that can handle diverse linguistic structures and achieve good performance on a challenging benchmark.

Methods

The core of the BIT system is a CNN-Transformer architecture.

  • CNNs for Downsampling and Local Features: The input speech features (MFCCs) are first processed by a two-layer CNN. This serves to downsample the feature sequence, reducing its length and extracting local acoustic information.
  • Transformer for Global Context and Translation: The downsampled features are then fed into a Transformer encoder-decoder model. The Transformer's self-attention mechanism is key for modeling global dependencies within the speech and for generating the translated text.
  • Speech Recognition Pre-training: A crucial innovation is pre-training the Transformer encoder using a speech recognition (ASR) task. This leverages large amounts of ASR data to initialize the encoder's parameters, enabling it to better extract acoustic and semantic features before the MST training begins. This is particularly beneficial as the ASR task shares similar feature extraction needs with the encoder's role in speech translation.
  • Data Augmentation (SpecAugment): To improve robustness and diversity, SpecAugment is applied to the training data. This technique involves time warping and masking frequency or time bands in the spectrograms.
  • Model Ensembling: For final testing, the authors employed a parameter fusion technique, averaging the parameters from the last 10 training epochs to create a more robust model.

Datasets

  • Training Data: The system was trained using the MUST-C corpus (versions v1.0, v1.2, and v2.0), which provides speech-to-text translation data for multiple language pairs. The authors merged data for the 10 target languages.
  • Evaluation Data: The system was evaluated on the MUST-C test set for the 10 target languages.

Key Insights and Ideas

  • Pre-training is Key: Pre-training the encoder with a speech recognition task significantly improves performance. This highlights the benefit of leveraging related tasks and readily available data to bootstrap complex models.
  • Simplicity Can Be Effective: The authors demonstrate that a relatively simple CNN-Transformer architecture, combined with effective pre-training and data augmentation, can achieve competitive results without resorting to overly complex model modifications.
  • Language Differences Matter: The paper acknowledges that translation difficulty varies significantly across languages, even with similar amounts of data. This is attributed to differences in grammar rules and the challenge of consistent cross-lingual information sharing.

Key Results

  • The system achieved satisfactory results on the test sets of all 10 target languages in the MUST-C corpus.
  • The BLEU scores exceeded 20 for 5 of the 10 languages.
  • The paper reports Word Error Rates (WER) for the pre-trained ASR model, with scores ranging from 8.79% (Japanese) to 16.01% (Arabic).
  • BLEU scores on the MUST-C test set show a wide range, from 8.80 (Turkish) to 32.59 (French).

Limitations and Future Work

  • Language-Specific Performance: The system's performance varies across languages, with some (e.g., Arabic, Farsi, Japanese) scoring lower than others. This is attributed to grammatical differences and challenges in cross-lingual information sharing.
  • Future Work: The authors suggest that their system's simplicity and efficiency make it suitable for memory-constrained devices, implying potential for deployment in edge computing scenarios. Further research could explore more advanced optimization techniques or architectural modifications to further boost performance across all languages.

Summary updated

BibTeX
@inproceedings{wang2023bits,
  author = {Zhipeng Wang and Yuhang Guo and Shuoying Chen},
  title = {{BIT’s System for Multilingual Track}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure