The USYD-JD Speech Translation System for IWSLT 2021

Liang Ding, Di Wu, Dacheng Tao

Abstract

The USYD-JD team secured first place in the IWSLT 2021 Swahili-to-English speech translation task by leveraging a sophisticated pipeline that combines Automatic Speech Recognition (ASR) with a heavily optimized Neural Machine Translation (NMT) system. Their core contribution lies in a suite of data-centric strategies—including novel bidirectional and denoising pre-training—that yielded a massive 10.8 BLEU point improvement over a standard Transformer baseline.

Introduction

Speech Translation (ST) is notoriously difficult, especially in "low-resource" scenarios where parallel data (audio in one language paired with text in another) is scarce. The authors focus on the Swahili-to-English track of the IWSLT 2021 competition. The primary challenges include data scarcity for Swahili, error propagation where ASR errors derail subsequent translation, and domain mismatch between general training data and competition test sets.

Methodology

The authors employ a "divide and conquer" pipeline approach rather than an end-to-end model, focusing their innovation on the NMT component.

ASR and Post-processing

The system uses the Kaldi toolkit for speech recognition. To mitigate error propagation, a specialized model called SlotRefine is used to fix punctuation and casing errors before the text reaches the translator. This step is crucial for ensuring that numerical formats and sentence structures are preserved correctly during the translation process.

Example of translation refinement showing the correction of numerical formatting from the initial prediction to the post-processed output.

Advanced Pre-training and Augmentation

  • Bidirectional Pre-training (BiPT): The model is trained to translate in both directions simultaneously, fostering deeper bilingual representations.
  • Denoising Pre-training (DPT): The model learns to reconstruct clean sentences from "noised" versions, improving robustness to ASR errors.
  • Bidirectional Self-Training: An iterative process where "teacher" models generate synthetic data for "student" models.
  • Tagged Back-Translation: Large amounts of monolingual English data are translated back into Swahili. A special "tag" is added to these synthetic sentences to help the model distinguish them from authentic human translations.

Data and Experimental Setup

The team utilized a mix of officially provided and publicly available datasets. The parallel corpus totaled approximately 2.4 million sentence pairs, drawing from sources like CCAligned and Tanzil.

Summary of available parallel corpora used for training, totaling over 2.4 million sentence pairs.

In addition to parallel data, the team used ~4.3M English monolingual sentences from CommonCrawl and speech data from OpenSLR and IARPA Babel for the ASR component.

Results and Discussion

The system achieved the best sacreBLEU score of 25.3 on the official IWSLT 2021 test set, outperforming the baseline Transformer model by 10.8 BLEU points.

Ablation studies highlighted the impact of individual components:

  • Pre-training (BiPT + DPT): +1.7 BLEU
  • Bidirectional Self-Training: +2.3 BLEU
  • Tagged Back-Translation: +1.9 BLEU

The authors argue that combining Denoising and Bidirectional pre-training mimics human language acquisition. Furthermore, tagged back-translation proved vital in preventing the model from being "poisoned" by machine-generated phrasing.

Conclusion

While end-to-end models are increasingly popular, this work demonstrates that a well-tuned pipeline—specifically one that addresses the interface between ASR and NMT—can achieve state-of-the-art results. Future work will explore end-to-end models incorporating cross-modal information and adversarial training to further enhance robustness.

Summary updated

BibTeX
@inproceedings{ding2021usyd,
  author = {Liang Ding and Di Wu and Dacheng Tao},
  title = {{The USYD-JD Speech Translation System for IWSLT 2021}},
  booktitle = {IWSLT},
  year = {2021},
  doi = {10.18653/v1/2021.iwslt-1.22},
}
Figure