TL;DR
This paper presents SRI-B's end-to-end speech translation systems for the Marathi-Hindi low-resource track at IWSLT 2023. They achieved top performance by leveraging Conformer encoders and pre-training with Marathi ASR data, demonstrating significant improvements over Transformer-based models.
What Problem Does This Paper Address?
Speech translation (ST) has seen rapid progress, but this has largely been confined to high-resource languages. This leaves a significant portion of the world's population, who speak low-resource languages with limited speech data, underserved. The IWSLT 2023 Dialectal and Low-resource track aims to address this by encouraging research on such language pairs, specifically focusing on Marathi-Hindi in this paper. The challenge lies in building effective ST systems with scarce data, a problem that traditional cascaded ASR-MT systems struggle with due to error propagation and increased latency.
Methods
The authors developed end-to-end speech translation systems using two main architectures: a Transformer-based model and a Conformer-based model. Both systems employ an encoder-decoder structure. A key innovation is the pre-training of the encoder using Marathi Automatic Speech Recognition (ASR) data before training the full speech translation model. This pre-training step is crucial for improving performance, especially in low-resource settings. They also explored the impact of using different training data splits (train-only vs. train+dev) for the final ST model.
Datasets
The primary dataset used for training and evaluation is the Marathi speech to Hindi text translation data from the news domain, provided by the IWSLT 2023 organizers. This dataset includes train, development (dev), and test splits.
For ASR pre-training, the following datasets were utilized:
- Constrained Condition:
- Indian Language Corpora
- Common Voice
- OpenSLR
- Unconstrained Condition:
- All datasets from the constrained condition.
- IIIT-H Voices
- IITM Indic TTS
Key Insights and Ideas
- Conformers Shine: Conformer encoders significantly outperform Transformer encoders for speech translation tasks, leading to substantial improvements in BLEU scores. This suggests that the architectural advantages of Conformers, already proven in ASR, translate well to ST.
- ASR Pre-training is Key: Pre-training the encoder with Marathi ASR data is a critical step for boosting speech translation performance, especially in low-resource scenarios. This strategy helps the model learn robust speech representations before tackling the translation task.
- Leveraging Dev Data: Including the development split in the speech translation training process, in addition to the training split, leads to a notable increase in performance on the test set. This indicates that even limited additional data can be beneficial.
- End-to-End Advantage: The paper reinforces the benefits of end-to-end ST systems over traditional cascaded approaches, highlighting their reduced latency and architectural complexity.
Key Results
SRI-B's primary submissions achieved impressive results:
- Constrained Condition: A BLEU score of 31.2%, securing the top position.
- Unconstrained Condition: A BLEU score of 32.4%, securing the second position.
The paper quantifies the impact of their ideas:
- Replacing Transformer encoder blocks with Conformer encoder blocks increased the dev split BLEU score by 3.2 points.
- Encoder pre-training with Marathi ASR data improved BLEU scores by 8.5 and 11.8 points on Transformer and Conformer models, respectively.
- Utilizing the dev split for ST training boosted test split BLEU scores by 5.5 and 2.6 points for constrained and unconstrained conditions, respectively.
Limitations and Future Work
- Data Scarcity: The primary limitation, inherent to low-resource scenarios, is the limited availability of speech translation data. While pre-training and architectural choices help, more data would undoubtedly lead to further improvements.
- Future Directions: The authors express hope that their findings will be useful for other researchers and practitioners working on low-resource speech translation. They implicitly suggest that further exploration of pre-training strategies, architectural variations, and data augmentation techniques could yield even better results.
Summary updated