TL;DR
This paper presents a system for low-resource speech translation between Maltese and North Levantine Arabic into English, leveraging a pipeline approach with ASR and MT components. The authors explore data augmentation through transliteration and multi-stage fine-tuning to improve performance, highlighting the challenges and potential of these under-resourced languages.
What Problem Does This Paper Address?
The paper tackles the significant challenge of speech translation for low-resource languages, which suffer from a severe lack of available data for natural language processing (NLP) tasks. This data scarcity creates a substantial gap compared to high-resource languages, impacting research and excluding entire speech communities. The authors focus on Maltese and North Levantine Arabic, two languages with linguistic similarities that they aim to exploit to improve translation quality. The core problem is how to build effective speech translation systems when parallel speech and text data are scarce, requiring innovative approaches to data utilization and model training.
Methods
The system employs a pipeline approach, separating Automatic Speech Recognition (ASR) and Machine Translation (MT).
- ASR: The authors fine-tune a Wav2Vec 2.0 XLS-R model, a powerful multilingual pre-trained model, on available Maltese and Modern Standard Arabic (MSA) data. To enhance ASR accuracy, they incorporate language models (n-grams) to correct the output transcriptions.
- MT: For machine translation, they utilize the NLLB (No Language Left Behind) model, a large multilingual translation model. They explore a two-stage fine-tuning process, first using a large general dataset (OPUS-100 for Maltese, MSA for Arabic) and then fine-tuning on task-specific data.
- Data Augmentation (Transliteration): A key innovation is the use of transliteration to bridge the gap between Maltese and Arabic. They adapt a method that maps Maltese tokens to Arabic script, aiming to leverage cross-lingual similarities. This transliterated data (MLTARA) is then used to augment the training data for the Arabic MT system.
Datasets
- ASR Training:
- Maltese: Common Voice 7.0, MASRI (approx. 50 hours total).
- North Levantine Arabic: Common Voice (approx. 50 hours), trained on MSA data due to the lack of specific North Levantine Arabic ASR data.
- MT Training:
- Maltese: Common Voice (CV), MASRI (CV+MASRI), OPUS-100.
- North Levantine Arabic: North Levantine (APC)-MSA-English textual data, IWSLT 2022 Tunisian Arabic (AEB), Arab-Acquis MSA-English parallel data, and transliterated Maltese (MLTARA).
- Evaluation: Development and test sets provided by the IWSLT 2024 shared task.
Key Insights and Ideas
- Transliteration as a Bridge: Transliterating Maltese into Arabic script is a promising strategy to leverage the inherent linguistic similarities between the two languages, effectively augmenting data for the Arabic MT system.
- Multi-Stage Fine-Tuning: A two-stage fine-tuning approach for MT, starting with a large general dataset and then fine-tuning on task-specific data, significantly improves performance compared to single-stage fine-tuning.
- Language Model Enhancement for ASR: Using n-gram language models to correct ASR output is crucial for improving transcription accuracy, especially in low-resource scenarios where initial ASR performance might be limited.
- MSA as a Proxy for Dialectal Arabic ASR: Due to the scarcity of North Levantine Arabic ASR data, using MSA data for training proved to be a viable, albeit imperfect, proxy, highlighting the challenges of dialectal variation.
Key Results
- Maltese ASR: The best Maltese ASR system achieved a Word Error Rate (WER) of 0.096 with a 6-gram language model.
- North Levantine Arabic ASR: The system struggled with dialectal variations, achieving a WER of 1.08, significantly higher than Maltese, due to the reliance on MSA data.
- Maltese MT: The best Maltese MT system achieved a BLEU score of 60.6 using a two-stage fine-tuning process with OPUS-100 and CV+MASRI.
- North Levantine Arabic MT: The best Arabic MT system achieved a BLEU score of 53.78 using a two-stage fine-tuning process with MSA, APC, AEB, and MLTARA.
- Speech Translation Pipelines:
- Maltese: The best pipeline achieved a BLEU score of 52.4.
- North Levantine Arabic: The best pipeline achieved a BLEU score of 53.78.
Limitations and Future Work
- Limited North Levantine Arabic ASR Data: The primary limitation was the lack of specific ASR data for North Levantine Arabic, forcing the use of MSA data which led to challenges with dialectal variations and non-standard pronunciation.
- Lack of Tunisian Arabic Test Data: The absence of testing data for Tunisian Arabic, a language closely related to Maltese, hindered a complete evaluation of cross-lingual transfer.
- Future Work:
- Hyper-parameter tuning for further optimization.
- Supplementing data with back-translation, especially using larger monolingual corpora.
- Exploring more advanced data augmentation techniques.
Summary updated