TL;DR
This paper explores data augmentation techniques to improve Automatic Speech Recognition (ASR) for Maltese, a low-resource language with limited training data. By combining unsupervised training, multilingual data, and synthesized speech, they achieved a significant 15% absolute Word Error Rate (WER) improvement, demonstrating the effectiveness of these methods for under-resourced languages.
What Problem Does This Paper Address?
Developing robust ASR systems typically requires vast amounts of transcribed speech data. However, many languages, like Maltese, lack such extensive resources, making it challenging to build high-performing ASR models. This paper tackles the problem of low-resource ASR by investigating various data augmentation strategies to effectively leverage limited available data and data from related, higher-resource languages. The goal is to find the most effective augmentation techniques when starting with a small, curated corpus (around 7 hours) of Maltese speech.
Methods
The researchers explored three primary data augmentation strategies:
- Unsupervised Training: This involves using transcriptions generated by an ASR system itself, which are inherently noisy. They experimented with both noisy transcriptions of Maltese and transcriptions of other languages generated by a Maltese ASR model.
- Multilingual Training: This strategy leverages data from typologically or historically related languages (Arabic, Italian, English) to improve Maltese ASR. They explored using both noisy transcriptions of these languages and, crucially, "gold" (human-verified) transcriptions from English.
- Training with Synthesized Data: This method involves generating artificial speech data using a Text-to-Speech (TTS) system. Even though the synthesized data was acknowledged as imperfect and noisy, the researchers investigated its utility.
A key aspect of their methodology was a pretraining and fine-tuning pipeline. Data augmentation was applied during the pretraining phase, and the models were then fine-tuned on the limited, high-quality MASRI-Headset corpus. They also deliberately excluded language model (LM) rescoring to isolate the impact of data augmentation on the acoustic model itself.
Datasets
The primary dataset for evaluation and fine-tuning was the MASRI-HEADSET corpus, a curated collection of approximately 7 hours of read Maltese speech.
For data augmentation, they utilized:
- MASRI-SYNTHETIC Corpus: Artificially generated Maltese speech using a diphone-based TTS system, creating approximately 99 hours of data.
- Maltese audiobooks: Approximately 12 hours of untranscribed Maltese audio.
- LibriSpeech: A large English ASR corpus (100 hours used).
- Arabic Speech MGB-2 corpus: A portion of an Arabic broadcast news corpus (100 hours used).
- M-AILABS Speech Dataset (Italian portion): Approximately 118 hours of Italian read speech.
Key Insights and Ideas
- Noisy data can be surprisingly effective: Even imperfect, automatically generated transcriptions (unsupervised or synthesized) can significantly boost ASR performance, especially when target language data is scarce.
- Gold standard data from related languages is powerful: Combining noisy Maltese data with high-quality, transcribed data from a related language like English yielded substantial improvements, highlighting the value of transfer learning.
- Language mixtures offer benefits, but with caveats: Mixing data from multiple related languages improved performance, but the gains were not always additive and could be less effective than using a single, high-quality related language dataset.
- Synthesized speech is a viable option: Despite its inherent noise and unnaturalness, synthesized Maltese speech provided significant gains, demonstrating that even low-quality synthetic data can be beneficial for pretraining.
- Focus on augmentation over architecture complexity: The study suggests that for low-resource scenarios, the choice of data augmentation strategies can be more impactful than using overly complex model architectures.
Key Results
The paper reports a significant absolute 15% improvement in Word Error Rate (WER) when combining the most effective data augmentation techniques.
- The baseline system (without augmentation) achieved a WER of 63.71%.
- The best augmented system achieved a WER of 48.97%.
- This improvement was achieved without the use of a language model, isolating the impact of data augmentation.
- Pretraining with 100 hours of noisy data from non-target languages (English, Arabic) yielded improved WER results compared to using only Maltese data.
- Mixing gold English transcriptions with noisy Maltese transcriptions resulted in a WER of around 55%, exceeding language mixture experiments.
- Using synthesized Maltese speech alone brought the WER down to 52%, and combining it with gold English data further reduced it to 48.97%.
Limitations and Future Work
- Limited Maltese data: The study acknowledges the inherent limitation of working with a very small amount of high-quality Maltese data (around 7 hours).
- Noisy synthesized data: The TTS system used produced imperfect and noisy speech, which, while beneficial, still represents a limitation.
- Exclusion of Language Models: The paper deliberately excluded LM rescoring to focus on data augmentation's impact on acoustic models. However, the authors note that LM rescoring would likely further improve results.
Future work suggested includes:
- Combining the best data augmentation strategies with more advanced neural architectures, particularly Transformer-based models like Wav2Vec 2.0.
- Integrating language modeling into future experiments to achieve state-of-the-art performance for Maltese ASR.
- Exploring the generalizability of these findings to other under-resourced languages.
- Continuing efforts in crowd-sourced data collection for Maltese.
Summary updated