TL;DR
This paper tackles the challenge of creating high-quality text-to-speech (TTS) for low-resource indigenous languages like Te Reo Māori and ‘Ōlelo Hawaiʻi, enabling them to run on resource-constrained edge devices. By leveraging knowledge transfer from a high-resource language (Spanish) and employing a compact end-to-end TTS model (Piper TTS), they demonstrate the feasibility of delivering these crucial accessibility tools to communities previously underserved by technology.
What Problem Does This Paper Address?
Modern text-to-speech (TTS) systems, while impressive, are often built for high-resource languages and rely on large, complex models that require significant computational power. This makes them impractical for users in remote areas or those with limited internet connectivity. Furthermore, speakers of low-resource languages, particularly endangered indigenous languages like Te Reo Māori and ‘Ōlelo Hawaiʻi, are often excluded from these advancements. Existing TTS for these languages is either unintelligible or forces users to rely on the colonizer's language. This paper aims to bridge this gap by developing efficient TTS models for Te Reo Māori and ‘Ōlelo Hawaiʻi that can run on edge devices, thereby empowering speakers of these languages with essential communication tools.
Methods
The core of this work lies in two key methodological advancements:
- Knowledge Transfer: To overcome the scarcity of training data for Te Reo Māori and ‘Ōlelo Hawaiʻi, the researchers employed a transfer learning approach. They first trained a base TTS model on a large, public domain Spanish dataset. This Spanish model was then "fine-tuned" using the smaller datasets of Te Reo Māori and ‘Ōlelo Hawaiʻi. The rationale behind this is the significant overlap in phonemic structures between these Polynesian languages and Spanish, allowing the model to learn general speech synthesis principles from Spanish before specializing in the target languages.
- Compact End-to-End TTS Model: The researchers utilized the Piper TTS framework, specifically an "x-low" model. This model is an end-to-end VITS-based system that has been compressed using knowledge distillation to achieve a small parameter count (7.07M). This compact size is crucial for deployment on edge devices like the Raspberry Pi 4. The end-to-end nature means the model directly converts text to speech waveforms, simplifying the pipeline compared to older methods that required separate vocoders.
Datasets
The following datasets were used:
- Spanish: Public domain single-speaker data from LibriVox (approximately 99.4 hours of a male voice).
- Te Reo Māori:
- Male voice: Recordings made by Te Hiku Media from radio studio interviews and readings (approximately 5.5 hours).
- Female voice: Recordings made by Pae Tū Ltd in a recording studio (approximately 2.4 hours).
- ‘Ōlelo Hawaiʻi: Data curated, prepared, and recorded by Hina Puamohala Kneubuhl (approximately 58 minutes of a female voice).
The paper highlights that the limited duration of the Te Reo Māori and ‘Ōlelo Hawaiʻi datasets underscores the necessity of the knowledge transfer approach.
Key Insights and Ideas
- Transfer Learning is Crucial for Low-Resource Languages: The success of the female Te Reo Māori model, despite its limited training data, strongly suggests that transfer learning from a high-resource language is a viable and effective strategy for developing TTS for endangered languages.
- Phonemic Overlap Enables Cross-Lingual Transfer: The paper hypothesizes and demonstrates that the shared phonemic inventory between Spanish and Polynesian languages facilitates the transfer of learned speech synthesis patterns.
- IPA Phonemisation as a Foundation: Developing custom IPA phonemisers for Te Reo Māori and ‘Ōlelo Hawaiʻi was a necessary preprocessing step, as existing tools like eSpeak-ng did not provide adequate alignment with linguistic literature.
- Edge Deployment is Achievable: The successful real-time synthesis on an Accent 1000 device demonstrates that high-quality TTS for these languages can indeed be deployed on resource-constrained edge devices.
Key Results
- Female Te Reo Māori Model: Achieved clear pronunciation and simulated key pronunciation features differentiating native from non-native speakers, meeting qualitative criteria.
- Male Te Reo Māori Model: Demonstrated good pronunciation but exhibited some anomalies, such as not always lengthening or emphasizing the particle "a" before pronouns, which are attributed to potential data quality issues.
- ‘Ōlelo Hawaiʻi Model: Received positive initial feedback, though some reviewers noted elongated "l" sounds and incorrect emphasis on certain words, potentially due to transfer learning influence.
- Real-time Factor: The synthesis on the Accent 1000 achieved a real-time factor of approximately 0.5 (synthesis duration / audio duration), indicating efficient performance.
Limitations and Future Work
- Data Quality for Male Te Reo Māori: Anomalies in the male Te Reo Māori voice are attributed to potential issues with recordings made outside of a studio. Future work includes improving data quality, potentially by fine-tuning with only studio-recorded data.
- ‘Ōlelo Hawaiʻi Data Volume: The limited training data for ‘Ōlelo Hawaiʻi (under an hour) is acknowledged as a potential factor in observed anomalies. Increasing this data volume is a key area for future improvement.
- Punctuation Noise: Noise generated by adjacent punctuation may be resolved by removing lower-quality male Te Reo Māori recordings from the training set and retraining.
- Qualitative Evaluation: While initial findings are promising, more thorough qualitative analysis and opinion scoring from native speakers are planned.
- Sub-Sentence Level Synthesis: To address potential delays with longer sentences, the researchers plan to investigate implementing synthesis at the sub-sentence level.
- Cross-Platform Support: The current tools are for Windows devices. Future work aims to explore compatibility with macOS and iOS, which currently lack easy voice library extension.
- Bilingual Speech: A significant future direction is the development of a bilingual speech package supporting Te Reo Māori, ‘Ōlelo Hawaiʻi, and English, including reliable language detection.
Summary updated