Leveraging Deep Learning to Shed Light on Tones of an Endangered Language: A Case Study of Moklen

Sireemas Maspong, Francesco Burroni, Teerawee Sukanchanon, Warunsiri Pornpottanamas, Pittayawat PittayapornACL 2024

TL;DR

This paper uses deep learning to analyze the acoustic cues of Moklen, an endangered tonal language. By employing a BiLSTM with self-attention, the study reveals that a combination of pitch and vowel quality is crucial for distinguishing Moklen's two lexical tones, achieving high classification accuracy. This work demonstrates the power of ML for understanding understudied languages and aiding linguistic documentation.

What Problem Does This Paper Address?

Moklen, an Austronesian language spoken in Thailand, presents a linguistic puzzle: while most Austronesian languages are not tonal, Moklen exhibits minimal pairs suggesting the presence of two lexical tones. Previous research has identified pitch, vowel quality, and voice quality as potential acoustic correlates for these tones, similar to register contrasts found in other Southeast Asian languages. However, the relative importance of these acoustic cues in distinguishing Moklen tones remained unclear. This ambiguity poses a challenge for both linguistic analysis and the development of automatic speech recognition systems for this endangered language. The paper aims to quantify the contribution of each acoustic feature to tone distinction and identify the most effective combination for accurate classification.

Methods

The researchers employed a Bidirectional Long Short-Term Memory (BiLSTM) Neural Network with self-attention for sequence classification. This architecture is well-suited for sequential data like speech. The self-attention mechanism allows the model to focus on the most relevant parts of the acoustic signal for tone classification.

To understand the contribution of different acoustic features, the study conducted a feature ablation study. This involved training and evaluating separate classification models using various combinations of acoustic features. The features extracted were:

  • Pitch: Fundamental frequency (f0).
  • Vowel Quality: First and second formant frequencies (F1, F2).
  • Voice Quality: Corrected first harmonics and corrected spectral amplitude of F3 (H1-A3), and Cepstral Peak Prominence (CPP).

The data was preprocessed by handling missing values with linear interpolation and then z-scoring each participant's data. Due to an imbalanced class distribution (more Tone 1 than Tone 2), the training data was augmented using time-warping and adding random Gaussian noise to create a more robust model. Hyperparameters were optimized using Bayesian Optimization.

Datasets

The study utilized a dataset of 98 attested Moklen words collected from eight native speakers. These words were systematically varied in terms of tone, onset voicing, vowel length, and coda categories. The dataset was imbalanced, with 74 words for Tone 1 and 24 for Tone 2, reflecting the natural distribution in the Moklen lexicon. The recordings were manually segmented, and acoustic measurements were extracted from the stressed vowel intervals.

Key Insights and Ideas

  • Synergy is Key: The most significant finding is that a combination of pitch and vowel quality provides the highest accuracy in distinguishing Moklen tones. This suggests that these two features work together synergistically, rather than independently, to convey tonal information.
  • Pitch Dominance: While multiple features contribute, pitch (f0) emerges as the primary cue for Moklen tone. Models that excluded pitch features performed significantly worse than those that included them.
  • Vowel Onset Salience: Analysis using Grad-CAM revealed that the vowel onset is the most critical part of the acoustic signal for tone classification. This aligns with observations in register contrasts, where the onset of the vowel is often where the distinction is most pronounced.
  • Linguistic Innovation: The paper posits that Moklen's tonal contrast might be an innovation that developed from reanalyzing existing contrasts like stress, which also utilize pitch and vowel quality. This provides a potential historical linguistic explanation for the observed acoustic correlates.
  • ML for Endangered Languages: The study successfully demonstrates the efficacy of deep learning for analyzing the phonetics of endangered languages, even with limited documentation. This approach can provide valuable insights for linguists and aid in the creation of resources like dictionaries.

Key Results

  • The best-performing model, using pitch and vowel quality features, achieved an overall classification accuracy of 84% and an F1-score of 0.89.
  • Models excluding pitch features had the lowest accuracy, confirming pitch as the primary cue.
  • The model using pitch and vowel quality also showed significantly better performance in classifying Tone 2 tokens (76.5% accuracy) compared to other models.
  • An error analysis indicated that vowel length and coda place of articulation significantly impacted classification accuracy, with long vowels and velar codas being more prone to misclassification.

Limitations and Future Work

  • Limited Time Interval: The study focused on acoustic features within the vowel interval. The authors suggest that analyzing a larger time interval, such as the entire syllable, might reveal additional tonal aspects, like f0 peak location, which could further improve classification.
  • Perceptual Studies: While the study analyzed acoustic features, investigating the perception of Moklen tones by speakers would provide deeper insights into the nature of this tonal system.
  • Understanding Error Factors: Further investigation is needed to understand why vowel length and coda place of articulation impacted the model's classification performance.
  • Broader Application: The authors highlight the potential of these models to assist fieldworkers in identifying tones in undocumented Moklen words and in creating dictionaries, which are crucial steps for language revitalization.

Summary updated

BibTeX
@inproceedings{maspong2024leveraging,
  author = {Sireemas Maspong and Francesco Burroni and Teerawee Sukanchanon and Warunsiri Pornpottanamas and Pittayawat Pittayaporn},
  title = {{Leveraging Deep Learning to Shed Light on Tones of an Endangered Language: A Case Study of Moklen}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure