Explicit Tone Transcription Improves ASR Performance in Extremely Low-Resource Languages: A Case Study in Bribri

Rolando Coto-SolanoNAACL 2021

TL;DR

This paper explores how to best represent tonal information for Automatic Speech Recognition (ASR) in extremely low-resource languages, using Bribri as a case study. They found that explicitly separating tone from vowels significantly improves ASR accuracy, demonstrating the potential of ASR for documenting and revitalizing endangered languages.

What Problem Does This Paper Address?

Documenting and revitalizing indigenous languages often relies on transcribing audio recordings. This process is incredibly time-consuming, with experts spending up to 50 hours to transcribe just one hour of audio. Automated Speech Recognition (ASR) could significantly speed this up, but it struggles with low-resource languages that have limited training data. A key challenge in ASR for tonal languages (where pitch changes meaning) is how to represent the tone information effectively. This paper investigates different transcription methods for the Bribri language, a critically endangered Chibchan language, to find the optimal way to feed tonal data into ASR systems.

Methods

The researchers experimented with several ways to transcribe Bribri speech, focusing on how to represent tones and nasality. They tested two main ASR approaches:

  • Traditional GMM/HMM: Using the Kaldi toolkit, they trained models with different combinations of acoustic (monophone/triphone) and language models (unigram/bigram/trigram).
  • End-to-End CTC: Using the DeepSpeech toolkit with bidirectional RNNs, they explored two training strategies: "Just Bribri" (training solely on Bribri data) and "Transfer" (leveraging pre-trained English models).

The core of their methodology involved creating different "conditions" for transcribing the Bribri word "diki" (underneath), varying how tone and nasal features were encoded. These conditions ranged from combining all features into a single character (AllFeats) to separating tone and nasal information into distinct characters.

Datasets

The primary dataset used was the spontaneous speech corpus of Flores Solórzano (2017a), which contains 1571 utterances from 28 speakers, totaling 68 minutes of transcribed speech. This corpus was used for both training and validation. No new benchmarks were introduced; rather, the study focused on adapting existing ASR techniques to a new, low-resource language.

Key Insights and Ideas

  • Tone Separation is Key: The most significant finding is that explicitly separating tonal markings from vowel representations dramatically improves ASR performance. This allows the ASR system to learn tone patterns independently.
  • Low Tone Handling Matters: The "ToneSep" condition, which separates tones but implicitly treats the low tone (the most frequent in Bribri) as a default, yielded the best results in the GMM/HMM system. This suggests that simplifying the representation of the most common feature can be beneficial.
  • Nasality's Role: The impact of nasal vowels varied. In the GMM/HMM system, keeping nasality coupled with the vowel (as in ToneSep) was better than separating it. This hints that features that are phonetically intertwined with vowels might be best kept together.
  • End-to-End vs. Traditional ASR: The optimal transcription strategy flipped between the GMM/HMM and CTC systems. While ToneSep excelled in Kaldi, ToneNasSepWL (where both tone and nasal features were explicitly marked and separated) performed best in DeepSpeech. This highlights that the choice of ASR architecture influences the ideal input representation.
  • Transparency Aids Learning: The paper suggests that transparent representations (where features are clearly delineated) help ASR systems generalize phonetic cues, especially for low-resource languages.

Key Results

  • Significant Accuracy Improvements: The best transcription methods (like ToneSep) led to improvements of 4-26% in Character Error Rate (CER) and 3-25% in Word Error Rate (WER) compared to the worst-performing methods.
  • First Bribri ASR Models: This work represents the first attempt to train ASR models for Bribri. The best models achieved a CER of 33% and a WER of 50%.
  • GMM/HMM vs. CTC: In the GMM/HMM system, ToneSep achieved the best performance. In the CTC system, ToneNasSepWL performed best, with a CER of 70% for "Just Bribri" training.
  • Transfer Learning Limitations: Transfer learning from English models did not improve performance for Bribri in the DeepSpeech experiments, suggesting the Bribri dataset might be too small or that English phonetic knowledge doesn't transfer well.

Limitations and Future Work

  • Hand-Engineered Representations: The authors acknowledge that hand-engineered representations are suboptimal compared to fully data-driven approaches, but they are crucial for kickstarting ASR in low-resource settings.
  • Small Dataset Size: The limited amount of Bribri data (68 minutes) is a significant constraint, particularly for end-to-end models.
  • Transfer Learning Challenges: The lack of improvement from transfer learning needs further investigation.
  • Future Directions: The paper suggests continuing to experiment with other transcription methods, exploring different end-to-end architectures (like Listen, Attend and Spell, and wav2vec), and applying these findings to other Chibchan languages to aid in their documentation and revitalization.

Summary updated

BibTeX
@inproceedings{cotosolano2021explicit,
  author = {Rolando Coto-Solano},
  title = {{Explicit Tone Transcription Improves ASR Performance in Extremely Low-Resource Languages: A Case Study in Bribri}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure