Modeling of sentence-medial pauses in bangla readout speech: occurrence and duration

Shyamal Das Mandal, Arup Saha, Tulika Basu, Keikichi Hirose, Hiroya FujisakiInterspeech 2010

TL;DR

This paper tackles the crucial problem of generating natural-sounding speech by modeling sentence-medial pauses in Bangla. The authors developed linear models to predict both the occurrence and duration of these pauses, achieving promising results that have been integrated into a Bangla Text-to-Speech system.

What Problem Does This Paper Address?

Creating human-like speech from text (Text-to-Speech, TTS) requires more than just correctly pronouncing words. Prosody – the rhythm, intonation, and pauses – plays a vital role in making synthesized speech intelligible and natural. While sentence-boundary pauses are relatively straightforward, sentence-medial pauses (those occurring within sentences) are more complex. Their occurrence and duration are influenced by various linguistic factors, and improper handling can lead to unnatural or even incomprehensible speech. This paper addresses the challenge of accurately modeling these sentence-medial pauses specifically for the Bangla language, a task that had not been extensively studied before.

Methods

The researchers employed a data-driven approach to model sentence-medial pauses. They analyzed Bangla readout speech to identify key factors influencing pause occurrence and duration. The core of their methodology involves:

  • Feature Extraction: They identified three primary linguistic features:
  • Phrase Type: Categorizing phrases into Noun Phrase (NP), Adjective Phrase (AJP), Adverb Phrase (ADVP), Post-positional Phrase (PP), and Verb Phrase (VP).
  • Phrase Length (l): The number of syllables in a phrase.
  • Distance (d): The number of words between the end of the current phrase and the end of its dependent counterpart, determined using a binary tree representation of sentence structure.
  • Linear Modeling:
    • Occurrence Probability: A linear model of the form Pc(X) = al + bd + c was used to predict the probability of a pause occurring at a phrase boundary, where X represents the phrase type.
  • Pause Duration: A linear model of the form Dx = αl + βd + γ was developed to predict the duration of a pause, again with X representing the phrase type.
  • Model Training and Evaluation: The models were trained on a corpus of Bangla readout speech and then evaluated on unseen test data to assess their predictive accuracy.

Datasets

  • Training Data: A corpus of Bangla text was read by 8 native Bengali speakers (2 male, 6 female) with 6 repetitions each. The text was specifically prepared by removing sentence-medial punctuation to allow speakers to naturally insert pauses.
  • Testing Data: A separate set of 17 sentences was recorded by 4 of the original informants, also with 6 repetitions, at the same speech rate.

Key Insights and Ideas

  • Linguistic Factors Matter: The study strongly confirms that phrase type, phrase length, and the distance to dependent phrases are significant predictors of sentence-medial pause behavior in Bangla.
  • Verb Phrases are Key: Verb Phrases (VPs) are particularly prone to pauses, with higher occurrence probabilities and longer durations compared to other phrase types.
  • Linearity is a Good Approximation: For the studied features, linear models proved to be effective in approximating both pause occurrence probability and duration, simplifying the modeling process.
  • Bridging the Gap for Bangla TTS: This research provides a foundational model for pause generation in Bangla TTS, addressing a critical gap in the language's speech technology landscape.

Key Results

  • Occurrence Prediction: The models achieved a high accuracy in predicting pause occurrence, correctly identifying 87% of phrase boundaries where pauses should occur.
  • Duration Prediction: The pause duration models could predict durations within ±100 ms for 80% of the sentence-medial pauses.
  • Model Integration: The developed pause models have been successfully integrated into an ESNOLA-based Text-to-Speech synthesis system for Bangla.

Limitations and Future Work

  • Limited Scope: The study focused on sentence-medial pauses in readout speech without specific emotional intonations. The models might need adjustments for more expressive speech.
  • False Insertion (FI) Issue: The models showed a tendency for False Insertion (predicting a pause where none occurred) slightly more than False Deletion. The authors suggest a simple rule adjustment (prioritizing pauses after VPs) could mitigate this.
  • Future Directions: The authors imply that further refinement and integration into a full TTS system would be the next logical step, building upon these foundational pause models.

Summary updated

BibTeX
@inproceedings{mandal2010modeling,
  author = {Shyamal Das Mandal and Arup Saha and Tulika Basu and Keikichi Hirose and Hiroya Fujisaki},
  title = {{Modeling of sentence-medial pauses in bangla readout speech: occurrence and duration}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure