Adapting prosodic chunking algorithm and synthesis system to specific style: the case of dictation

Elisabeth Delais-Roussarie, Damien Lolive, Hiyon Yoo, Nelly Barbot, Olivier RosecInterspeech 2014

TL;DR

This paper tackles the challenge of making speech synthesis systems sound natural and effective for specific tasks like dictation to young children. By analyzing how teachers dictate and then adapting a chunking algorithm and synthesis parameters, they achieve a more appropriate prosodic style without needing to train a new voice.

What Problem Does This Paper Address?

Speech synthesis systems are becoming increasingly sophisticated, but often struggle to adapt their prosody (intonation, rhythm, and speech rate) to specific communicative contexts. This is particularly true for educational applications, where a "one-size-fits-all" synthesized voice can be ineffective. The authors focus on the specific task of dictation for primary school pupils, a context requiring clear, well-paced speech with appropriate phrasing. Existing systems, even with high-quality voices, might deliver dictations too quickly or with unnatural chunking, hindering comprehension for young learners. The goal is to adapt an existing TTS system to this specific style, making it more suitable for educational dictation without the prohibitive cost of training entirely new voices.

Methods

The core of the approach involves a two-pronged strategy:

  1. Analysis of Teacher Dictations: The researchers collected and analyzed real dictations from teachers to primary school pupils. This involved:
  2. Corpus Study: Analyzing 10 dictations from teachers, focusing on how they segment the text into prosodic chunks and their speech rate.
  3. User Evaluation: Conducting a preliminary evaluation with children and teachers to gather feedback on the intelligibility, chunking, and speech rate of an existing synthesized dictation.
  4. Algorithm Adaptation and Preprocessing: Based on the analysis, they developed a preprocessing module to enrich the text before it's fed to the synthesis system. This module aims to:
  5. Improve Chunking: Refine the existing chunking algorithm to better reflect the prosodic boundaries identified in teacher dictations. This involved analyzing discrepancies between the algorithm's output and teacher annotations, particularly concerning the merging of function words and the handling of syntactic boundaries.
  6. Adjust Temporal Variables: Analyze and propose adjustments for articulation rate and pause durations. The goal is to slow down the speech and insert pauses strategically to enhance intelligibility for young learners.

Datasets

  • Preliminary Evaluation Data: Synthesized dictation stimuli evaluated by 7 children (cycle 2, 6-8 years old) and 14 teachers.
  • Main Analysis Data:
    • A written corpus of 38 sentences from 10 dictations for cycle 2 pupils, manually chunked by 10 teachers and then automatically chunked by their algorithm.
  • An audio corpus of these 38 sentences recorded by 3 teachers.
  • An audio corpus of the same 38 sentences produced by their synthesis system using the adapted chunking algorithm.

Key Insights and Ideas

  • User Feedback is Crucial: The initial evaluation highlighted that pupils found the synthesized speech too fast and the chunks too long, even if the overall meaning was understood. This directly informed the subsequent research direction.
  • Teachers and Pupils Differ: While teachers emphasized syntactic boundaries for chunking, pupils were more concerned with overall chunk length. This suggests a need to balance linguistic accuracy with perceptual ease for the target audience.
  • Chunk Size is Comparable, Boundary Placement is Key: The analysis showed that the length of chunks produced by the algorithm was similar to teachers, but the placement of boundaries, especially concerning function words and syntactic structures, needed refinement.
  • Micro-Pauses for Intelligibility: The research suggests that inserting very short pauses (micro-pauses) at word boundaries, even to block liaisons, can significantly improve intelligibility for young learners by effectively slowing down the speech.
  • No New Voice Needed: A significant finding is that by adapting the preprocessing and temporal parameters, the existing high-quality synthesized voice can be made suitable for the dictation task, avoiding the cost and effort of training a new voice.

Key Results

  • The adapted chunking algorithm shows comparable chunk lengths to teacher dictations.
  • The algorithm's boundary predictions are observed in 80% of teacher annotations, with a recall of 70%, indicating good agreement.
  • The articulation rate of the synthesized voice is comparable to some teachers' rates and is deemed acceptable, though pupils still found it too fast.
  • Analysis of pauses revealed they constitute a significant portion of dictation duration (63-68%) and their duration is influenced by syntactic position.
  • The proposed preprocessing module can insert punctuation, indicate pause durations, and block certain liaisons, leading to a more appropriate prosodic style for dictation.

Limitations and Future Work

  • Limited Number of Speakers: The analysis of temporal variables was initially restricted to two speakers (FD and ER) due to their slower tempo. While the synthesized voice's AR was comparable to one teacher (AP), further analysis with more speakers would be beneficial.
  • Algorithm Refinement: The paper identifies the merging mechanism in the chunking algorithm as a key area for improvement, particularly how single-word chunks are regrouped. Future work will focus on constraining this mechanism by morpho-syntax.
  • Further Temporal Variable Analysis: While articulation rate was deemed acceptable, the study suggests that incorporating a duration model or further refining pause insertion could enhance the outcome.
  • Broader Style Adaptation: The current work focuses on dictation. Future research could explore adapting the system to other specific styles (e.g., storytelling, reading aloud) using similar analysis and adaptation methodologies.

Summary updated

BibTeX
@inproceedings{delaisroussarie2014adapting,
  author = {Elisabeth Delais-Roussarie and Damien Lolive and Hiyon Yoo and Nelly Barbot and Olivier Rosec},
  title = {{Adapting prosodic chunking algorithm and synthesis system to specific style: the case of dictation}},
  booktitle = {Interspeech},
  year = {2014},
}

Loading related content...

Figure