Adaptive Latency for Part-of-Speech Tagging in Incremental Text-to-Speech Synthesis

Maël Pouget, Olha Nahorna, Thomas Hueber, Gérard BaillyInterspeech 2016

TL;DR

This paper tackles the challenge of real-time speech synthesis (iTTS) by improving the accuracy of part-of-speech (POS) tagging, a crucial step for natural-sounding speech. They introduce an adaptive latency system that intelligently delays synthesis until a word's grammatical role is stable, balancing reactivity with accuracy. This approach leads to more natural-sounding synthesized speech chunks compared to immediate, word-by-word synthesis.

What Problem Does This Paper Address?

Incremental Text-to-Speech (iTTS) aims to make synthesized speech more interactive by delivering audio "on-the-fly" as a user types, mimicking real-time conversation. A major hurdle in iTTS is accurately processing text for speech synthesis when only a limited "left-context" (words already typed) is available, and the "right-context" (words yet to be typed) is unknown. This is particularly problematic for Part-of-Speech (POS) tagging, which assigns grammatical categories (like noun, verb, adjective) to words. POS tags are vital for correctly converting text to phonemes and determining prosody (intonation, rhythm). Traditional POS taggers rely on full sentence context, making them unsuitable for the limited context of iTTS. Without accurate POS tags, the synthesized speech can sound unnatural or even incorrect. This paper addresses the need for accurate POS tagging in iTTS without sacrificing too much interactivity.

Methods

The core of the proposed solution is an adaptive latency mechanism built around POS tag stability estimation. Instead of synthesizing each word immediately, the system uses a set of decision trees to predict whether the current POS tag for a word is likely to change once more right-context becomes available.

Here's a breakdown:

  • Incremental POS Tagging: The paper adapts standard POS tagging algorithms (specifically, the COMPOST tagger for French) to work with limited lookahead. They use the forward-backward algorithm to estimate POS tags based on available left-context.
  • POS Tag Stability Estimation: This is the novel contribution. They train three binary decision trees. Each tree is designed to predict the stability of a POS tag given a specific lookahead window (0, 1, or 2 words into the future). The input features for these trees include the current word's POS tag, its probability, and the tags and probabilities of preceding words. The output is a binary decision: "stable" or "unstable."
  • Adaptive Latency: The synthesis of a word is triggered only when its POS tag is deemed "stable" by the decision trees. If a tag is predicted to be unstable, the system waits, buffering the word and potentially subsequent words, until more context is available and the tag is confirmed. This results in variable-length speech chunks being synthesized, hence "adaptive latency." The maximum latency is limited to 3 words.

Datasets

The paper uses two French books for training and evaluation:

  • "Notre-Dame de Paris" by Victor Hugo
  • "Le tour du monde en 80 jours" by Jules Verne

This corpus consists of 20,154 sentences and 290,801 words. It was split into a training set (2/3) and a testing set (1/3). The decision trees were trained using Matlab.

Key Insights and Ideas

  • The "Right-Context" Problem is Key: The paper clearly articulates that the lack of right-context is the fundamental challenge for accurate POS tagging in iTTS, directly impacting speech naturalness.
  • POS Stability as a Proxy for Accuracy: Instead of trying to achieve perfect POS tags with limited context, the system focuses on predicting when a tag is likely to be correct. This pragmatic approach allows for intelligent delays.
  • Adaptive Latency is More Natural than Fixed Latency or Zero Latency: The perceptual evaluation strongly suggests that allowing synthesis to be grouped into variable-sized chunks (based on POS stability) leads to more acceptable prosodic phrasing than either synthesizing word-by-word (zero latency) or using a fixed lookahead.
  • Prosodic Phrasing Matters: The results highlight that even with accurate POS tags, how words are grouped into prosodic units significantly impacts perceived naturalness. The proposed method, while good, still has room for improvement in this area.

Key Results

  • Objective Evaluation:
    • Without any lookahead, the POS tagger (COMPOST) makes errors on ~40% of words.
  • With a 1-word lookahead, errors drop to ~9%.
  • With a 2-word lookahead, errors drop to <2%.
  • The decision trees for stability estimation achieve high accuracy: 92.0% for 0-word lookahead, 95.4% for 1-word, and 97.7% for 2-word lookahead.
  • The overall adaptive latency system achieves ~90% correlation with non-incremental tagging.
  • Perceptual Evaluation:
    • The proposed adaptive latency iTTS (WG4) was ranked second out of four strategies.
  • It was rated significantly better than word-by-word synthesis (WG1) and random grouping (WG2).
  • It was rated slightly lower than expert-defined prosodic grouping (WG3), indicating its effectiveness but also room for refinement.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the proposed method, while improving prosody, can still produce non-natural prosodic phrasing in some cases. An example given is the chunk "en retard en" from the sentence "Il arrive en retard en ce moment," which is a non-standard prosodic unit ("Preposition Noun Preposition"). This suggests that POS tag stability alone might not be sufficient for perfect prosodic chunking.
  • Future Work:
    • Improve prosodic phrasing by combining the current approach with predictive incremental parsing techniques.
  • Evaluate the performance of the adaptive latency POS-tagger on other text types, such as text messages and tweets, which often have different linguistic structures.

Summary updated

BibTeX
@inproceedings{pouget2016adaptive,
  author = {Maël Pouget and Olha Nahorna and Thomas Hueber and Gérard Bailly},
  title = {{Adaptive Latency for Part-of-Speech Tagging in Incremental Text-to-Speech Synthesis}},
  booktitle = {Interspeech},
  year = {2016},
}

Loading related content...

Figure