A flexible front-end for HTS

Matthew P. Aylett, Rasmus Dall, Arnab Ghoshal, Gustav Eje Henter, Thomas MerrittInterspeech 2014

TL;DR

This paper introduces "Idlak," a new, flexible front-end for the HTS speech synthesis system, built within the Kaldi toolkit. Idlak simplifies the process of creating context-dependent acoustic models for HTS by offering a modular design, XML-based configuration, and a more permissive licensing environment, aiming to make parametric speech synthesis more accessible for research and development.

What Problem Does This Paper Address?

Parametric speech synthesis systems, like HTS (Hidden Markov Model Speech Synthesis System), rely heavily on "front-end" processing to analyze text and extract linguistic and phonetic features that inform the acoustic models. The current front-ends for HTS are often complex, difficult to modify, and tied to restrictive licensing, hindering research into optimizing context features and developing new voices or languages. This paper addresses the challenge of creating a more accessible, flexible, and modern front-end that can generate full context models compatible with HTS, thereby facilitating experimentation and innovation in parametric speech synthesis.

Methods

The Idlak front-end is built within the Kaldi speech recognition toolkit, leveraging its modular architecture and C++ codebase. Key methodological aspects include:

  • XML-based Configuration: Idlak uses XML files for defining text processing rules, linguistic data, and context extraction, making it easier to modify and experiment with different settings.
  • Modular Design: The text processing pipeline is composed of independent modules (e.g., normalization, tokenization, syllabification, pronunciation) that can be chained together.
  • Kaldi Integration: By integrating with Kaldi, Idlak benefits from state-of-the-art ASR techniques and a permissive Apache 2.0 license.
  • Context Extraction for HTS: A dedicated module (idlakcex) is designed to generate context model names compatible with HTS, mapping Idlak's internal representations to HTS's required format.
  • Phrase-based Pausing: Idlak models pauses at the phrase level rather than utterance or sentence level, aiming to improve consistency and potentially naturalness.
  • Python-based Voice Building Framework: This framework allows for single-click, end-to-end voice building, with a specific module (hts_test) added for HTS compatibility.

Datasets

  • CMU General American Lexicon: Used as a basis for building the lexicon and letter-to-sound rules.
  • Arctic 16 kHz release of SLT: The voice data used for evaluation.
  • HTS-demo: The standard, publicly available HTS demo dataset from Lewis Carroll's "Alice in Wonderland" was used for experimental evaluation.

Key Insights and Ideas

  • Democratizing Parametric Synthesis: Idlak's flexible design and permissive licensing significantly lower the barrier to entry for researchers wanting to experiment with HTS front-ends and develop new voices.
  • XML as a Unifying Interface: Using XML for configuration and data representation simplifies the modification and extension of text processing pipelines.
  • Phrase-based Pausing: Modeling pauses at the phrase level, rather than utterance, appears to be a viable approach that doesn't degrade quality and might offer advantages.
  • Context is King (Still): While Idlak performs comparably to Festival with fewer contexts, the paper acknowledges that more contexts could potentially improve naturalness, suggesting further research into context selection is warranted.
  • Non-Native Speaker Perception: There's a marginal indication that non-punctuated pause insertion (as done by Idlak) might be more beneficial for the perceived naturalness of speech for non-native English speakers.

Key Results

  • Comparative Naturalness: The Idlak front-end achieved a Mean Opinion Score (MOS) of 2.83, which is comparable to the mature Festival front-end's MOS of 2.85 when evaluated on the HTS demo.
  • Pause Impact: While pause insertion by Festival did not significantly improve MOS overall, a separate analysis for non-native speakers showed a weakly significant positive effect for pause insertion with system.
  • Efficiency: Idlak's architecture makes it easier to add, alter, and experiment with contexts compared to traditional HTS front-ends.

Limitations and Future Work

  • Limited Contexts: The current Idlak front-end uses significantly fewer contexts (21 features) than standard HTS front-ends (which can use many more). The authors acknowledge that more contexts might improve naturalness, and further research is needed to identify optimal context sets.
  • Pause Insertion Nuances: While Idlak aims for pause-free output between phrases, Festival's pause insertion (based on n-grams) showed some positive informal feedback, particularly for non-native speakers. Further investigation into the optimal way to handle pauses is suggested.
  • Future Work: The authors plan to extend Idlak to more languages, explore the impact of context choice on speech synthesis quality more deeply, and build full context acoustic models using Idlak's front-end output to compare against baseline HTS demo output.

Summary updated

BibTeX
@inproceedings{aylett2014flexible,
  author = {Matthew P. Aylett and Rasmus Dall and Arnab Ghoshal and Gustav Eje Henter and Thomas Merritt},
  title = {{A flexible front-end for HTS}},
  booktitle = {Interspeech},
  year = {2014},
}
Figure