TL;DR
This paper introduces a novel method for automatically extracting parameters for a Fujisaki-style generation process model of fundamental frequency (Fo) contours. By using these extracted parameters to generate more natural Fo contours for training HMM-based speech synthesis models, the authors demonstrate significant improvements in synthetic speech quality.
What Problem Does This Paper Address?
HMM-based speech synthesis has become a popular technique due to its ability to produce high-quality speech and enable flexible control over prosody. However, a key limitation arises from the way fundamental frequency (Fo) contours are handled. While HMMs process Fo frame-by-frame, this often leads to over-smoothed or unnatural Fo contours, especially with limited training data. The Fujisaki generation process model offers a more linguistically grounded way to represent and generate Fo contours, but automatically extracting its parameters from observed speech has been a significant challenge. Previous methods struggled with accuracy and robustness, often failing to align extracted commands with linguistic information. This paper aims to bridge this gap by proposing a more effective automatic extraction method for these crucial Fo model parameters.
Methods
The core of this paper is a new algorithm for automatically extracting parameters for the Fujisaki generation process model. Unlike previous methods that focused on accent components first, this new approach is inspired by human speech perception and prioritizes extracting phrase components first. It also leverages linguistic information from the text, aligning with the data used in HMM-based speech synthesis.
The method involves three main steps:
- Pre-processing: This step focuses on cleaning the Fo contours by identifying and removing problematic data points within vowel segments, particularly those with large discontinuities. This ensures that the subsequent analysis is based on more reliable Fo information.
- Parameter Extraction: This is the most novel part. The algorithm first identifies "breath groups" (segments of speech separated by pauses). Within these groups, it classifies vowel segments as "Low" (L) or "High" (H) based on accent types. Phrase commands are then extracted by analyzing the "Low" parts, and accent command amplitudes are derived from the "High" parts. This approach directly incorporates linguistic information about accent phrases and types.
- Optimization: After the initial parameter extraction, a steepest descent method is used to refine the extracted command parameters. This optimization process is constrained to ensure that accent commands do not overlap and maintain a logical temporal relationship with phrase commands.
The generated Fo contours from this model are then used to train HMMs, replacing the original, potentially problematic, Fo contours from the speech corpus.
Datasets
The experiments were conducted using the ATR continuous speech corpus B set, a standard dataset for speech research, featuring a male narrator. The paper does not introduce a new benchmark dataset.
Key Insights and Ideas
- Human-inspired Extraction: Mimicking how humans might perceive prosody by focusing on phrase structure before accent details proves more effective for parameter extraction.
- Linguistic Information is Key: Integrating linguistic information (like accent phrase boundaries and types) directly into the extraction process significantly improves accuracy and robustness.
- Phrase-First Approach: Prioritizing phrase component extraction over accent component extraction leads to better overall results, particularly in handling subtle prosodic variations.
- Improved Training Data: Generating Fo contours using the Fujisaki model provides cleaner, more linguistically informed training data for HMMs, leading to better synthetic speech.
Key Results
- The proposed method achieved higher recall rates for both phrase and accent commands compared to a conventional method.
- While the precision for phrase commands was slightly lower, the authors argue this is due to over-extraction at accent phrase boundaries, which is compensated by accent components and doesn't degrade speech quality.
- Listening experiments showed a positive average score (0.143) for synthetic speech generated using the proposed method, indicating a preference for the improved quality over the conventional HMM approach.
Limitations and Future Work
The authors acknowledge that the Fujisaki model, even with their improved extraction, does not fully capture all aspects of Fo movements, particularly micro-prosody. They suggest that these residual Fo movements could be estimated separately and incorporated. Future work could also explore applying this method to other languages and further refining the integration of linguistic information for even more precise prosody control.
Summary updated