TL;DR
This paper introduces a "robust Viterbi algorithm" for unit selection Text-to-Speech (TTS) systems. By allowing the algorithm to skip potentially poor-quality speech units during the search, it prevents them from degrading the overall synthesis quality. These skipped units are then replaced using Hidden Markov Model (HMM)-based TTS, leading to demonstrably improved speech quality.
What Problem Does This Paper Address?
Traditional unit selection TTS systems excel at producing natural-sounding speech by concatenating pre-recorded units. However, they face a significant challenge: limited database coverage means that for certain phonetic contexts, no suitable units might be available. When this happens, the standard Viterbi algorithm, used for finding the best sequence of units, is forced to select suboptimal units. This can lead to audible mismatches and a decrease in overall speech quality. While HMM-based TTS offers more consistent quality and avoids database coverage issues, it often results in less natural-sounding speech. This paper aims to bridge this gap by combining the strengths of both approaches.
Methods
The core innovation is the application of a robust Viterbi algorithm (RVA) to the unit selection TTS framework. Unlike the standard Viterbi algorithm, RVA can be configured to skip a predefined number of units (denoted by K) during the search. This "skipping" capability allows the algorithm to ignore units that are deemed unsuitable (i.e., have a high cost) without letting them negatively impact the selection of subsequent units.
The process works as follows:
- Cost Calculation: For each candidate unit, target and join costs are calculated, similar to traditional unit selection.
- Robust Viterbi Search: The RVA explores possible unit sequences, considering the option to skip units. It calculates the minimum cost for retaining a unit and the minimum cost for excluding it. The overall cost for a unit is the minimum of these two options.
- Unit Replacement: When the RVA identifies units that are skipped (due to poor quality or unavailability), these units are replaced using an HMM-based TTS system. Crucially, the HMM-TTS system uses the same parameterization as the unit selection system, ensuring a smooth transition in voice quality.
The paper also details the cost function formulation, which uses Mahalanobis distances between candidate unit parameters and corresponding HMM models for target costs, and similar distances for join costs.
Datasets
The evaluations were conducted using two distinct datasets:
- English: A publicly available database from the CMU ARCTIC collection (speaker "slt").
- Finnish: A prosodically rich male voice database comprising approximately 650 sentences.
Labels for the data were automatically aligned using HMM models trained for synthesis.
Key Insights and Ideas
- The "Forced Choice" Problem: The standard Viterbi algorithm's inability to skip bad units is a critical bottleneck in unit selection TTS, leading to averaged quality or the inclusion of noticeably poor units.
- RVA as a "Quality Filter": The robust Viterbi algorithm acts as an intelligent filter, identifying and allowing the exclusion of problematic units without compromising the search for the best overall sequence.
- Seamless Hybridization: By using the same parameterization for both unit selection and HMM-based TTS, the replacement of skipped units with HMM-generated speech results in a smooth transition in voice quality and prosody, avoiding abrupt changes.
- Efficiency of RVA: While RVA theoretically increases computational load, the paper suggests that shared cost calculations and path backtracking can mitigate this overhead.
Key Results
The primary result is a significant improvement in synthesis quality, as demonstrated by listener evaluations.
- Preference Scores: In pairwise comparison tests, the proposed approach (RVA + HMM replacement) was preferred over the traditional Viterbi algorithm.
- Finnish voice: 78.6% preference (± 9.9% confidence interval).
- English voice: 78.3% preference (± 10.7% confidence interval).
- Qualitative Improvement: The RVA approach prevents single outliers from negatively impacting the rest of the synthesized sentence and allows for the effective replacement of poor-quality units.
Limitations and Future Work
- Optimal Skip Number: Determining the optimal number of units to skip (K) is noted as a challenge. The paper suggests that the change in total cost due to skipping could be used as a criterion, but this requires further refinement.
- Difficult Segments: The paper acknowledges that some phonetic segments might be inherently difficult to model with HMM-TTS, leading to poor quality even in the replacement stage. Future work could involve detecting and handling these difficult segments more explicitly.
- Comparison with Other Hybrid Methods: While the paper contrasts its approach with traditional unit selection and HMM-TTS, it briefly mentions other hybrid methods and notes that some approaches combining synthetic and real units can lead to less naturalness due to quality variations at switching boundaries. The proposed method aims to mitigate this.
Summary updated