TL;DR
This paper introduces a novel two-stage algorithm for reducing the size of speech databases used in corpus-based Text-to-Speech (TTS) systems. By intelligently selecting speech segments based on phonetic element coverage and acoustic variations (like F0), they demonstrate significant database reduction (e.g., from 13 hours to 10 hours) with no loss in synthesized speech quality, and even maintain high quality with much smaller databases.
What Problem Does This Paper Address?
Modern corpus-based TTS systems rely on large, meticulously recorded speech databases to achieve natural-sounding speech. However, these large databases are expensive to create and store, posing a significant barrier to practical application. The challenge is to reduce the size of these databases without sacrificing the quality of the synthesized speech, ensuring that the essential phonetic and prosodic elements needed for natural synthesis are preserved. Existing methods for selecting sentences for databases often struggle with estimation errors in text analysis or prosody, leading to degraded synthesis quality.
Methods
The authors propose a two-stage greedy algorithm for speech database reduction.
- Stage 1: Basic Element Coverage: This stage prioritizes sentences that contribute to the coverage of basic phonetic elements (syllables, denoted as SYL). It calculates a score for each sentence based on how much it increases the coverage of SYL in the reduced database. Sentences with the highest scores are selected iteratively.
- Stage 2: Expansion Element Coverage: When multiple sentences offer the same SYL coverage increase, this stage uses a weighted combination of scores for "expansion elements." These expansion elements include:
- CV: Syllables with various F0 variations.
- S: Syllables with F0 variations and phoneme context.
- M: Morphemes with phoneme context. A weighted sum of the coverage increase for these elements (s(CV), s(S), s(M)) determines the sentence selection. If scores are still tied, the shortest sentence is chosen. This process loops until the desired database size is reached.
Datasets
The paper uses a large, hand-labeled speech database as the "source speech database." This database, approximately 13 hours in duration, was generated using the authors' previously developed recording script generation method, ensuring a wide variety of speech elements. For evaluation, reduced speech databases of various sizes (10, 8, 6, 4, 2, 1, and 0.5 hours) were created. Additionally, two smaller, task-specific databases (A and B) were generated for a task-specific reduction experiment.
Key Insights and Ideas
- Phonetic Element Coverage is Key: The core idea is that intelligent selection based on the coverage of specific phonetic elements (basic and expansion) is more effective than random selection for maintaining speech quality.
- Expansion Elements Matter for Quality: While basic syllable coverage is important, the inclusion of expansion elements (CV, S, M) that capture F0 variations and phoneme context significantly contributes to preserving naturalness, especially as database size is reduced.
- Weighted Combination is Optimal: Using a weighted combination of expansion element scores (set(d) where all weights are 1.0) provides a better overall coverage of phonetic elements compared to focusing on a single expansion element.
- Task-Independent Databases are Generally Better: For task-specific TTS applications, using a reduced database derived from a task-independent source database can lead to better or equivalent performance compared to a database derived from a task-specific source.
Key Results
- Significant Reduction with No Quality Loss: The proposed method reduced a 13-hour database to 10 hours with no degradation in synthesized speech quality, as confirmed by listening tests.
- High Quality with Smaller Databases: Even with databases as small as 8 and 6 hours, the synthesized speech achieved a Mean Opinion Score (MOS) of over 3.5, representing 95% of the MOS achieved with the full-size database.
- Superior to Random Selection: The proposed method consistently outperformed random selection in terms of phonetic element coverage, especially for smaller database sizes.
- Task-Specific Experiment: For newscast and newspaper tasks, there was no significant difference in preference between databases derived from task-independent vs. task-specific sources. However, for mixed tasks, the task-independent source database was preferred.
Limitations and Future Work
- Limitations: The paper acknowledges that for very small database sizes (e.g., 0.5 hours), there can still be significant differences in MOS compared to the full database, although the proposed method still shows an advantage over random selection.
- Future Work: The authors plan to investigate the difference in opinion scores between databases created using their recording script method and equivalent databases created using their reduction method. They also suggest further exploration into optimizing the weight coefficients for expansion elements.
Summary updated