TL;DR
This paper tackles the challenge of automatically pinpointing phonetic boundaries in Mandarin Chinese speech, a crucial step for building natural-sounding speech synthesis systems. They propose a hybrid approach that combines statistical pattern recognition with a specialized heuristic method for difficult "voiced-to-voiced" transitions, significantly improving accuracy over existing methods.
What Problem Does This Paper Address?
Creating high-quality, natural-sounding speech synthesis systems often relies on concatenating pre-recorded speech segments. To do this effectively, these segments need to be precisely labeled with phonetic information, including the exact start and end times of each sound (phonetic boundaries). Manually labeling large speech corpora is incredibly time-consuming and expensive. While automatic methods exist, they often struggle with accuracy, especially for certain types of sound transitions. This paper focuses on improving the accuracy of automatic phonetic boundary detection and labeling for Mandarin Chinese speech, aiming to reduce the human effort required for corpus creation.
Methods
The authors propose a two-pronged approach:
- Statistical Pattern Recognition (SPR) with Feature Selection:
- They start with a large set of candidate phonetic transitions and manually labeled boundaries from a training corpus.
- For each transition type, they systematically select the most discriminative acoustic features (out of seven candidates like zero-crossing rate, log energy, pitch, MFCCs, etc.) using a Sequential Forward Selection (SFS) algorithm.
- A K-Nearest Neighbor Rule (KNNR) classifier with Leave-One-Out (LOO) cross-validation is used to evaluate feature sets and determine classification rates.
- This SPR approach works well for most transitions.
- Hybrid Approach for Difficult "P+P" Transitions:
- They identify that transitions between two "periodic voiced" sounds (labeled "P+P") are particularly challenging and have low accuracy with the SPR method alone.
- They further divide these "P+P" transitions into "better" (group B) and "worse" (group W) based on initial SPR performance.
- For the "worse" group (group W), they develop a heuristic rule-based method. This method involves:
- Identifying candidate boundaries around an initial HMM-based estimate.
- Searching for a region with lower log energy.
- Within that low-energy region, selecting the boundary with the maximum absolute difference in MFCCs between its left and right frames.
Datasets
- TRAIN3700: A speech corpus containing approximately 3700 syllables from 30 long sentences, manually labeled for phonetic boundaries. This dataset is used for training the SPR models and feature selection.
- TTS-455: A corpus of 455 sentences (about 6000 syllables) with manually labeled phonetic boundaries. This dataset is used for evaluating the performance of the proposed boundary refinement methods.
- TCC-300: A corpus used to train HMM models for initial forced alignment.
Key Insights and Ideas
- Not all acoustic features are created equal: Different phonetic transitions benefit from different sets of acoustic features. A systematic feature selection process is crucial for optimizing performance.
- "P+P" transitions are a bottleneck: The statistical pattern recognition approach, while effective for many transitions, struggles significantly with transitions between two periodic voiced sounds. This highlights a specific area requiring specialized handling.
- A "divide and conquer" strategy is effective: By separating the problematic "P+P" transitions and applying a tailored heuristic method to the most difficult subset, overall accuracy can be substantially improved.
- Log energy and MFCC differences are useful heuristics: The proposed heuristic for "P+P" transitions leverages the observation that boundaries often occur in lower energy regions and can be refined by looking at spectral changes (MFCC differences).
Key Results
- The SPR approach (using SFS, KNNR, LOO) achieved good classification rates for most phonetic transitions, with some reaching over 95%.
- However, "P+P" transitions had significantly lower classification rates, around 63-64%.
- The hybrid approach, combining SPR with the heuristic rule for the "worse" "P+P" cases, showed a significant improvement in segmentation accuracy for "P+P" transitions on the TTS-455 test corpus, outperforming even HMM segmentation alone.
- Figure 5 visually demonstrates this improvement, showing the refined accuracy curve for "P+P" cases (using SKL & heuristic rules) significantly higher than the HMM accuracy and the SKL-only refinement.
Limitations and Future Work
- "P+P" group W still not perfect: While improved, the performance for the "worse" "P+P" transitions is still not considered good enough for fully automatic labeling in corpus-based TTS applications.
- Coarticulation challenges: Error cases in group W often involve strong coarticulation between neighboring syllables, which is also difficult for human labelers.
- Data limitations: The TRAIN3700 training data is considered not large enough for more comprehensive analysis.
- Future Directions:
- Investigate incorporating text-dependent information like phone durations.
- Consider treating difficult "P+P" transitions as single synthesis units to avoid segmentation errors.
- Use a larger corpus to support more extensive training and analysis.
Summary updated