Introduction
Developing high-quality Text-to-Speech (TTS) systems traditionally requires extensive datasets from a single speaker. However, real-world applications often demand the ability to adapt a pre-existing "source" model to a new "target" speaker using minimal data—sometimes as few as 100 sentences. While Speaker Adaptation techniques like Maximum Likelihood Linear Regression (MLLR) are standard, they often struggle with significant physical differences between speakers, such as those found in cross-gender adaptation.
A common approach to bridge this gap is Vocal Tract Length Normalization (VTLN), which compensates for anatomical differences in the vocal tract. Traditionally, VTLN relies on a computationally intensive "grid search" to find an optimal warping factor. This paper proposes a more direct, formant-based frequency warping method that leverages the physical landmarks of speech to improve both the efficiency and quality of speaker adaptation in Hidden Semi-Markov Model (HSMM) based TTS.
Methodology
The proposed system integrates formant-based warping into the standard HMM-based TTS adaptation pipeline. The process involves four primary stages: feature extraction, frequency warping, statistical training, and synthesis.
Formant-Based Warping
The core innovation lies in the direct mapping of formants—the resonant frequencies of the vocal tract. The researchers identify the first four formants ($F_1$ to $F_4$) from five specific long vowels (choose, need, force, hard, heard) for both the source and target speakers.
Instead of a single global warping factor, a piecewise linear warping function is constructed. This function maps the source formants to the target formants, using these four pairs as internal knots and anchoring the function at 0 Hz and 8000 Hz. This approach effectively "stretches" or "squashes" the source speaker's frequency spectrum to align with the target's physical vocal tract characteristics.
Adaptation and Synthesis
Following spectral warping, the source speaker’s fundamental frequency ($F_0$) is shifted to match the target's mean and standard deviation. An HSMM is then trained on these warped features. Finally, MLLR is applied to the warped model to further refine the adaptation, ensuring the system captures the target speaker's unique nuances beyond basic vocal tract length.
Experimental Setup
The system was evaluated using a male source speaker (4,000 sentences) and two target speakers: a female speaker (100 sentences) and a male speaker (100 sentences from a noisy internet source). Evaluation was conducted on 10 reserved test sentences per target.
Results and Discussion
The formant-based approach yielded significant improvements in both objective and subjective metrics.
Objective Performance
In male-to-female (M_S to F_T) adaptation, the Log Spectral Distance (LSD) decreased from 9.01 dB to 7.44 dB using warped features alone. This indicates a much closer spectral match to the target speaker. The gains were less pronounced for the male target, likely due to the lower signal-to-noise ratio (20 dB) of the target data, which hindered precise formant extraction.
Subjective Evaluation
Subjective ABX similarity tests demonstrated a strong preference for the warped system. Listeners identified the warped versions as sounding more like the target speaker 79% of the time for the female target and 89% of the time for the male target. These results suggest that formant-based warping captures perceptually important speaker characteristics that standard adaptation methods might miss.
Conclusion
By using formants as physical proxies for vocal tract length, this method provides a computationally efficient and perceptually grounded alternative to traditional VTLN. The approach is particularly effective for cross-gender adaptation and remains robust even with very limited target data. Future work will focus on automating the formant extraction process to eliminate the need for manual correction and exploring the selection of optimal phonetic units for universal warping functions.
Summary updated