TL;DR
This paper introduces a novel voice morphing technique that achieves more natural-sounding transitions by independently interpolating vocal tract characteristics and vocal cord characteristics. By using AR-HMM analysis to separate these features and interpolating in the vocal tract area function domain, the method ensures smoother formant transitions, leading to perceptually improved morphed speech.
What Problem Does This Paper Address?
Voice morphing aims to smoothly transform one speaker's voice into another's while preserving the original phonological content. Traditional methods, often relying on cepstral interpolation, struggle with maintaining formant continuity. This is because interpolating power spectra (or cepstra) doesn't guarantee continuous formant transitions; instead, it can lead to weak, mixed formants, degrading the perceived voice quality. The paper addresses this by proposing a method that explicitly models and interpolates the vocal tract's resonant characteristics (formants) and the vocal cord's excitation signal separately, aiming for better phonological continuity, especially in extrapolated regions.
Methods
The core of the proposed method lies in its feature extraction and interpolation strategy:
- AR-HMM Analysis: This technique is used to decompose speech signals into two components: vocal tract resonance characteristics (modeled by an AR model) and vocal cord wave characteristics (modeled by an HMM). This separation is crucial for independent processing.
- Vocal Tract Area Function (VTF) Interpolation: Instead of interpolating cepstra, the paper uses the estimated vocal tract area functions. These functions are converted to their logarithmic form to ensure positivity and stability. The authors argue that interpolating in this domain leads to more continuous formant transitions.
- Vocal Cord Wave Interpolation: The vocal cord wave is transformed into cepstral coefficients and then interpolated using a statistical mapping approach (Gaussian Mixture Models - GMMs).
- Independent Morphing: The vocal tract features (log VTF) and vocal cord features (cepstra) are morphed independently using linear interpolation for the VTF and GMM-based conversion for the vocal cord wave. The morphing rate ($\lambda$) controls the blend between the source and target speaker.
- Synthesis: The morphed features are then used to synthesize speech using the STRAIGHT synthesis-by-analysis software.
Datasets
- Training and Evaluation Data: The study used a dataset of 50 Japanese sentences, each uttered by two male and two female speakers.
- Training: 45 sentences were used to train the conversion functions (GMMs).
- Synthesis/Evaluation: 5 sentences were used for synthesizing morphed speech and evaluating conversion quality.
- No new benchmarks were introduced. The evaluation focused on comparing the proposed method against conventional LPC analysis and AR-HMM based cepstral interpolation.
Key Insights and Ideas
- Formant Continuity is Key: The paper strongly emphasizes that smooth formant transitions are critical for natural-sounding voice morphing, and cepstral interpolation fails to achieve this.
- Vocal Tract vs. Vocal Cord Separation: The AR-HMM analysis is a powerful tool for decoupling the spectral envelope (vocal tract) from the excitation signal (vocal cord), enabling independent and more targeted morphing.
- Log Vocal Tract Area Function Domain: Interpolating in the log VTF domain is shown to produce significantly smoother and more continuous formant movements compared to cepstral interpolation. This is a central contribution of the paper.
- Perceptual Relevance: The authors conducted listening tests that confirmed the perceptual difference between their method and cepstral interpolation, especially at higher extrapolation rates.
Key Results
- Spectral Distortion: Table 1 shows that the proposed method (specifically option (c) using log VTF and vocal cord cepstra) achieved lower spectral distortion (a measure of conversion quality) compared to conventional LPC analysis (a) and AR-HMM based cepstral interpolation (b). The difference was particularly noticeable when comparing the target speaker's speech (100% morphing rate) to the original source.
- Formant Transition Observation: Figure 3 visually demonstrates that interpolation in the log VTF domain results in continuous formant shifts, while cepstral interpolation leads to formants that appear to "mix" rather than transition smoothly. Figure 5 and 6 further illustrate these differences for both interpolation and extrapolation scenarios.
- Perceptual Confirmation: Listening tests confirmed that the differences in voice quality between the proposed method and cepstral interpolation are perceptually recognizable, especially for extrapolation.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while the proposed method improves formant continuity, the overall voice quality is still dependent on the quality of the AR-HMM analysis and the GMM-based conversion.
- The study primarily focused on spectral distortion and perceptual listening tests for a limited set of conditions.
- Future Work:
- The authors suggest exploring more sophisticated interpolation methods beyond linear interpolation for VTFs and GMMs for vocal cord waves.
- Further investigation into the perceptual impact of different morphing rates and extrapolation levels is warranted.
- Extending the method to handle a wider range of acoustic conditions and speaker characteristics could be beneficial.
Summary updated