TL;DR
This paper introduces a flexible voice simulation system that allows independent control over vocal tract and excitation parameters. This system enables detailed studies into what makes synthetic speech sound unnatural, revealing that the glottal pulse shape is a critical factor for naturalness, and proposes a signal-dependent analysis approach to improve intelligibility, especially for dynamic sounds.
What Problem Does This Paper Address?
Creating synthetic speech that sounds truly natural and is highly intelligible remains a significant challenge. While linear predictive coding (LPC) vocoders have been successful in generating intelligible speech, they often produce a synthetic quality that lacks naturalness. This paper aims to bridge this gap by developing a system that allows researchers to systematically investigate the factors contributing to both the quality and naturalness of synthetic speech. Specifically, it addresses the need to understand how variations in vocal tract characteristics, pitch, and excitation signals (like the glottal pulse) affect the perceived quality and intelligibility of synthesized speech.
Methods
The core of the paper is a flexible analysis-synthesis system built upon the standard LPC vocoder framework. Key innovations include:
- Independent Parameter Control: The system allows for independent modification of vocal tract system parameters and excitation parameters (pitch, pitch contour, and glottal waveshape). This is a departure from some methods that modify these jointly or implicitly.
- Smooth Parameter Interpolation: Pitch and gain contours are smoothed using a 3-point median filter, and parameters are updated every 200 samples (rather than pitch-synchronously), which the authors claim avoids instability issues associated with interpolated parameters and results in smoother synthetic speech.
- Excitation Models: The system incorporates several models for generating the excitation signal, ranging from simple impulse excitations to more complex ones like the Hilbert transform and the first derivative of Fant's glottal pulse model.
- Signal-Dependent Analysis (Proposed): For improved intelligibility, particularly for dynamic sounds like stops, the paper proposes a signal-dependent analysis-synthesis scheme. This approach suggests adapting analysis parameters (like frame size, frame rate, and window shape) based on the characteristics of the input speech segment, mimicking human listeners' dynamic analysis.
Datasets
The paper describes studies conducted using two sentences spoken by a male speaker:
- "WE WERE AWAY A YEAR AGO"
- "SHOULD WE CHASE THOSE COWBOYS"
Speech data was sampled at 10kHz. The paper also lists a comprehensive set of recordings made from various studies (Appendix A), which serve as a benchmark for evaluating different synthesis conditions. No new datasets were introduced.
Key Insights and Ideas
- Glottal Pulse is King for Naturalness: The shape of the glottal pulse approximation significantly impacts the naturalness of synthetic speech, more so than vocal tract system models or pitch period errors.
- Phase Matters for Excitation: The phase function of the excitation signal plays a crucial role in determining speech quality, suggesting that simplified excitation models that don't capture phase information adequately will limit naturalness.
- Signal-Dependent Analysis for Intelligibility: To improve intelligibility, especially for dynamic speech segments, a system needs to adapt its analysis parameters dynamically based on the signal's characteristics. This is a promising direction beyond fixed-parameter LPC vocoders.
- Flexibility Enables Deeper Understanding: The developed flexible analysis-synthesis system is a powerful tool for systematically dissecting the contributions of various speech parameters to quality and intelligibility.
Key Results
- The basic analysis-synthesis system produced highly intelligible speech, though distinctly synthetic.
- Studies using different excitation models (Figure 2) indicated that Model-5 (first derivative of Fant's model) produced the best quality speech, though determining its parameters from speech was a challenge.
- The paper demonstrated the system's capability to perform time expansion/compression, pitch period expansion/compression, and spectral expansion/compression while preserving other characteristics.
- Transformations between different voice types (e.g., male to female) were shown to be feasible by manipulating pitch, vocal tract, and glottal waveshape.
Limitations and Future Work
- Excitation Source Modeling: While significant progress has been made in modeling the vocal tract, a convenient and effective way to model the excitation source (specifically its phase information) remains an open problem. The authors suggest that simply preserving the LP residual or its transform is not the solution.
- Signal-Dependent Analysis Implementation: The paper proposes a signal-dependent analysis-synthesis scheme but provides only a conceptual outline. Further research is needed to fully implement and evaluate its benefits for intelligibility and bandwidth compression.
- Parameter Determination: For advanced excitation models like Model-5, determining the model parameters directly from speech is a crucial next step that was not fully addressed.
Summary updated