TL;DR
This paper introduces a simplified and more robust framework for representing non-periodic components in speech, crucial for high-quality speech manipulation. By refining how "noise" and "events" in speech are modeled, the authors significantly improve the realism and flexibility of speech synthesis systems like TANDEM-STRAIGHT.
What Problem Does This Paper Address?
Speech is a complex signal, and while periodic components (like voiced phonemes) are relatively well-understood and modeled, the non-periodic aspects (like unvoiced fricatives, plosives, and subtle breathiness) have historically been a weak point in speech synthesis and manipulation systems. Existing methods, particularly in systems like TANDEM-STRAIGHT, struggled to accurately represent these non-periodic elements, leading to artifacts like "wet" timbres or "musical noise" in synthesized speech. This paper tackles the challenge of creating more perceptually relevant and robust representations for these non-periodic excitation sources, aiming to enhance the quality and naturalness of manipulated speech.
Methods
The authors propose a two-part extension to existing speech analysis frameworks:
- Aperiodicity Analysis (Colored Noise Source): Instead of complex, detailed models, they introduce a simplified sigmoid model to represent the colored noise component of speech. This model uses two key parameters: a "boundary frequency" and a "slope parameter," which capture the general spectral tilt of the noise. To estimate these parameters, they employ a pitch-adaptive temporal axis warping technique before applying linear prediction to sub-band signals. This warping helps to normalize for pitch variations, making the residual signal more stable for analysis. The estimation process involves fitting the sigmoid model to the residual data using a weighted least squares method.
- Event Analysis (Pulse Components): To capture perceptually salient, isolated events like plosive onsets, they utilize kurtosis, a statistical measure sensitive to sharp spikes in a signal. Kurtosis is calculated on a high-pass filtered version of the speech signal. Event candidates are identified by looking for local peaks in the kurtosis signal. The precise event location is then refined by calculating the centroid of the fourth power of the windowed filtered signal. This approach is designed to be computationally efficient, running faster than real-time even in MATLAB.
Datasets
The primary dataset used for evaluation consists of utterances from a speech database prepared for fundamental frequency (FO) estimation algorithms. This database includes recordings from 14 male and 14 female speakers, each reading 30 Japanese sentences in an anechoic chamber. The recordings were made at a 48 kHz sampling rate with 16-bit resolution. The authors also mention using simulations to test the consistency of their extraction procedure.
Key Insights and Ideas
- Simplification is Key: The paper demonstrates that simplifying the representation of non-periodic components, moving away from overly detailed models, can lead to more robust and perceptually pleasing results.
- Kurtosis for Events: Using kurtosis as a detector for sharp, transient events in speech is an effective and computationally efficient approach. It highlights that while kurtosis peaks might not perfectly align with acoustic events, the derivative of kurtosis is a reliable indicator.
- Temporal Warping for Stability: Preprocessing speech signals with pitch-adaptive temporal axis warping significantly improves the stability of residual analysis, making it easier to estimate parameters for the colored noise model.
- Perceptual Importance of Aperiodicity: The work underscores the perceptual significance of accurately modeling the temporal distribution of aperiodic components, suggesting it's a crucial factor for naturalness.
Key Results
- Quality Improvement: Informal listening tests indicated that the proposed aperiodicity subsystem "almost completely removed" degradations and "significantly improved" perceived sound quality.
- Robustness: Consistency tests, where synthesized speech was re-analyzed, showed that the proposed procedure effectively recovers the parameters used in synthesis, with regression lines close to identity mapping.
- Computational Efficiency: The event detection method is noted to be computationally efficient, running faster than real-time even with MATLAB implementation.
- Sigmoid Fitting: Figure 2 shows successful sigmoid fitting to represent the colored noise component, with the model closely matching the target values derived from the residual signals.
Limitations and Future Work
- Listener Variability: While many listeners found the synthesized sound clearer with event detection, some reported no difference, suggesting that the perceptual impact of event representation can vary among individuals.
- Formal Subjective Evaluation: The authors mention that formal subjective evaluation tests are currently undergoing, implying that the current perceptual findings are preliminary.
- Next Target: Temporal Distribution: The paper identifies the "temporal distribution of aperiodic component" as the next area for investigation, suggesting that further refinements in representing how brief noise bursts vary over time are needed for even greater perceptual clarity.
Summary updated