Voice source waveform analysis and synthesis using principal component analysis and Gaussian mixture modelling

Jon Gudnason, Mark R. P. Thomas, Patrick A. Naylor, Dan P. W. EllisInterspeech 2009

TL;DR

This paper introduces a novel data-driven approach to model the voice source waveform, a crucial component in speech production. By combining Principal Component Analysis (PCA) and Gaussian Mixture Modeling (GMM), the authors create a flexible and accurate representation that can be used for various speech applications like synthesis, coding, and analysis.

What Problem Does This Paper Address?

Speech production is often modeled as a source-filter system, where the vocal tract acts as a filter and the glottis produces the sound source. While modeling the vocal tract is relatively straightforward using techniques like linear prediction, accurately capturing the complex and varied shape of the voice source waveform (specifically, the glottal flow derivative) has been a long-standing challenge. Existing models often rely on predefined mathematical functions or physically motivated assumptions, which can be difficult to estimate and may not fully represent the natural variability of voice sources across different speakers and phonations. This paper aims to address this by developing a data-driven method that learns voice source characteristics directly from speech data.

Methods

The core of this paper lies in its two-pronged approach:

  • Principal Component Analysis (PCA): The authors first extract the voice source waveform by inverse-filtering speech signals. These normalized waveforms are then subjected to PCA. PCA decomposes the waveforms into a set of principal components (eigenvectors) and their corresponding weights (eigenvalues). This allows for a compact representation of the voice source, where complex waveforms can be approximated by a linear combination of a few principal components. This is beneficial for both reducing the dimensionality of the data for subsequent modeling and for efficient coding.
  • Gaussian Mixture Modeling (GMM): The PCA coefficients (spectra) are then modeled using GMM. GMM represents the distribution of these coefficients as a sum of multiple Gaussian distributions, each with its own mean and covariance. The means of these Gaussian components, when transformed back into the waveform domain, represent "prototype" voice source waveforms. These prototypes capture the typical shapes and variations observed in the training data.

The analysis/synthesis process involves decomposing an unknown voice source into its PCA coefficients, finding the closest GMM prototype(s), and then reconstructing the waveform.

Datasets

The primary dataset used for training and analysis is the APLAWD database. This database contains recordings of five male and five female talkers uttering ten repetitions of five short sentences. For this study, sentence 2, "Why are you early you owl?", was specifically chosen as it consists entirely of voiced speech, providing approximately 22,000 glottal cycles for model training. The speech was recorded at 20 kHz and includes contemporaneous electroglottograph (EGG) recordings, which were used with the SIGMA algorithm to obtain glottal closure instants (GCIs) necessary for the analysis.

Key Insights and Ideas

  • Data-Driven Flexibility: The PCA+GMM approach moves away from rigid, predefined models towards a data-driven representation that can capture the natural variability of voice source waveforms.
  • Prototype Discovery: GMM effectively identifies "prototype" voice source waveforms that represent common shapes and characteristics, offering insights into the underlying structure of vocalization.
  • Dimensionality Reduction for Efficiency: PCA significantly reduces the dimensionality of the voice source representation, making it more amenable to efficient modeling and coding.
  • Trade-off in Synthesis Quality: The paper demonstrates a clear trade-off between the number of PCA components used for synthesis and the resulting signal-to-noise ratio (SNR), allowing for flexible application in low-bitrate or high-quality scenarios.
  • Meaningful Prototypes: The discovered prototypes exhibit interpretable features, such as variations in the "basic shape parameter" and the "closed phase," which are relevant to phonetics and voice production research.

Key Results

The paper demonstrates that using a relatively small number of principal components (e.g., K'=64) can capture over 95% of the variance in the voice source waveforms. The GMM model was optimized with M=16 mixture components, balancing model complexity with performance.

Qualitatively, the paper shows that:

  • Reconstructing voice sources using a low number of PCA components (e.g., K'=4) leads to significant loss of detail.
  • Increasing the number of PCA components (e.g., K'=128) allows for the capture of finer details in the waveform.
  • Synthesizing speech using a single GMM prototype results in a lower SNR (1.43 dB) compared to using PCA-based synthesis (up to 10.2 dB for K'=128). This highlights that while prototypes are useful for low-bitrate coding, a more detailed PCA-based reconstruction is needed for higher quality.

Limitations and Future Work

  • Low-Bitrate Synthesis Quality: The paper acknowledges that synthesizing the voice source from a single GMM prototype results in a significantly lower SNR compared to using PCA-based reconstruction. This suggests that for high-fidelity synthesis, a more complex representation than a single prototype is necessary.
  • Future Work: The authors suggest that this technique is well-suited for applications in speech analysis, synthesis, coding, enhancement, and recognition. The ability to decompose and represent voice sources in a flexible manner opens doors for further research in these areas, potentially leading to more natural-sounding synthetic speech or more robust speech recognition systems. The paper also hints at exploring interdependencies between the discovered prototypes.

Summary updated

BibTeX
@inproceedings{gudnason2009voice,
  author = {Jon Gudnason and Mark R. P. Thomas and Patrick A. Naylor and Dan P. W. Ellis},
  title = {{Voice source waveform analysis and synthesis using principal component analysis and Gaussian mixture modelling}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure