TL;DR
This paper introduces a novel "Sub-band Basis Spectrum Model" (SBM) for representing speech log-spectra and phase. By leveraging sparse coding and a perceptually motivated frequency scale, the SBM achieves a more compact and interpretable representation than traditional methods, leading to high-quality speech synthesis and potential for voice adaptation and conversion.
What Problem Does This Paper Address?
Traditional speech spectrum parameters like LPC, MFCC, and mel-cepstrum, while widely used, have limitations for modern speech processing tasks. They often lack direct interpretability in the frequency domain, making frequency-based manipulations difficult. Furthermore, parameters from methods like STRAIGHT or HNM can be inconsistent in dimensionality or difficult to use for statistical modeling. The authors aim to develop a spectrum representation that is both high-quality for synthesis and suitable for statistical processing, addressing issues like dimensionality, frequency domain manipulability, and perceptual relevance.
Methods
The core of the paper is the proposed Sub-band Basis Spectrum Model (SBM). The authors first apply sparse coding to pitch-synchronously analyzed log-spectra of voiced speech segments. Sparse coding, a technique for finding efficient representations of data using a minimal set of basis vectors, is adapted here by incorporating a centroid regularization term. This process yields basis vectors that exhibit "sub-band features" with varying bandwidths across frequencies, mimicking perceptual properties.
The SBM then represents the log-spectrum as a linear combination of these learned sub-band basis vectors. Crucially, the frequency scale for these basis vectors is designed to be mel-scale for lower frequencies (mimicking human hearing) and equally spaced for higher frequencies. This creates basis vectors with a "1-cycle sinusoidal shape" (similar to Hanning windows). The coefficients for this linear combination are then calculated using a non-negative least squares method to ensure a good fit to the actual spectrum.
The same approach is extended to model the phase spectrum. By fitting the sub-band basis vectors to the phase spectrum, the SBM can also generate phase parameters. Pre-processing steps are applied to the phase spectrum to stabilize unwrapping and improve parameter extraction.
Datasets
The paper mentions using "50 sentences of a female speaker" for training the sparse coding basis. For experimental evaluation, "five sentences of one Japanese female speaker" were used. The specific datasets are not named or detailed beyond these descriptions.
Key Insights and Ideas
- Perceptually Motivated Basis Vectors: The learned sub-band basis vectors naturally exhibit characteristics that align with human auditory perception, with narrower bands at lower frequencies and wider bands at higher frequencies. This is a significant departure from more uniform basis functions.
- Unified Log-Spectrum and Phase Modeling: The SBM provides a consistent framework for parameterizing both the magnitude (log-spectrum) and phase of speech, which is crucial for high-fidelity speech synthesis.
- Frequency Domain Manipulability: Because the SBM is inherently defined in the frequency domain, it opens doors for easier frequency-based voice adaptation, interpolation, and conversion, unlike parameters that are not directly frequency-localized.
- Compact and Interpretable Representation: The SBM coefficients are proposed to be more interpretable and potentially lower-dimensional than raw spectral representations, making them suitable for statistical modeling.
Key Results
The paper reports positive experimental results from subjective listening tests (MOS scores).
- Analysis-synthesis speech using the proposed SBM (specifically SBM+SBPH) was found to be close to the original speech.
- The SBM+SBPH method achieved MOS scores that were significantly better than mel-cepstrum based synthesis (MCEP+SBPH).
- A substantial portion (66.6%) of sentences synthesized using SBM+SBPH were indistinguishable from the original waveform (MOS score of "5").
- The proposed parameter was deemed to have sufficient precision for unit selection and fusion-based speech synthesizers, with no significant difference between TTS-ORIG (using original waveform) and TTS-SBM+SBPH.
Limitations and Future Work
- Limitations: The paper acknowledges that the analysis-synthesis speech of sparse coding itself was not formally evaluated in the main experiment, though informal listening suggested it was close to the original. The specific datasets used for training and evaluation are not detailed.
- Future Work: The authors propose several future directions:
- Applying the SBM to voice conversion using GMM-based frequency warping and filtering.
- Integrating the SBM parameter into HMM-based speech synthesis.
Summary updated