TL;DR
This paper explores using Quantized Hidden Markov Models (QHMMs) to significantly reduce the memory footprint of HMM-based Text-to-Speech (TTS) systems. By quantizing model parameters, they achieve substantial compression with minimal perceived loss in speech quality, making TTS more feasible for memory-constrained devices.
What Problem Does This Paper Address?
Text-to-Speech (TTS) systems are becoming increasingly ubiquitous, but their deployment on resource-limited devices like mobile phones and game consoles is often hindered by their substantial memory requirements. Traditional HMM-based TTS systems, while offering advantages like smooth speech and adaptability, store complex statistical distributions (means and variances) as high-precision floating-point numbers, leading to large model sizes. This paper aims to address this by investigating methods to compress these HMM models without sacrificing too much speech quality.
Methods
The core of this paper is the application of Quantized Hidden Markov Models (QHMMs) to TTS. QHMMs, previously successful in Automatic Speech Recognition (ASR), work by quantizing the mean and variance parameters of the HMMs. Instead of storing full floating-point numbers, these parameters are represented by discrete values from a learned codebook.
The authors employ a Lloyd-Max quantizer to iteratively optimize scalar codebooks for both means and variances. This process involves:
- Initialization: Uniformly segmenting the data.
- Iterative Cluster Assignment: Assigning data points to the closest codebook entry.
- Update Clusters: Recalculating the representative value for each cluster.
- Quantization Error Check: Stopping when the error falls below a threshold.
They explore various quantization configurations, defined by the number of bits used for encoding means and variances, and evaluate their impact on compression ratio and spectral distortion.
Datasets
The experiments were conducted using a four-hour speech database recorded by a US English male speaker. The acoustic observation vectors comprised mel-cepstral coefficients, mixed excitation parameters, and logarithmic domain F0 values. The paper does not introduce new benchmark datasets but rather uses a standard dataset for evaluating their proposed method.
Key Insights and Ideas
- Quantization is a viable compression strategy for TTS: The paper demonstrates that quantizing HMM parameters, a technique proven in ASR, can effectively reduce TTS model sizes.
- Trade-off between compression and quality is crucial: There's an inherent balance to be struck. More aggressive quantization leads to higher compression but can introduce spectral distortion.
- Subjective quality is surprisingly resilient: Despite measurable spectral distortion, listeners often couldn't distinguish between compressed and original speech in listening tests, suggesting that the perceptual impact of quantization might be less severe than spectral metrics suggest.
- Configuration matters: The choice of how many bits are allocated to means versus variances significantly impacts both compression and distortion.
- 16-bit quantization offers a better balance: While 8-bit configurations achieve higher compression ratios, 16-bit systems generally exhibit lower spectral distortion, making them a more attractive option when slightly more memory is available.
Key Results
- Significant Compression:
- 8-bit configurations achieved compression ratios as low as 0.148 (e.g., 8_4_4 configuration), representing an 85.2% size reduction for the original representation.
- 16-bit configurations achieved compression ratios as low as 0.359 (e.g., 16_8_8 configuration), representing a 64.1% size reduction.
- Spectral Distortion:
- The average spectral distortion for 16-bit configurations was 0.472, while for 8-bit configurations it was 2.156.
- The best 16-bit configuration (16_10_6) had a spectral distortion of 0.15, while the best 8-bit configuration (8_7_1) had a distortion of 0.62.
- Subjective Evaluation:
- In a listening test with 25 listeners, four compression configurations (two 8-bit, two 16-bit) were compared against the lossless system.
- The results showed no significant preference for the lossless system, indicating that the quality degradation was not perceptually obvious.
Limitations and Future Work
- Limitations:
- The paper acknowledges that some spectral distortion is reported, indicating a degradation, even if not always perceptually significant.
- One configuration (16_12_4) resulted in no compression because the number of codebooks exceeded the available densities, highlighting the importance of careful configuration selection.
- Future Work:
- Combine QHMMs with variance tying: The authors suggest combining their QHMM approach with recently reported variance tying techniques for further compression improvements.
- Quantization during training: Investigating quantization during the training stage could lead to reduced training times.
- More rigorous mathematical frameworks: Exploring information-theoretic distortion measures for clustering, which have outperformed traditional methods in ASR, could lead to better quantization strategies.
Summary updated