TL;DR
This paper introduces a novel "add-on" perceptual postfilter for low bit-rate speech coders that enhances speech naturalness without altering the existing encoder. By using Gaussian Mixture Models (GMMs) to estimate optimal postfilter gains based on quantized speech parameters (LSFs and LTP gain), the method significantly improves perceived audio quality.
What Problem Does This Paper Address?
Low bit-rate speech coders, while efficient, often sacrifice speech naturalness, particularly in the high-frequency spectral valleys, leading to audible coding noise. Conventional adaptive postfilters attempt to mitigate this by shaping the coding noise spectrum, but their effectiveness is limited, especially at very low bit rates. The challenge lies in improving the perceived quality of decoded speech without requiring modifications to the encoder or sending additional side information, which would increase the bit rate. This paper aims to bridge this gap by developing a decoder-only postfilter that intelligently estimates and applies gains to enhance the naturalness of the speech.
Methods
The core of this work is a perceptual postfilter designed to operate as an "add-on" at the decoder. The filter's gains are estimated using a Minimum Mean Square Error (MMSE) estimator. This estimator requires knowledge of the joint probability density function (pdf) of the input features and the target postfilter gains. To model this complex joint pdf, the authors employ Gaussian Mixture Models (GMMs).
The input feature vector, denoted as y, consists of the quantized Linear Spectral Frequencies (LSFs) and the long-term prediction (LTP) gain from the decoded speech frame. The target vector, z, represents the perceptual postfilter gains. The MMSE estimator aims to find the expected value of z given y, i.e., ẑ = E{z|y}.
The GMM is trained on a dataset to learn the relationship between the input features and the desired postfilter gains. The conditional pdf of the postfilter gains given the input features is derived from the GMM, allowing the MMSE estimator to compute the optimal gains for each frame. The paper also discusses the use of diagonal covariance matrices for the GMM components to reduce the number of parameters and mitigate overfitting.
Datasets
The paper uses a speech database consisting of 23 speakers (12 female, 11 male). A portion of this database (10 female, 9 male speakers) was used for GMM training, while the remaining speakers (2 female, 2 male) were used for performance evaluation. The experiments were conducted using the ITU-T G.723.1 speech codec at a rate of 5.3 kbps.
Key Insights and Ideas
- Decoder-Only Enhancement: The most significant insight is the development of a perceptual postfilter that functions entirely at the decoder, requiring no encoder modifications. This makes it a practical "add-on" solution for existing speech coding systems.
- GMMs for Perceptual Modeling: Utilizing GMMs to model the joint probability distribution of speech parameters and perceptual filter gains is a novel approach. This allows for a data-driven estimation of optimal postfilter characteristics.
- LSFs and LTP Gain as Key Features: The paper highlights the importance of quantized LSFs and the LTP gain as crucial features for estimating perceptual postfilter gains. These parameters effectively capture spectral envelope and voicing information, which are vital for perceptual quality.
- MMSE for Optimal Gain Estimation: Deriving the postfilter gains from an optimal MMSE estimator ensures that the estimated gains minimize the mean square error between the estimated and true gains, leading to improved perceptual performance.
- Improved Naturalness without Intelligibility Loss: The proposed postfilter not only enhances speech naturalness but also maintains intelligibility, a critical balance for speech coding applications.
Key Results
- Improved Speech Naturalness: Informal listening tests indicated that the proposed perceptual postfilter results in more natural-sounding speech compared to the conventional adaptive postfilter.
- Correlation between Input and Target: Analysis showed a positive correlation between the "input" (LSFs, LTP gain) and "target" (postfilter gains) vectors, suggesting that more GMM mixtures lead to better estimates.
- Performance with GMM Mixtures: Using 128 GMM mixtures yielded the highest mutual information between input and target features, indicating a more accurate modeling of their relationship.
- Spectrogram Analysis: Spectrograms of clean speech, conventionally postfiltered speech, and perceptually postfiltered speech demonstrated that the proposed method recovers some of the high-frequency naturalness lost in low bit-rate coding.
Limitations and Future Work
- Diagonal Covariance Matrices: While reducing parameters, the use of diagonal covariance matrices might limit the model's ability to capture complex correlations within the feature vectors. The paper acknowledges this by stating that a higher degree of freedom (full covariance) could potentially describe finer structures but risks overfitting.
- Informal Listening Tests: The evaluation relied on informal listening tests. More rigorous subjective listening tests (e.g., using MOS scores) would provide more quantitative evidence of the perceptual improvements.
- Future Work: The authors suggest that further exploration of GMMs with full covariance matrices and more extensive subjective evaluations could lead to even greater improvements. Investigating other psychoacoustic models within the GMM framework is also a potential avenue.
Summary updated