TL;DR
This paper evaluates a novel time-varying complex speech analysis method for more robust fundamental frequency (f0) estimation, particularly in noisy conditions. The findings suggest that this advanced technique excels in accurately estimating f0 for stationary voiced speech segments, outperforming previous methods in specific scenarios.
What Problem Does This Paper Address?
Accurately estimating the fundamental frequency (f0) of speech is crucial for many audio processing tasks, including speech coding, recognition, and enhancement. However, real-world speech is often corrupted by noise, which significantly degrades the performance of traditional f0 estimation algorithms. While previous work proposed robust methods using complex-valued speech analysis, these were primarily evaluated using time-invariant models. This paper addresses the gap by evaluating the performance of a time-varying complex speech analysis approach, investigating whether its dynamic nature can lead to more accurate f0 estimation, especially across different degrees of speech stationarity.
Methods
The core of the proposed method lies in a Time-Varying Complex Autoregressive (TV-CAR) model. Unlike conventional Autoregressive (AR) models that assume fixed coefficients, the TV-CAR model allows the AR coefficients to change over time. This is achieved by representing the time-varying coefficients as a linear combination of complex basis functions.
The paper builds upon two main algorithmic approaches for estimating these time-varying coefficients:
- MMSE-based algorithm: This approach minimizes the Mean Squared Error (MSE) of the equation error to find optimal complex AR coefficients.
- ELS-based algorithm (Extended Least Square): This method further whitens the MMSE equation error using an AR filter, aiming for more robust spectral estimation, which is hypothesized to lead to more accurate f0 estimation.
For f0 estimation itself, the paper utilizes a criterion proposed by Shimamura, which combines the Autocorrelation Function (AUTOC) and the Average Magnitude Difference Function (AMDF). This combined criterion is applied to the complex AR residual signal obtained from the TV-CAR analysis. The real part of the AUTOC is used for complex-valued signals.
Datasets
The experiments were conducted using the Keele Pitch Database, which contains speech signals from both male and female speakers. The speech data was filtered using an IRS (Intermediate Reference System) filter to simulate conditions relevant to speech coding. Additive noise, specifically white Gauss noise and pink noise, was introduced at various levels (30 dB to -5 dB) to evaluate robustness.
Key Insights and Ideas
- Time-varying analysis shows promise for stationary speech: The most significant finding is that the time-varying complex speech analysis, particularly when combined with the ELS-based estimation, performs best for strongly stationary voiced speech frames (Mode 3). This suggests that the model's ability to adapt to changing speech characteristics is beneficial when those changes are minimal.
- ELS-based methods are generally more robust: Across various conditions, ELS-based algorithms (both time-invariant and time-varying) consistently outperformed MMSE-based algorithms in terms of Gross Pitch Error (GPE).
- Stationarity matters: The performance of the time-varying approach is highly dependent on the stationarity of the voiced segment. It performs well for stationary segments but not as well for non-stationary ones.
- Complex analysis offers advantages: The use of complex-valued speech analysis, which leverages the analytic signal to focus on positive frequencies, contributes to more accurate spectral estimation, especially in lower frequency regions.
Key Results
The paper presents results in terms of Gross Pitch Error (GPE) across different noise levels and for different degrees of voiced stationarity (Modes 1, 2, and 3).
- Overall (All Modes): For all voiced frames, the time-varying analysis did not consistently outperform other methods. The ELS-based time-invariant complex speech analysis (CLPC_E) showed strong performance.
- Mode 3 (Strongly Stationary): The time-varying complex analysis (TVC_E) demonstrated superior performance compared to time-invariant methods.
- Mode 2 (Ordinary Voiced): The ELS-based time-invariant complex speech analysis (CLPC_E) performed slightly better.
- Mode 1 (Weak Voiced/Non-Stationary): The time-varying complex analysis (TVC_E) showed better performance than time-invariant methods, though the paper notes that MMSE-based time-varying analysis also performed better for non-stationary segments, albeit with lower overall accuracy.
Limitations and Future Work
- Performance on non-stationary segments: The time-varying ELS-based complex speech analysis does not perform optimally for non-stationary voiced segments. While the MMSE-based time-varying analysis shows some improvement in these cases, its overall accuracy is not as high.
- Phoneme-level evaluation: The authors state that evaluating the performance for each individual phoneme is a subject for future study.
- Application in speech coding: The paper suggests that the proposed frame-based f0 estimation could be integrated into open-loop adaptive codebooks in CELP speech coding to improve noisy speech quality.
This research lays important groundwork for developing more robust f0 estimation techniques by exploring the benefits of time-varying complex speech analysis, particularly highlighting its strengths in stationary voiced speech.
Summary updated