A long state vector kalman filter for speech enhancement

Stephen So, Kuldip K. PaliwalInterspeech 2008

TL;DR

This paper introduces a "long state vector Kalman filter" that significantly improves speech enhancement by using longer, overlapping audio frames. This novel approach leads to higher signal-to-noise ratios (SNR) and better perceptual quality (PESQ scores) compared to existing Kalman filter methods, making enhanced speech sound clearer and less "buzzy."

What Problem Does This Paper Address?

Speech enhancement aims to remove unwanted noise from audio recordings, making the speech clearer and more intelligible. This is crucial for applications like mobile phone calls, hearing aids, and voice assistants. Traditional Kalman filter-based methods, while effective, have limitations. One key challenge is that to improve the filter's performance, you'd ideally want to use a longer "state vector" (which essentially captures more history of the speech signal). However, estimating the necessary audio features (autocorrelation coefficients) becomes unreliable at longer lags when using short, non-overlapping audio frames. This paper tackles this by proposing a way to effectively use longer state vectors without sacrificing the reliability of these crucial estimates.

Methods

The core innovation is the "long state vector Kalman filter" that leverages an "analysis-modification-synthesis" (AMS) framework. Instead of processing short, non-overlapping audio frames, this method uses longer, overlapped frames.

Here's a breakdown:

  • Standard Kalman Filter: Uses short frames, estimates audio features (LPCs), and applies a recursive filter. The "scalar" version estimates the current sample, while the "vector" version uses past and future information for a "smoother" estimate.
  • The Challenge: Increasing the state vector length (to capture more speech history) requires estimating autocorrelation coefficients at higher lags. With short frames, these estimates become unreliable, limiting performance gains.
  • The Proposed Solution (Long State Vector Kalman Filter):
    • Overlapped Frames: The audio is divided into longer frames (e.g., 100ms) with significant overlap (e.g., 1/8th of the frame length).
  • Analysis Window: A tapered window (like Hamming) is applied during the analysis phase (LPC estimation) to smooth the frame boundaries.
  • Kalman Filtering: The actual Kalman filtering is performed on the original, non-windowed samples.
  • Frame Overlap Management: Crucially, the filter's state (covariance matrix and state estimate) is preserved across frame overlaps. This allows the filter to maintain its "memory" and effectively process longer segments of speech.
  • Synthesis: The enhanced frames are then combined using a windowed overlap-add (WOLA) technique, similar to standard methods, to reconstruct the continuous enhanced audio signal.

This AMS approach allows for longer state vectors without the unreliability issues associated with estimating high-lag autocorrelation coefficients from short frames.

Datasets

The experiments were conducted using the NOIZEUS corpus. This corpus contains 30 phonetically-balanced sentences sampled at 8 kHz, spoken by six different speakers (three male, three female). The paper notes that the first 1000 samples of each utterance were used as a "noise-only" segment for initial noise estimation.

Key Insights and Ideas

  • Longer is Better (with the right technique): The paper strongly suggests that increasing the state vector length of a Kalman filter can lead to better speech enhancement, but only if the underlying feature estimates remain reliable.
  • The Overlap is Key: The AMS framework, specifically the use of overlapped frames, is the critical enabler for using longer state vectors effectively. It bridges the gap between the need for long-term speech information and the limitations of short-frame analysis.
  • Reduced "Musical Noise": The proposed method not only improves objective metrics (SNR, PESQ) but also subjectively results in less annoying residual noise, often referred to as "musical noise" in spectral-based enhancement techniques. This suggests a more natural-sounding enhancement.
  • Colored Noise Handling: The approach is shown to be effective for both white and colored noise (specifically tested with car noise), with colored noise being modeled using an augmented Kalman state vector.

Key Results

The paper presents quantitative results in Tables 1 and 2, comparing four cases at an initial SNR of 0 dB:

  • (I) Scalar Kalman filter (short frames): Baseline.
  • (II) Short state vector Kalman filter (short frames): Slightly better than scalar.
  • (III) Long state vector Kalman filter (short frames): Performance decreased slightly compared to (II), highlighting the unreliability issue with short frames.
  • (IV) Long state vector Kalman filter (long, overlapped frames - the proposed method): Achieved the highest SNR and PESQ scores for both white and car noise.

Specific examples (from Table 1 for white noise):

  • Case (I): SNR = 4.223 dB, PESQ = 1.739
  • Case (II): SNR = 4.461 dB, PESQ = 1.749
  • Case (III): SNR = 4.083 dB, PESQ = 1.694 (worse than II)
  • Case (IV): SNR = 7.404 dB, PESQ = 1.922 (significantly better)

Spectrograms (Figures 1 and 2) visually confirm the reduction in residual noise for case (IV) compared to others.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, using longer frames and more complex state vectors inherently increases computational load.
  • Parameter Tuning: The paper mentions parameters like predictor order (p), frame length (L), and overlap ratio (R) were varied. Optimizing these for different noise types and speech conditions could be a direction for future work.
  • Subjective Listening Tests: While preliminary listening tests are mentioned, more extensive subjective evaluations could further validate the perceived quality improvements.
  • Real-world Noise: While car noise was used as a colored noise example, testing with a wider variety of real-world noise conditions would be beneficial.

Summary updated

BibTeX
@inproceedings{so2008long,
  author = {Stephen So and Kuldip K. Paliwal},
  title = {{A long state vector kalman filter for speech enhancement}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure