On using missing-feature theory with cepstral features - approximations to the multivariate integral

Frank Seide, Pei ZhaoInterspeech 2010

TL;DR

This paper tackles the challenge of applying Missing Feature Theory (MFT) to modern cepstral features like MFCCs, which are crucial for accurate speech recognition but computationally intractable with MFT's standard formulation. The authors propose two novel approximations to this integral, making MFT feasible with MFCCs and achieving comparable accuracy to precise integration on reverberated speech.

What Problem Does This Paper Address?

Robust speech recognition systems often need to handle degraded or "missing" parts of the audio signal. Missing Feature Theory (MFT) offers a principled way to deal with this by treating these parts as unknown and marginalizing them out. However, MFT's core operation involves calculating a multivariate integral over a Gaussian distribution, which becomes computationally intractable when applied to popular, linearly transformed features like Mel-Frequency Cepstral Coefficients (MFCCs) or Heteroscedastic Linear Discriminant Analysis (HLDA). This intractability has limited the optimal application of MFT to these state-of-the-art features, forcing researchers to use suboptimal workarounds. This paper aims to bridge this gap by developing approximations that allow MFT to be effectively used with MFCCs.

Methods

The paper proposes two key approximations to the intractable multivariate integral required by MFT:

  • Numeric Integration by Linear Sampling: This method approximates the integral by sampling the feature space at a fixed number of equidistant points within the defined bounds for each missing feature dimension. To make this computationally feasible for MFCCs, the authors introduce a "Cepstrally Decorrelated Filterbank Features" (CDFF) approximation. CDFF interprets MFCCs as a bandpass filtering operation on filterbank energies and simplifies this to a tridiagonal transform, significantly reducing computational complexity.
  • Maximum Approximation: This is a more aggressive approach that approximates the integral by finding the maximum value of the integrand within the integration region. This is motivated by the observation that in many speech recognition contexts, sums of probabilities can be well-approximated by their maximum. This method leads to a tractable solution that can be solved using quadratic programming.

Datasets

The experiments were conducted on the TIMIT corpus, using the standard training/test split. The test set was artificially augmented with reverberation to simulate real-world acoustic conditions.

Key Insights and Ideas

  • MFCCs as Bandpass Filters: The paper offers an interesting interpretation of MFCCs, suggesting that the DCT transform combined with diagonal covariance weighting can be viewed as a bandpass filtering operation on filterbank energies. This is presented as a physiologically plausible interpretation, aligning with how mammalian auditory systems might process sound.
  • Approximations Enable MFT for MFCCs: The core insight is that by approximating the intractable integral, MFT can be effectively applied to MFCCs, a significant step towards more robust speech recognition.
  • Maximum Approximation is Versatile: The "maximum approximation" method is highlighted as particularly promising because it is computationally efficient and, crucially, works directly with original MFCC features without requiring the CDFF approximation. It's also noted to be admissible for other linear transforms and covariance structures.
  • MFT Benefits Even Clean Speech: Interestingly, applying MFT even to clean speech (with artificially created "missing" data based on temporal masking) led to improved accuracy, suggesting that the model can learn to ignore even subtle, imperceptible signal components.

Key Results

  • Numeric Integration (CDFF): Achieved comparable accuracy to precise integration for filterbank features, and a loss reduction of around 33-37% on reverberated speech. The CDFF approximation itself incurred only about a 1% point accuracy loss compared to standard MFCCs.
  • Maximum Approximation: Showed similar accuracy to linear sampling for CDFF features. Crucially, it performed well with original MFCCs, yielding results only about 1% point worse than CDFF with linear sampling. For full features (deltas and accelerations), the maximum approximation still provided significant loss reduction (26-28%), though less than for static features.
  • Overall: The proposed approximations allow MFT to recover a substantial portion of the accuracy lost due to reverberation, demonstrating their practical utility.

Limitations and Future Work

  • CDFF Computational Cost: While significantly reduced, the CDFF approximation for numeric integration is still too computationally expensive for real-time operation, though it serves the purpose of studying MFT.
  • Admissibility for Other Transforms: The paper suggests that the maximum approximation is admissible for other linear transforms (like HLDA, MLLR) and covariance structures, but this effectiveness needs to be experimentally verified.
  • Future Work: The authors hope to enable research into more direct, model-based applications of MFT on state-of-the-art baselines. They also mention that linear sampling could be useful for experimenting with more precise convolution-based channel models for delta features.

Summary updated

BibTeX
@inproceedings{seide2010using,
  author = {Frank Seide and Pei Zhao},
  title = {{On using missing-feature theory with cepstral features - approximations to the multivariate integral}},
  booktitle = {Interspeech},
  year = {2010},
}

Loading related content...

Figure