Invariant integration features combined with speaker-adaptation methods

Florian Müller, Alfred MertinsInterspeech 2010

TL;DR

This paper explores a novel feature extraction method called Invariant Integration Features (IIFs) for speech recognition. They demonstrate that IIFs, when combined with standard speaker adaptation techniques like VTLN and MLLR, significantly outperform traditional Mel-Frequency Cepstral Coefficients (MFCCs), especially in challenging mismatched training and testing conditions.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems often struggle with variations in how individuals speak, particularly differences in vocal tract length (VTL). Traditional methods like Vocal Tract Length Normalization (VTLN) aim to mitigate these variations by adjusting spectral features. However, the effectiveness of newer feature extraction techniques, like Invariant Integration Features (IIFs) – which are designed to be inherently speaker-independent – when used with these established adaptation methods, remained an open question. This paper investigates whether IIFs can maintain their advantages when integrated into more sophisticated ASR systems that already employ speaker normalization and adaptation.

Methods

The core of this work lies in combining Invariant Integration Features (IIFs) with speaker adaptation techniques.

  • Invariant Integration Features (IIFs): These features are derived from time-frequency (TF) representations of speech. The key idea is to leverage mathematical invariants, inspired by image analysis, to create features that are less sensitive to speaker-specific variations like VTL. The paper uses a contextual definition of IIFs, which involves averaging spectral values over specific frequency ranges (subbands) and time offsets.
  • Vocal Tract Length Normalization (VTLN): This is a standard speaker normalization technique. The paper applies a translational VTLN, which shifts spectral values along the frequency axis. This is done by estimating a warping factor that best aligns the speech signal to a reference.
  • Mel-Frequency Cepstral Coefficients (MFCCs): These are the standard, widely used features in ASR, serving as a baseline for comparison.
  • Maximum Likelihood Linear Regression (MLLR): This is a common speaker adaptation technique that adapts the acoustic models themselves to a specific speaker.
  • Feature Selection: For IIFs, a feature selection process based on a linear classifier was employed to choose the most effective features from a large pool.
  • Interpolation: To reduce computational complexity, the paper also investigates interpolating spectral values from a lower-resolution TF representation (24 subbands) to a higher resolution (110 subbands) before computing IIFs.

Datasets

  • TIMIT Corpus: This is the primary dataset used for experiments. It's a widely recognized corpus for phonetic recognition research, containing utterances from 630 male and female adult speakers.
  • Training Set: 3696 sentences (excluding SA-sentences).
  • Test Set: 1344 utterances.
  • Phonetic Labels: Collapsed from 61 to 48 labels, and further to 39 labels for evaluation.

Key Insights and Ideas

  • IIFs Benefit from Adaptation: Contrary to a potential concern that speaker adaptation might negate the inherent invariance of IIFs, the study shows that IIFs actually benefit from VTLN and MLLR, leading to further performance improvements.
  • IIFs Outperform MFCCs Even with Adaptation: The core finding is that IIFs, even when combined with sophisticated adaptation methods, consistently outperform MFCCs in both matching (training and test speakers are from the same group) and mismatching (training and test speakers are from different groups, simulating VTL variations) scenarios.
  • Robustness in Mismatch: The performance gains of IIFs over MFCCs are particularly pronounced in the mismatching scenario, highlighting their effectiveness in handling speaker variability.
  • Computational Trade-offs: Using a lower number of subbands (24) for the TF representation, even with interpolation, yields accuracies only slightly lower than using a full 110 subbands, suggesting a viable path for reducing computational load without significant performance loss.
  • IIFs Without Adaptation are Competitive: Remarkably, IIFs without any adaptation already perform as well as MFCCs with VTLN and MLLR in the matching scenario.

Key Results

  • Matching Scenario: IIFs with VTLN and MLLR achieved an accuracy of 77.16%, compared to 75.36% for MFCCs with the same adaptation.
  • Mismatching Scenario: IIFs with VTLN and MLLR achieved an accuracy of 74.32%, a significant improvement over MFCCs with VTLN and MLLR, which achieved 71.84%.
  • IIFs vs. MFCCs (No Adaptation): IIFs (75.20% matching, 60.06% mismatching) already outperformed MFCCs (72.16% matching, 54.42% mismatching) without any adaptation.
  • Interpolation Impact: Using 24 subbands with interpolation resulted in slightly lower accuracies than using 110 subbands, but the highest accuracy for the mismatching case was achieved with 24 subbands without interpolation.

Limitations and Future Work

  • Feature Selection: The feature selection for IIFs was performed on data from a matching scenario. The authors suggest that future work could explore feature selection on mismatching data.
  • Order of IIFs: Only IIFs of order one were used in this study. Future work could investigate higher-order IIFs.
  • Physiologically Motivated Principles: The authors propose exploring physiologically motivated principles for feature extraction to potentially achieve even more robust features.
  • Software Availability: A software tool for computing the IIFs used in this work is available for download.

Summary updated

BibTeX
@inproceedings{muller2010invariant,
  author = {Florian Müller and Alfred Mertins},
  title = {{Invariant integration features combined with speaker-adaptation methods}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure