Feature transformations and combinations for improving ASR performance

Panu Somervuo, Barry Y. Chen, Qifeng ZhuInterspeech 2003 · DOI

TL;DR

This paper explores how to improve automatic speech recognition (ASR) by transforming acoustic features. They found that while individual feature transformations didn't beat the baseline, combining them with the original features significantly reduced word error rates, especially when using non-linear transformations like Multi-Layer Perceptrons (MLPs).

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems rely heavily on how they represent the raw audio signal as features. The goal is to extract features that are robust to variations in speech (like speaker, accent, noise) while preserving information crucial for distinguishing between different sounds and words. Traditional features like MFCCs and PLP cepstra have been successful, but researchers are constantly looking for ways to enhance them. This paper investigates whether applying linear and non-linear transformations to these existing features, or combining them with other transformed features, can lead to better recognition performance. The challenge lies in finding transformations that capture more discriminative information without introducing too much noise or complexity.

Methods

The authors experimented with several feature transformation techniques:

  • Unsupervised Linear Transformations:
    • Principal Component Analysis (PCA): Aims to reduce dimensionality by finding the directions of maximum variance in the data, effectively decorrelating features.
  • Independent Component Analysis (ICA): Goes beyond PCA to find statistically independent components, aiming to remove higher-order dependencies.
  • Discriminative Transformations:
    • Linear Discriminant Analysis (LDA): Seeks to find a feature space that maximizes the separability between different classes (e.g., phonemes).
  • Non-linear Discriminant Analysis (using Multi-Layer Perceptrons - MLP): Leverages neural networks to learn complex, non-linear mappings that can better separate classes.

They also explored two methods for combining feature streams:

  • Feature Vector Concatenation: Simply appending the transformed feature vectors to the original baseline feature vectors.
  • ROVER (Recognition Output Voting Error Reduction): Training separate recognizers for each feature stream and then combining their n-best lists of recognition hypotheses.

Datasets

  • Training Data: A subset of the HUB5 training data (over 60 hours of speech from 6273 speakers), consisting of a mix of conversational telephone speech and read speech from the Macrophone Corpus.
  • Testing Data: The OGI Numbers corpus, which contains strings of continuous numbers (e.g., "five hundred fifty eight"). This corpus has 32 words and was used for a specific number recognition task. The test set comprised 1227 utterances (0.6 hours, 4670 words).

Key Insights and Ideas

  • No Single Transformation Wins Alone: Surprisingly, none of the individual feature transformations (PCA, ICA, LDA, MLP) managed to outperform the baseline feature set (PLP cepstrum + energy + deltas) when used by themselves. This suggests that while transformations can capture different aspects of the signal, they might not be sufficient on their own for optimal recognition.
  • Combinations are Key: The real gains were seen when combining the baseline features with the transformed features. This indicates that the baseline features capture some essential information, and the transformations add complementary discriminative power.
  • Non-linear Transformations Shine in Combinations: MLP-based features consistently provided the best improvements when combined with the baseline, both through concatenation and ROVER. This highlights the power of non-linear models in learning complex speech characteristics.
  • Multi-frame Features Offer More Context: Computing transformations over multi-frame windows (capturing temporal context) generally yielded better results than single-frame transformations, especially for PCA and LDA.
  • ROVER vs. Concatenation: Both combination methods showed improvements, but ROVER offered a more robust way to leverage multiple feature streams, particularly when dealing with linear transformations.

Key Results

  • The word error rate (WER) on the number recognition task was reduced from a baseline of 4.1% to 3.1% when combining the baseline PLP cepstrum with MLP features using concatenation.
  • Using ROVER, the WER was reduced to 3.1% with the combination of PLP cepstrum and MLP features, and 3.2% with PLP cepstrum and LDA features.
  • For single-frame transforms, the baseline achieved 4.1% WER, while PCA (5.1%), ICA (6.7%), and LDA (4.9%) did not improve.
  • For multi-frame transforms, the baseline achieved 4.1% WER, while PCA (4.5%), ICA (5.0%), and LDA (4.5%) showed marginal improvements, with MLP (4.1%) performing similarly to the baseline.

Limitations and Future Work

  • No Single Transform Outperforms Baseline: The authors acknowledge that none of the tested feature transformations could outperform the baseline system when used alone. This is attributed to differences in the recognition systems and tasks compared to previous work.
  • Limited Exploration of Linear Transforms: While MLP features showed significant promise, the linear feature transformations (PCA, ICA, LDA) did not yield substantial improvements when combined with the baseline, even with ROVER. Further research could explore more advanced linear techniques or different ways to integrate them.
  • Tuning of Combination Weights: The weights used in the ROVER combination were not extensively tuned, suggesting potential for further gains with optimized weighting schemes.
  • Future Work: The paper implicitly suggests exploring more sophisticated non-linear transformations and investigating optimal ways to combine multiple feature streams, potentially beyond just two. The success of MLP features also points towards deeper exploration of neural network-based feature extraction.

Summary updated

BibTeX
@inproceedings{somervuo2003feature,
  author = {Panu Somervuo and Barry Y. Chen and Qifeng Zhu},
  title = {{Feature transformations and combinations for improving ASR performance}},
  booktitle = {Interspeech},
  year = {2003},
  doi = {10.21437/eurospeech.2003-174},
}
Figure