Low-dimensional space transforms of posteriors in speech recognition

Jan Zelinka, Jan Trmal, Luděk MüllerInterspeech 2010

TL;DR

This paper introduces novel methods for transforming high-dimensional posterior probability estimates from speech recognition models into much lower-dimensional representations. These transforms aim to retain crucial discriminative information, leading to improved speech recognition accuracy when combined with existing features, particularly in challenging acoustic conditions.

What Problem Does This Paper Address?

In modern speech recognition systems, Artificial Neural Networks (ANNs) are often used to estimate posterior probabilities of phonetic units given acoustic observations. However, these posterior probability vectors are typically very high-dimensional, making them impractical for direct use as features. Standard dimensionality reduction techniques like PCA or LDA often struggle to preserve the discriminative power of these posteriors when reducing them to very low dimensions (1D or 2D). This paper addresses the challenge of creating effective low-dimensional representations of these posteriors that retain their discriminative capabilities, thereby improving overall speech recognition performance.

Methods

The authors propose three novel linear transforms for reducing the dimensionality of posterior probability vectors. These transforms are designed to be applied after the posteriors have been computed. The core idea is to map the high-dimensional posterior space to a low-dimensional space (1D or 2D) while preserving the discriminative information necessary for accurate classification.

The paper also discusses two main approaches for using ANNs in speech recognition:

  1. ANNs for posterior estimation: These ANNs output posterior probabilities, which are then used as features (often after transformation).
  2. ANNs for feature extraction (Bottleneck features): These ANNs have a specific architecture with a narrow "bottleneck" layer that directly outputs low-dimensional features.

For training the ANNs, the authors explored optimization methods beyond standard backpropagation, including L-BFGS and iRPROP+, noting their faster convergence. They also introduced a stochastic version of the deterministic ANN parameter update step to potentially compensate for poor initializations.

The proposed transforms are linear and are defined by a transformation matrix $M$ and a function $\mu(x)$, such that $y = M \cdot \mu(x)$. The function $\mu(x)$ augments the input posterior vector $x$ with its maximum value and delta indicators. The paper details specific linear transformations for 1D and 2D target spaces, aiming to maximize a "Discriminative Ratio" (DR) which balances the minimum distance between points in the transformed space with the maximum distance.

Datasets

The experiments were conducted on two speech corpora:

  • SpeechDat-East (SD-E): A telephone speech corpus recorded at 8 kHz sampling frequency. This corpus was used for evaluating the performance of the proposed transforms in a monophone-based HMM system.
  • Czech Audio-Visual Corpus (CAVC): This corpus features audio recorded at 44.1 kHz sampling frequency. It was used to evaluate the transforms in an audio-visual speech recognition context, where posteriors from different modalities are combined.

Key Insights and Ideas

  • Low-dimensional posteriors can be discriminative: Contrary to the intuition that significant dimensionality reduction might destroy discriminative information, the proposed transforms demonstrate that it's possible to retain enough information for improved performance.
  • Transformations matter, especially without HLDA: Concatenating transformed posteriors with standard features (like PLP) significantly boosts accuracy, particularly when standard linear transformations like HLDA are not applied to the original features.
  • Stochastic ANN training can be beneficial: The authors proposed and tested a stochastic version of ANN parameter updates, suggesting it can help overcome issues with suboptimal initializations.
  • Bottleneck features' performance: The performance of bottleneck features did not exceed that of PLP features in their experiments, which they attribute to the bottleneck ANN not being specifically adapted to utterances with silence removed.

Key Results

  • Improved Accuracy: Concatenating the proposed 1D and 2D transformed posteriors with PLP features led to noticeable accuracy improvements on the SpeechDat-East corpus. For example, PLP + 1D achieved 79.47% accuracy, and HLDA(PLP) + 2D(square) reached 81.07%.
  • Effectiveness in Audio-Visual Recognition: In the CAVC experiments, the combination of transformed posteriors with PLP features also showed increased accuracy, especially under noisy conditions (e.g., SNR = 0 dB).
  • No single "best" transform: While all three proposed transforms were beneficial, the authors could not definitively state which one yielded the highest accuracy, as the differences were not statistically significant.
  • L-BFGS practicality: The L-BFGS quasi-Newton method was found to be practical for ANN-based posterior estimation.

Limitations and Future Work

  • Bottleneck Feature Adaptation: The authors suggest that the performance of bottleneck features could be improved by adapting the ANN to utterances with silence removed.
  • No definitive best transform: While the transforms are beneficial, the paper doesn't identify a single superior transform, indicating that the choice might depend on specific application needs or further tuning.
  • Limited exploration of fusion methods: In the audio-visual context, the paper mentions using a fusion method that aggregates several elementary fusion methods but doesn't delve deeply into the specifics or compare different fusion strategies extensively.
  • Future work could explore: Further investigation into optimal projection criteria for the transforms, more advanced ANN training techniques, and a broader range of feature combinations and fusion methods for multimodal recognition.

Summary updated

BibTeX
@inproceedings{zelinka2010low,
  author = {Jan Zelinka and Jan Trmal and Luděk Müller},
  title = {{Low-dimensional space transforms of posteriors in speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure