Improvements to generalized discriminative feature transformation for speech recognition

Roger Hsiao, Florian Metze, Tanja Schultz

Abstract

This paper introduces rGDFT, a regularized version of the Generalized Discriminative Feature Transformation algorithm. By adding a penalty term that prevents feature transforms from straying too far from a "backoff" model, the authors enable the use of thousands of transforms without overfitting. This approach results in significant accuracy gains and a 50% reduction in training time through joint optimization of feature and acoustic model parameters.

Introduction

In Automatic Speech Recognition (ASR), discriminative feature training aims to transform raw audio features into a space where speech sounds are more easily distinguishable. A previous method, GDFT, utilized Lagrange relaxation to adapt likelihood-based transforms into discriminative ones. However, GDFT was unstable; increasing the number of transforms to capture more detail often led to overfitting and performance degradation. This work addresses these stability issues through regularization.

Regularized GDFT (rGDFT)

The core innovation is the addition of a regularization term to the objective function. Specifically, a Frobenius norm distance measure, $\frac{D}{2} ||W - W^0||^2_F$, is added, where $W$ is the learned transform, $W^0$ is a "backoff" transform (typically the identity matrix), and $D$ is a tunable penalty parameter.

By converting the primal problem into a dual problem using Lagrange multipliers, the authors derived an iterative optimization process compatible with model-space discriminative training, such as Boosted MMI (BMMI). This compatibility allows for joint training, where both the feature space and the acoustic model are optimized simultaneously.

Table showing the impact of regularization on WER as the number of transforms increases.

As shown in the table above, the original GDFT method suffered a regression in Word Error Rate (WER) when increasing from 16 to 1024 transforms (36.7% to 38.5%). In contrast, rGDFT remains stable, with performance continuing to improve as the number of transforms increases to 2048 and beyond.

Experimental Results

The researchers evaluated rGDFT on two challenging tasks: Iraqi Arabic (450 hours) and a large-scale Modern Standard Arabic (MSA) system (1,150 hours).

Iraqi Arabic Task

On the Iraqi task, rGDFT combined with BMMI achieved a 32.0% WER, representing a 10.4% relative improvement over the baseline ML system (35.7%).

WER vs. Iteration for various training methods, highlighting the efficiency and accuracy of joint rGDFT+BMMI training.

The figure above illustrates the convergence of different training procedures. Notably, the joint training approach (rGDFT+BMMI) achieves superior accuracy compared to sequential training (rGDFT -> BMMI) while requiring significantly less total training time—effectively cutting the pipeline's computational cost in half.

Large-Scale Arabic Task

For the GALE 2009 evaluation, the rGDFT+BMMI approach reduced the WER from 16.1% to 15.2% on the evaluation set. The system demonstrated robustness across different acoustic setups, though gains were more pronounced in unvowelized systems compared to vowelized ones.

Conclusion

The introduction of regularization to GDFT enables the use of high-resolution feature transformations without the risk of overfitting. The ability to perform joint optimization with BMMI provides a significant efficiency boost, achieving state-of-the-art accuracy while halving the training time. Future work will explore more sophisticated backoff strategies and the integration of I-smoothing for further stability.

Summary updated

BibTeX
@inproceedings{hsiao2010improvements,
  author = {Roger Hsiao and Florian Metze and Tanja Schultz},
  title = {{Improvements to generalized discriminative feature transformation for speech recognition}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-21},
}
Figure