Acoustic class specific VTLN-warping using regression class trees

S. P. Rath, S. UmeshInterspeech 2009

TL;DR

This paper introduces a more efficient and effective way to normalize speech for speaker independence by adapting vocal tract length (VTLN) warping factors to specific acoustic classes within an utterance. By using regression class trees, they achieve better word recognition accuracy than traditional global VTLN, especially when data is limited for certain acoustic classes.

What Problem Does This Paper Address?

Speaker-independent speech recognition systems struggle with inter-speaker variability, meaning the same word can sound very different depending on who is speaking. Vocal Tract Length Normalization (VTLN) is a common technique to reduce this variability by "warping" the speech spectrum to a standard vocal tract length. Traditionally, a single warp factor is applied to the entire utterance. However, this paper argues that this "one-size-fits-all" approach is suboptimal because different speech sounds (acoustic classes) naturally vary spectrally in different ways due to physiological differences. Applying a single warp factor might not optimally normalize all sounds. The challenge is to efficiently estimate and apply different warp factors for different acoustic classes within an utterance without significantly increasing computational complexity or requiring excessive adaptation data.

Methods

The authors build upon their previous work on VTLN using a Linear Transformation (LT) of Mel-Frequency Cepstral Coefficients (MFCCs). This LT approach allows for computationally efficient VTLN by pre-computing warp matrices, avoiding the need for repeated spectral warping and re-computation of features.

The core innovation here is the use of regression class trees to estimate acoustic class-specific warp factors. Here's how it works:

  1. Acoustic Classes: Instead of a single global warp factor, they define multiple "acoustic classes." These classes can be derived in two ways:
  2. Phonetic Knowledge: Grouping Gaussian components in the acoustic model based on broad phonetic categories (e.g., vowels, fricatives).
  3. Data-Driven Approach: Using a binary regression tree to recursively split Gaussian components based on their Euclidean distance, forming classes automatically.
  4. Sufficient Statistics: For each acoustic class, they collect specific statistical measures (similar to those used in CMLLR speaker adaptation) from the un-warped MFCC features.
  5. Warp Factor Estimation: Using these class-specific statistics and the pre-computed LT warp matrices, they efficiently estimate a unique warp factor for each acoustic class. The regression class tree structure is crucial because it allows warp factors to be estimated even for classes with very little data by leveraging information from their parent classes in the tree.

This approach is computationally efficient because the LT framework avoids repeated grid searches for warp factors, and the use of sufficient statistics makes adaptation data requirements low.

Datasets

  • Wall Street Journal (WSJ) database: Used for training and testing. Specifically, the WSJ0-84 training set was used for acoustic model training, and the November 1992 WSJ test set (5K closed non-verbalized vocabulary) was used for evaluation.
  • Acoustic Models: Cross-word tri-phone models with 3 states and 8 diagonal-covariance components per state were used. A separate model was used for silence.
  • Features: 39-dimensional MFCCs (including normalized log-energy, derivatives) were used, with 20 ms frames and 10 ms overlap. Cepstral mean subtraction was applied per utterance.

Key Insights and Ideas

  • VTLN is not one-size-fits-all: The paper strongly supports the intuition that different speech sounds require different spectral warping for optimal normalization. The observed variations in warp factors across phonetic classes in Table 4 are a key piece of evidence.
  • Regression Class Trees are powerful for limited data: The ability of regression class trees to "borrow" information from parent nodes is critical. This allows for the estimation of class-specific warp factors even when a particular acoustic class has insufficient training data, a common problem in speech recognition.
  • Efficiency is maintained: By leveraging their LT-based VTLN framework and sufficient statistics, the proposed method achieves acoustic class-specific warping without the computational burden of traditional grid searches or complex alignment procedures.
  • Phonetic vs. Data-Driven Classes: Both approaches to defining acoustic classes yielded comparable results, suggesting flexibility in how classes can be defined.

Key Results

  • Improved Word Recognition Accuracy (WRA): Using acoustic class-specific warp factors consistently outperformed the global warp factor approach across various numbers of regression classes.
  • Trend of Improvement: WRA generally increased with the number of acoustic classes used for warp-factor estimation, with the highest accuracy achieved for 10 classes in the data-driven approach.
  • Phonetic Class Performance: The phonetic knowledge-based approach with 12 classes achieved comparable WRA to the data-driven approach with 12 classes.
  • Specific Warp Factor Variations: Table 4 shows significant differences in warp factors for certain phonetic classes (e.g., Unvoiced Stops, Strong Fricatives) compared to the global warp factor, validating the core hypothesis.
  • Baseline Performance: The baseline performance (no normalization) was 93.60%. With adaptation using acoustic class-specific VTLN, accuracies reached up to 94.25% (data-driven, 10 classes, adaptive training).

Limitations and Future Work

  • Silence Warping: The paper notes that it's generally better not to warp silence. This suggests that silence might be treated as a special case or excluded from warping altogether.
  • Optimal Number of Classes: While WRA increased with the number of classes, the paper doesn't definitively state the absolute optimal number, suggesting further exploration might be beneficial.
  • Data-Driven vs. Phonetic: While both approaches performed well, the paper doesn't delve deeply into why one might be preferred over the other in different scenarios, leaving room for future investigation into the trade-offs.
  • Further Exploration of Class Definitions: The authors suggest that the flexibility in defining acoustic classes is a strength, implying that exploring other methods for class definition could be a fruitful area for future research.

Summary updated

BibTeX
@inproceedings{rath2009acoustic,
  author = {S. P. Rath and S. Umesh},
  title = {{Acoustic class specific VTLN-warping using regression class trees}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure