An i-vector based approach to training data clustering for improved speech recognition

Yu Zhang, Jian Xu, Zhijie Yan, Qiang Huo2011 · DOI

TL;DR

This paper introduces an efficient i-vector based approach to cluster large speech training datasets. By clustering utterances into groups and training specialized acoustic models for each group, they achieve improved speech recognition accuracy, particularly for voice search tasks.

What Problem Does This Paper Address?

The field of Automatic Speech Recognition (ASR) has seen a surge in available training data from diverse sources and environments. While using more data generally improves ASR performance, simply training a single acoustic model (AM) on a massive, heterogeneous corpus can lead to suboptimal results. This is because a single model might struggle to capture the nuances present across different acoustic conditions, speaker characteristics, or recording devices. The challenge, therefore, is to effectively leverage this vast amount of data to build more robust and accurate ASR systems. Previous approaches have explored clustering training data to train multiple specialized AMs, but these methods can be computationally expensive, limiting their scalability to very large datasets.

Methods

The core of this paper's contribution lies in its novel approach to data clustering for AM training. Here's a breakdown of their methods:

  • i-Vector Extraction: Instead of directly clustering raw audio features, the authors first extract "i-vectors" from each training utterance. An i-vector is a low-dimensional representation that captures the "total variability" of speech, encompassing speaker and channel characteristics, independent of the linguistic content. This is achieved using a factor analysis model.
  • Hierarchical Divisive Clustering (LBG Algorithm): Once i-vectors are extracted for the entire training corpus, they employ the Linde-Buzo-Gray (LBG) algorithm, a hierarchical divisive clustering method, to group these i-vectors into multiple clusters. The similarity between i-vectors is measured using cosine similarity.
  • Cluster-Dependent Acoustic Model Training: For each identified cluster, a specialized acoustic model (AM) is trained. These "cluster-dependent" (CD) AMs are initialized using a "cluster-independent" (CI) AM trained on the entire dataset. This allows each CD AM to specialize in the characteristics of its corresponding cluster.
  • Multiple AM Recognition Strategies: The paper explores several strategies for using these multiple AMs during recognition. These range from simply selecting the best-matching CD AM based on the utterance's i-vector (Method 1) to more complex "parallel decoding" approaches that combine predictions from multiple AMs.

Datasets

  • Training Data: Approximately 7,500 hours of speech data were used for training. This dataset includes a mix of manually transcribed data (connected digits, read sentences, broadcast news, conversational telephony speech) and data from the Windows Live voice search service (with an approximate 85% word-level transcription accuracy).
  • Testing Data: 4,726 real-world voice search queries were used for evaluation.

Key Insights and Ideas

  • i-Vectors as a Powerful Representation for Clustering: The paper demonstrates that i-vectors, which capture speaker and channel variability, are an effective representation for clustering speech data. This is more efficient than directly clustering high-dimensional acoustic features.
  • Scalability of the i-Vector Approach: The proposed i-vector extraction and clustering method is highlighted as being highly efficient, making it suitable for very large training corpora that are common in modern ASR systems.
  • Effectiveness of Specialized AMs: Training AMs tailored to specific data clusters leads to improved recognition accuracy compared to a single, general-purpose AM.
  • Trade-offs in Recognition Strategies: The paper clearly illustrates the trade-off between computational cost and recognition performance when using multiple AMs. Simpler methods (like selecting a single best-matching AM) are faster but less accurate than more complex methods (like parallel decoding).

Key Results

The paper reports significant improvements in Word Error Rate (WER) on a voice search task.

  • The baseline system (single, cluster-independent AM) achieved a WER of 42.2%.
  • Method 1 (i-vector based cluster selection): Achieved a relative WER reduction of 3.1% against the baseline, with a WER of 40.9%. This method is the most computationally efficient.
  • Method 3 (i-vector based cluster selection with parallel decoding of top 2 AMs): This method offered the best trade-off between efficiency and effectiveness, achieving a WER of 39.9% (a 5.5% relative reduction).
  • Method 5 (Parallel decoding using all CD AMs and the CI AM): Also achieved a 39.4% WER (a 6.6% relative reduction), but at a higher computational cost.

The tables in the paper provide detailed breakdowns of WER across different subsets of testing sentences, showing that the performance gains vary depending on the specific acoustic characteristics of the test data.

Limitations and Future Work

  • Negative Effect of Non-Speech: The authors acknowledge that non-speech portions in utterances can negatively impact i-vector extraction. They suggest that using an efficient Voice Activity Detection (VAD) to remove these portions before i-vector extraction would be beneficial, especially for voice search data which often has long non-speech segments.
  • Larger and More Representative Testing Set: For future work, they plan to evaluate their approach on a larger and more representative testing set to gain deeper insights.
  • Comparison with Previous Methods: While their approach is more efficient than a previously reported method [1], a direct comparison of effectiveness is suggested as future work.
  • In-depth Analysis of Results: Further analysis of the results on larger datasets is needed to identify better strategies for leveraging the clustered AMs.

Summary updated

Abstract

We present a new approach to clustering training data for improved speech recognition. Given a training corpus, a so-called i-vector is extracted from each training utterance. A hierarchical divisive clustering algorithm is then used to cluster the training i-vectors into multiple clusters. For each cluster, an acoustic model (AM) is trained accordingly. Such trained multiple AMs can then be used in recognition stage to improve recognition accuracy. The proposed approach is very efficient therefore can deal with very large scale training corpus on current mainstream computing platforms. We report experimental results on a voice search task with 7,500 hours of speech training data.
BibTeX
@misc{zhang2011i,
  author = {Yu Zhang and Jian Xu and Zhijie Yan and Qiang Huo},
  title = {{An i-vector based approach to training data clustering for improved speech recognition}},
  year = {2011},
  doi = {10.21437/interspeech.2011-179},
}
Figure