Improved language recognition using mixture components statistics

Abualsoud Hanani, Michael Carey, Martin J. RussellInterspeech 2010

TL;DR

This paper introduces a novel approach to automatic spoken language recognition by using discriminative training of language models based on Gaussian Mixture Model (GMM) component statistics. By employing a Multi-Language Model (MLM) as a tokenizer and a Support Vector Machine (SVM) for discriminative weighting and feature selection, the authors achieve state-of-the-art performance, significantly outperforming traditional methods.

What Problem Does This Paper Address?

Automatic spoken language recognition (LRE) aims to identify the language of an unknown spoken utterance. Traditional approaches often rely on phonotactic models, which require expensive and time-consuming phonetic transcriptions for training. While GMMs have been used as acoustic front-ends, this paper addresses the challenge of building more effective and efficient language models that can leverage acoustic information without requiring explicit phonetic labeling. The authors aim to improve the discriminative power of language models by focusing on the statistical properties of acoustic features represented by GMM components, rather than relying solely on traditional n-gram models.

Methods

The core of the proposed method involves using GMMs as "tokenizers" to convert acoustic frames into sequences of component indices. This bypasses the need for phonetic transcriptions. The paper explores several key techniques:

  • GMM-UBM (Universal Background Model): A baseline system using a standard GMM-UBM for tokenization and inter-session compensation.
  • GMM-SVM (Supervectors): This system maps utterances to high-dimensional "supervectors" derived from GMM mean vectors, which are then used to train SVMs for classification. This approach aims to make languages more linearly separable.
  • Discriminative Language Modeling with SVM: The paper proposes using SVMs for discriminative training of language models. Instead of traditional n-gram probabilities, the SVM is trained on n-gram statistics of GMM component sequences.
  • LLR Weighting: To emphasize discriminative components, a Log-Likelihood Ratio (LLR) weighting technique is applied to the n-gram probabilities. This technique, adapted from information retrieval, assigns higher weights to components that are common in one language but rare in others.
  • Multi-Language Model (MLM): A novel approach that concatenates multiple language-dependent GMMs. This MLM is hypothesized to provide more language-specific components than a traditional UBM, leading to better discriminative power for the uni-gram system.
  • Feature Selection: An iterative SVM-based feature selection algorithm is used to minimize the number of N-gram components, reducing computational cost and memory requirements.
  • Fusion: The output scores from multiple systems (GMM-UBM, GMM-SVM, uni-gram, and bi-gram) are fused using multi-class linear logistic regression to achieve improved performance.

Datasets

  • Training and Development Data: The Callfriend corpus, containing twenty half-hour telephone conversations for each of fifteen languages and dialects.
  • Evaluation Data: The NIST 2003 Language Recognition Evaluation (LRE) closed-set task, specifically using 30-second test utterances. The evaluation subset consisted of 1280 utterances (80 per language), with some languages having specific corpus origins (Callfriend, Callhome). Russian utterances were excluded as the focus was on closed-set recognition.

Key Insights and Ideas

  • GMMs as Phonetic-Free Tokenizers: Using GMM components as tokens is a viable and efficient alternative to phonetic transcriptions for language recognition.
  • Discriminative Training for Language Models: SVMs can effectively learn discriminative language models from GMM component sequences, outperforming traditional n-gram approaches.
  • LLR Weighting is Crucial: Emphasizing language-specific GMM components through LLR weighting significantly boosts the discriminative power of the language models.
  • MLM's Advantage for Discriminative Models: The MLM, by providing more language-specific components, proves highly beneficial for discriminative uni-gram systems, but less so for generative GMM-UBM systems. This highlights a difference in how generative and discriminative models leverage acoustic information.
  • Complementarity of Systems: Fusing different types of systems (GMM-based, SVM-based, uni-gram, bi-gram) leads to substantial performance gains, indicating that these systems capture complementary information.

Key Results

The paper reports significant improvements in language recognition performance:

  • A discriminative uni-gram system using a large MLM tokenizer achieved an Equal Error Rate (EER) of 1.66% on the NIST'03 LRE 30s evaluation.
  • When fused with other acoustic approaches, the performance improved to an EER of 0.71% and a Cavg of 0.0098.
  • The GMM-SVM system achieved a pooled EER of 0.82%, demonstrating strong performance on its own.
  • Table 1 shows the performance of the four main systems: GMM-UBM (3.4% EER), GMM-SVM (0.82% EER), Uni-gram (1.66% EER), and Bi-gram (3.51% EER). The fused system achieved the best result at 0.71% EER.
  • Table 2 illustrates that the MLM background model is advantageous for the uni-gram system, yielding better results than traditional UBMs.

Limitations and Future Work

  • Computational Cost: While advancements like GPU acceleration are mentioned, training large MLMs and SVMs can still be computationally intensive.
  • MLM vs. UBM for Generative Models: The paper notes that the MLM is more beneficial for discriminative models (like the uni-gram system) than for generative GMM-UBM systems, suggesting further research into how generative models can best leverage MLM structures.
  • Future Work: The authors suggest that the MLM is more biased towards language-specific components, which is useful for discriminative approaches. This implies that further exploration of MLM architectures and their integration with various discriminative techniques could yield even better results. The success of fusion also points towards exploring more sophisticated fusion strategies.

Summary updated

BibTeX
@inproceedings{hanani2010improved,
  author = {Abualsoud Hanani and Michael Carey and Martin J. Russell},
  title = {{Improved language recognition using mixture components statistics}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure