Phonotactic language recognition using ivvectors and phoneme posteriogram counts

Luis Fernando D'Haro, Ondřej Glembek, Oldřich Plchot, Pavel Matějka, Mehdi Soufifar, Ricardo Cordoba, Jan ČernockýInterspeech 2012

TL;DR

This paper introduces a novel approach to language identification (LID) by using "phoneme posteriogram counts" as features, which are then reduced to low-dimensional "i-vectors." This method significantly improves upon existing phonotactic LID systems and achieves state-of-the-art results when fused with acoustic i-vectors.

What Problem Does This Paper Address?

Language Identification (LID) is the task of determining which language is being spoken in an audio recording. Traditional LID systems fall into two main categories: acoustic-based (analyzing speech features like pitch and intonation) and phonotactic-based (analyzing sequences of phonemes). Phonotactic systems, while promising, often struggle with data sparsity and the high dimensionality of n-gram counts derived from phone lattices. This paper addresses the challenge of creating more robust and compact feature representations for phonotactic LID, aiming to overcome the limitations of previous methods that relied on "soft-counts" from phone lattices or simple PCA for dimensionality reduction.

Methods

The core innovation lies in the feature extraction process. Instead of directly using n-gram counts from phone lattices, the authors propose using phoneme posteriogram counts. This involves:

  1. Phone Recognition: Using a phone recognizer to obtain posterior probabilities of phoneme occurrences for each speech frame.
  2. Averaged Posteriogram: Summing and averaging these posterior probabilities over frames identified as belonging to the same phoneme unit. This step incorporates prior information about phoneme boundaries and aims to create a more robust representation.
  3. Joint-Posteriogram n-grams: Computing joint probabilities of consecutive phonemes by taking the outer product of averaged posteriograms. This creates a sequence of matrices representing n-gram counts.
  4. Dimensionality Reduction with i-vectors: These n-gram counts are then converted into a high-dimensional vector. To handle this, the paper adapts the i-vector framework, originally developed for speaker recognition. This involves using a Subspace Multinomial Model (SMM) to learn a low-dimensional representation (i-vectors) in a total variability space. This process effectively reduces the dimensionality while preserving important information and mitigating sparsity issues.

The system uses a multiclass logistic regression classifier trained on these i-vectors. For comparison and fusion, an acoustic i-vector system is also employed.

Datasets

  • Training and Development Data: The paper utilizes a combination of standard LID databases, including Callfriend, Fisher English Part 1 & 2, Fisher Levantine Arabic, HKUST Mandarin, and Mixer (from NIST SRE 2004, 2005, 2006, 2008). A balanced dataset with a maximum of 500 utterances per language for the 23 target languages defined for NIST LRE 2009 was used for training the i-vector extractor and classifier. A separate development dataset, comprising data from NIST LRE 2007, Foreign Accented English, OGI-multilingual, SpeechDat-East, OGI 22 languages, Voice of America radio broadcast, and Switch Board, was used for calibration and fusion.
  • Test Data: The NIST LRE 2009 set was used for evaluation.

Key Insights and Ideas

  • Posteriograms as a Robust Feature: Using averaged posteriograms instead of raw phone lattice counts provides a more stable and less sparse representation of phoneme sequences.
  • i-vectors for Discrete Features: The successful adaptation of i-vectors, typically used with continuous acoustic features, to discrete phonotactic counts is a significant contribution. This demonstrates the versatility of the i-vector framework.
  • Subspace Multinomial Model (SMM) for Discrete Data: The use of SMM is crucial for learning i-vectors from discrete n-gram counts, effectively modeling the total variability in this discrete feature space.
  • Fusion is Key: While the proposed phonotactic system performs well on its own, fusing it with an acoustic i-vector system yields the best results, highlighting the complementary nature of these approaches.

Key Results

The proposed phonotactic LID system using i-vectors derived from posteriogram counts achieved better results than a system using soft-counts from phone lattices. Specifically, for the 30-second condition on the NIST LRE 2009 set, the proposed system achieved a Cavg of 3.15% compared to 3.43% for the soft-counts system.

When fused with an acoustic i-vector LID system, the proposed system achieved a Cavg of 1.25% on the 30-second condition, outperforming both the acoustic system alone (2.4%) and previous i-vector based phonotactic systems. The paper also shows that the proposed method outperforms a PCA-based dimensionality reduction technique applied to soft-counts.

Limitations and Future Work

  • Limitations: The paper acknowledges that the system is compared against other phonotactic systems and an acoustic system. While fusion yields excellent results, the performance of the phonotactic system alone is still lower than the acoustic system. The paper also mentions that the i-vector size can impact performance, and finding the optimal size is important.
  • Future Work: The authors suggest incorporating an additional mechanism for selecting discriminative n-grams to further reduce the input vector size. They also propose using this selection as a scaling factor during the fusion and calibration process.

Summary updated

BibTeX
@inproceedings{dharo2012phonotactic,
  author = {Luis Fernando D'Haro and Ondřej Glembek and Oldřich Plchot and Pavel Matějka and Mehdi Soufifar and Ricardo Cordoba and Jan Černocký},
  title = {{Phonotactic language recognition using ivvectors and phoneme posteriogram counts}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure