Subspace-based feature representation and learning for language recognition

Yu-Chin Shih, Hung-Shin Lee, Hsin-Min Wang, Shyh-Kang JengInterspeech 2012

TL;DR

This paper introduces a novel "subspace-based" approach for recognizing spoken languages. Instead of relying on traditional methods that can be computationally intensive and lose information, they represent each utterance as a unique "linear subspace." This new representation, combined with kernel-based learning, significantly improves language recognition accuracy, especially for shorter utterances.

What Problem Does This Paper Address?

Automatic Spoken Language Recognition (ASLR) aims to identify the language of an audio recording. Traditional methods often fall into two camps: acoustic modeling (analyzing raw speech features) and phonotactic modeling (analyzing sequences of speech sounds, or "phones"). Phonotactic approaches are promising because languages have distinct patterns in how sounds are combined. However, existing phonotactic methods, like those using Vector Space Modeling (VSM), can struggle with:

  • High dimensionality: Representing all possible phone combinations can lead to extremely large feature vectors.
  • Information loss: VSM can lose the order of phones, and limiting n-gram patterns (like bigrams or trigrams) to manage dimensionality means missing longer-term dependencies.
  • Computational cost: Processing and comparing these large feature vectors can be slow.

This paper seeks to overcome these limitations by developing a more efficient and informative way to represent phonetic information for language recognition.

Methods

The proposed approach has two main parts:

  1. Phonotactic Feature Extraction and Representation:
    • Phone Decoding: First, an utterance is processed by a universal phone recognizer to get a sequence of phone labels and their associated posterior probabilities. Crucially, this includes probabilities for all possible phones, not just the single most likely one, capturing more uncertainty.
  2. Acoustic Feature Aggregation: For each phone segment identified, acoustic features are aggregated based on the average posterior probabilities of phones within that segment. This creates a "phone frame" vector.
  3. Contextual Window: To capture phonotactic constraints (like how phones follow each other), phone frames within a "contextual window" (a set of frames around a central frame) are concatenated. This creates a larger representation for each utterance.
  4. Subspace Generation via SVD: Since utterances vary in length (and thus in the number of phone frames), Singular Value Decomposition (SVD) is used to map each utterance's concatenated feature matrix into a fixed-dimensional linear subspace. Each utterance is then represented by the basis vectors (eigenframes) of this subspace. This is a key innovation, as it represents an utterance not as a single point, but as a geometric space.
  5. Subspace Learning and Classification:
    • Grassmann Metric: The paper leverages the concept of the Grassmann manifold, which is the space of all possible linear subspaces. A specific distance metric, the "Projection metric," is used to measure the similarity between the subspaces representing different utterances. This metric is derived from the principal angles between subspaces.
  6. Kernel LDA (KLDA): To perform classification, Kernel Linear Discriminant Analysis (KLDA) is employed. KLDA maps the subspaces from the Grassmann manifold into a high-dimensional Hilbert space where they are expected to be more linearly separable. This allows a linear classifier to find the best separation between languages.
  7. k-Nearest Neighbor (k-NN) Classifier: For the final classification step, a simple k-NN classifier is used, leveraging the distances computed by KLDA.

Datasets

  • OGI-TS Corpus: This is the primary dataset used for experiments. It contains speech from 10 languages. The corpus is divided into training, development, and test sets.
  • A "3-s" test set was created by culling shorter utterances from the main test set to evaluate performance on short audio segments.
  • A "1-to-50-s" test set includes all test utterances.
  • The paper also mentions the use of 619 "story-before-tone" utterances with hand-generated phonetic transcriptions for supervised phone modeling.

Key Insights and Ideas

  • Representing utterances as subspaces: This is the core novelty. Instead of a single feature vector, an utterance is characterized by the geometric space its features occupy. This allows for capturing more complex relationships and potentially more robust representations.
  • Leveraging all phone probabilities: By not discarding less likely phone probabilities, the system can incorporate more nuanced information and uncertainty, which is crucial for distinguishing similar languages.
  • Subspace distance for similarity: Using metrics like the Projection metric on the Grassmann manifold provides a principled way to compare these learned subspaces, going beyond simple Euclidean distances.
  • Contextual window size matters: The experiments suggest that a moderate contextual window (e.g., n=1, meaning 3 consecutive phone frames) is optimal, indicating that very long-range dependencies might not always improve performance and could even introduce noise.

Key Results

The proposed "UPR-Subspace" method significantly outperforms two well-established phonotactic approaches (UPR-LM and UPR-VSM).

  • Performance Improvement: UPR-Subspace achieved relative reductions of 38.90% and 27.13% in Equal Error Rate (EER) compared to UPR-VSM on the 3-second and 1-to-50-second datasets, respectively.
  • Eigenframe Optimization: The number of eigenframes (the dimensionality of the subspace) has a significant impact on EER, with a moderate value yielding the best results.
  • Context Window Optimization: Larger contextual windows (n > 1) did not consistently improve EER, suggesting a trade-off between capturing context and introducing noise.

Limitations and Future Work

  • Classifier Choice: While KLDA and k-NN performed well, the authors suggest that integrating more discriminative kernel-based classifiers, such as SVM, with the subspace representation could yield even better results.
  • Dataset Generalization: The experiments were conducted on the OGI-TS corpus. Future work includes evaluating the framework on more standard corpora like the NIST LRE corpora to assess its broader applicability.
  • Comparison with Other Subspace Methods: The paper focuses on comparing its method to VSM-based approaches. Implementing and comparing against other subspace-based methods (like iVector or PCA, mentioned in the references) would be a valuable next step.

Summary updated

BibTeX
@inproceedings{shih2012subspace,
  author = {Yu-Chin Shih and Hung-Shin Lee and Hsin-Min Wang and Shyh-Kang Jeng},
  title = {{Subspace-based feature representation and learning for language recognition}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure