Target-aware language models for spoken language recognition

Rong Tong, Bin Ma, Haizhou Li, Engsiong Chng, Kong-Aik Lee

This paper, presented at INTERSPEECH 2009, introduces a novel methodology to enhance the accuracy of spoken language recognition (SLR) systems without requiring additional labeled training data. By transitioning from "hard" phone selection to "soft" probabilistic weighting, the authors developed Target-Aware Language Models (TALM). These models allow systems to better distinguish between languages by focusing on the most informative phonetic units.

The Challenge of Feature Selection in SLR

In Spoken Language Recognition, the "phonotactic" approach is widely considered the gold standard. This method involves converting audio into a sequence of phonetic tokens (e.g., "p," "t," "ah") and analyzing their patterns to identify the language. However, two significant hurdles persist:

  1. Data Scarcity: Developing high-quality phone recognizers for every target language requires massive amounts of annotated speech data, which is often unavailable.
  2. Inefficient Feature Selection: Not all sounds are equally useful for identification. Some phones are common across many languages, while others are unique to specific ones. Previous attempts to focus on "important" phones (the TOPT method) used a "hard decision" approach, discarding phones deemed less discriminative. This often led to limited coverage and high sensitivity to the specific phones chosen.

Methodology: The TALM Approach

The authors propose a "soft decision" alternative. Instead of discarding phones, they utilize a Target-Aware Language Model (TALM) to weight them based on their discriminative power.

Workflow of the Target-Aware Language Model (TALM) generation process.

The workflow involves several key steps:

  • Discriminative Mapping: A linear Support Vector Machine (SVM) analyzes phone sequences from training data to identify which phones are most influential in separating a target language from others.
  • Weighting: SVM weights are converted into a unigram language model. Highly discriminative phones are assigned higher probabilities.
  • Front-End Integration: Unlike traditional systems that apply language models at the end of the process, TALM is integrated into the front-end. During the initial tokenization phase, the decoder uses the TALM to prioritize the detection of these highly discriminative sounds.
  • Universal Phone Recognizer (UPR): The researchers also utilized a "Universal" recognizer trained on a merged inventory of 300 phones from seven different languages, providing a richer phonetic vocabulary.

Key Results and Insights

The TALM approach consistently outperformed both baseline and previous hard-selection (TOPT) methods across standard NIST Language Recognition Evaluation (LRE) datasets.

Performance and Robustness

On the NIST LRE 1996 dataset, the Universal Phone Recognizer (UPR) baseline achieved an Equal Error Rate (EER) of 4.14%. Applying the TALM method reduced this significantly to 1.34%.

One of the most critical findings was the system's robustness. As the phonetic inventory size increases, traditional hard-selection methods like TOPT often degrade because they cannot handle the added complexity. In contrast, TALM remains stable and continues to improve as more phones are added.

Comparison of EER between TOPT and TALM as phone inventory size increases.

The Power of the "Universal" Listener

Inspired by the observation that multilingual humans are better at identifying unfamiliar languages, the authors found that a Universal Phone Recognizer (UPR) significantly outperformed language-specific ones. By applying language models during the decoding process (the front-end), the system is guided to "pay attention" to the most relevant sounds from the start.

Conclusion and Future Directions

While the current study utilized unigram language models, the authors suggest that moving to n-grams (analyzing sequences of 2 or 3 phones) could capture more complex phonotactic patterns and further improve accuracy. Although TALM increases computational load by running multiple parallel tokenizers, it offers a powerful way to improve SLR performance without the need for new, expensive labeled datasets.

Summary updated

BibTeX
@inproceedings{tong2009target,
  author = {Rong Tong and Bin Ma and Haizhou Li and Engsiong Chng and Kong-Aik Lee},
  title = {{Target-aware language models for spoken language recognition}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-75},
}
Figure