Phonotactic language recognition using MLP features

Mohamed Faouzi BenZeghiba, Jean-Luc Gauvain, Lori LamelInterspeech 2012

TL;DR

This paper introduces a faster and more accurate phonotactic language recognition system by incorporating Multi-Layer Perceptron (MLP) features. By leveraging MLP's ability to capture temporal context and discriminative information, the proposed system achieves performance comparable to more complex methods while being significantly faster, making it more practical for real-world applications.

What Problem Does This Paper Address?

Language recognition systems often rely on phonotactic approaches, which analyze the sequence of sounds (phonemes) to identify a language. Traditional methods using context-independent (CI) phone models are fast but less accurate, while context-dependent (CD) models are more accurate but computationally expensive, hindering their use in real-time applications. Furthermore, phonotactic methods tend to perform poorly on short speech segments. This paper aims to bridge this gap by developing a system that is both accurate and efficient, addressing the speed and short-segment performance limitations of existing phonotactic language recognition systems.

Methods

The core of the proposed system is the Parallel Phone Recognizers followed by Language Modeling (PPRLM) framework. The key innovation lies in the use of MLP features concatenated with conventional Perceptual Linear Prediction (PLP) and pitch features (referred to as mlpplpf0).

  • MLP Features: These features are derived from a bottleneck layer of a Multi-Layer Perceptron. MLPs are trained to focus on phonetic content and reduce irrelevant information, providing more discriminative and temporally contextualized features compared to traditional PLP features.
  • Feature Concatenation: The mlpplpf0 features are created by combining these MLP-derived bottleneck features with conventional PLP features and pitch information.
  • PPRLM System: The system uses multiple phone recognizers, one for each target language, trained on these concatenated features. These recognizers generate phone lattices, which are then used to build phonotactic language models.
  • CMLLR Adaptation: For improved performance, especially on longer segments, Context-Independent Linear Regression (CMLLR) adaptation is applied. For the MLP-based system, a block-diagonal CMLLR transform is used to manage computational complexity.

Datasets

  • Training Data:
    • US English: Over 2000 raw hours of conversational telephone speech.
  • French: Approximately 1100 hours of French conversational data.
  • Spanish: 138 hours of Spanish data.
  • Target language phonotactic models were trained using the CallFriend database.
  • Evaluation Data: NIST LRE 2005 evaluation data sets, with most segments from the OHSU database.
  • Development Data: A development set composed of four NIST data sets (LRE96, LRE03 eval, LRE05 dev) was used for calibrating fusion module parameters.

Key Insights and Ideas

  • MLP Features for Temporal Context: The MLP's ability to capture long-span temporal context at the feature level is crucial for improving language recognition accuracy, especially for phonotactic approaches.
  • Discriminative Power of MLP: MLP training, which focuses on phonetic content and reduces noise, leads to more accurate phone recognizers and, consequently, better phone lattices.
  • Complementarity of Features: Concatenating MLP features with PLP and pitch features provides complementary information. This is evidenced by the fact that the MLPPLPF0 system, where both recognizers and features differ, shows greater improvement than PLP-based systems where only the recognizers differ.
  • Efficiency Gains: The proposed MLP-based system achieves performance comparable to more complex CD PLP systems but runs significantly faster (up to 6 times), making it more practical for real-world applications.
  • CMLLR Adaptation and MLP: CMLLR adaptation is beneficial, but the MLP features themselves already perform some of the normalization that CMLLR aims to achieve, making the adaptation less critical for MLP-based systems compared to PLP-based ones.

Key Results

  • Performance Improvement: The CI MLPPLPF0 PPRLM system (without CMLLR adaptation) showed relative improvements of 17%, 7.4%, and 4% for French, Spanish, and English PRLMs, respectively, compared to the adapted CI PLP system.
  • System-Level Improvement: The CI MLPPLPF0 PPRLM system achieved a 24% relative improvement over the CI PLP PPRLM system.
  • Speed vs. Accuracy: The CI MLPPLPF0 PPRLM system performed better than the CD PLP PPRLM system (7% relative improvement) while being approximately 6 times faster.
  • Short Segments: On 10s and 3s conditions, the MLPPLPF0 system outperformed the CI PLP system but was slightly less effective than the CD PLP system.
  • CMLLR Adaptation Impact: CMLLR adaptation provided relative improvements of 17% for CD PLP and 14% for CI MLPPLPF0 systems.

Limitations and Future Work

  • Decoding Parameters: The authors note that the decoding parameters for the proposed system have not yet been fully optimized, suggesting that further performance improvements are possible.
  • Context-Dependent Models with MLP: Future work will focus on efficiently incorporating MLP features within PPRLM systems that use context-dependent phone models to achieve even greater improvements. This is acknowledged as a computationally challenging task.
  • Optimization for Real-World Applications: The paper highlights the need for efficient methods to integrate MLP features with CD models without incurring prohibitive computational costs, aiming for real-world applicability.
  • Further Evaluation: Ongoing work includes optimizing the system and evaluating it on other NIST LRE eval data sets, with initial results on LRE07 showing a 20% relative improvement over the CMLLR adapted CD PLP system.

Summary updated

BibTeX
@inproceedings{benzeghiba2012phonotactic,
  author = {Mohamed Faouzi BenZeghiba and Jean-Luc Gauvain and Lori Lamel},
  title = {{Phonotactic language recognition using MLP features}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure