Abstract
In this paper, we propose an end-to-end short utterance speech language identification (SLD) approach based on a Long Short-Term Memory (LSTM) neural network, specifically designed for SLD applications in intelligent vehicles. Features for LSTM learning are generated via transfer learning: bottleneck features from a Deep Neural Network (DNN) trained for Mandarin acoustic-phonetic classification are utilized. To improve SLD accuracy for short utterances, a phase vocoder-based Time-Scale Modification (TSM) method is used to adjust the speech rate. By splicing the original, slowed, and sped-up utterances, we extend the effective length of test utterances to improve system performance. Experimental results on the AP17-OLR database demonstrate that the proposed method significantly improves SLD performance, particularly for short utterances of 1s and 3s durations.
Introduction
Speech Language Identification (SLD) serves as the critical front-end for multilingual voice assistants, particularly in the context of intelligent vehicles. While modern SLD systems perform well on long recordings, their accuracy significantly degrades when processing short utterances (under 3 seconds). Short clips lack the statistical depth required by traditional models like i-vectors or Gaussian Mixture Models (GMMs). This paper addresses the challenge of short-utterance SLD by leveraging transfer learning and temporal data augmentation.
Methodology
The proposed system utilizes a three-stage pipeline designed to maximize information extraction from limited audio data.
Deep Bottleneck Features
Instead of standard acoustic features like MFCCs, the authors employ Deep Bottleneck (BN) features. A Deep Neural Network (DNN) is first trained for phonetic classification on a large Mandarin dataset. The activations from the bottleneck layer are then extracted. These features capture high-level phonetic contrasts that are more discriminative across languages than raw acoustic signals.
Time-Scale Modification (TSM)
To compensate for the lack of temporal data in short utterances, a phase vocoder-based TSM method is applied. This technique adjusts the speech rate without altering pitch or prosody.
For each utterance, two additional versions are generated: a slowed-down version ($\alpha=0.8$) and a sped-up version ($\alpha=1.2$). These are concatenated with the original signal to create a longer, more diverse input sequence for the classifier.
LSTM Classification
The concatenated BN features are fed into a two-layer Long Short-Term Memory (LSTM) network. LSTMs are well-suited for this task as they can model the long-term temporal dependencies and phonetic "flow" characteristic of different languages.
Experimental Results
The system was evaluated on the AP17-OLR database, covering 10 languages including Kazakh, Tibetan, Uyghur, Cantonese, Mandarin, Indonesian, Japanese, Russian, Korean, and Vietnamese. The training data consisted of approximately 10 hours of recording per language.
The proposed TSM-BN-LSTM model demonstrated superior performance, particularly on the most challenging short-duration subsets.
| Model | Test-All (EER %) | Test-3s (EER %) | Test-1s (EER %) |
|---|---|---|---|
| i-Vector (Baseline) | 6.94 | 8.67 | 17.24 |
| Standard LSTM | 9.64 | 11.05 | 14.30 |
| Proposed (TSM-BN-LSTM) | 0.08 | 2.62 | 6.76 |
On 1-second clips, the proposed method reduced the Equal Error Rate (EER) from 17.24% (i-vector) to 6.76%. For full-length utterances, the EER reached a near-perfect 0.08%.
Discussion and Conclusion
The results highlight two key insights. First, phonetic features learned from one language (Mandarin) generalize effectively to others, suggesting a universal utility for bottleneck features in SLD. Second, artificial length extension via TSM provides the LSTM with a richer temporal context, significantly boosting robustness for short commands.
Future work will explore the impact of multilingual phonetic extractors and the system's robustness in high-noise automotive environments. The compact model size (~20MB) makes it a viable candidate for deployment on edge devices in intelligent vehicles.
Summary updated