Language Identification with Deep Bottleneck Features

Zhanyu Ma, Hong YuarXiv 2018 · OpenReview

Abstract

In this paper, we propose an end-to-end short utterance speech language identification (SLD) approach based on a Long Short-Term Memory (LSTM) neural network, specifically designed for SLD applications in intelligent vehicles. Features for LSTM learning are generated via transfer learning: bottleneck features from a Deep Neural Network (DNN) trained for Mandarin acoustic-phonetic classification are utilized. To improve SLD accuracy for short utterances, a phase vocoder-based Time-Scale Modification (TSM) method is used to adjust the speech rate. By splicing the original, slowed, and sped-up utterances, we extend the effective length of test utterances to improve system performance. Experimental results on the AP17-OLR database demonstrate that the proposed method significantly improves SLD performance, particularly for short utterances of 1s and 3s durations.

Introduction

Speech Language Identification (SLD) serves as the critical front-end for multilingual voice assistants, particularly in the context of intelligent vehicles. While modern SLD systems perform well on long recordings, their accuracy significantly degrades when processing short utterances (under 3 seconds). Short clips lack the statistical depth required by traditional models like i-vectors or Gaussian Mixture Models (GMMs). This paper addresses the challenge of short-utterance SLD by leveraging transfer learning and temporal data augmentation.

Methodology

The proposed system utilizes a three-stage pipeline designed to maximize information extraction from limited audio data.

Deep Bottleneck Features

Instead of standard acoustic features like MFCCs, the authors employ Deep Bottleneck (BN) features. A Deep Neural Network (DNN) is first trained for phonetic classification on a large Mandarin dataset. The activations from the bottleneck layer are then extracted. These features capture high-level phonetic contrasts that are more discriminative across languages than raw acoustic signals.

Time-Scale Modification (TSM)

To compensate for the lack of temporal data in short utterances, a phase vocoder-based TSM method is applied. This technique adjusts the speech rate without altering pitch or prosody.

Schematic of the Time-Scale Modification (TSM) process using STFT and phase modification.

For each utterance, two additional versions are generated: a slowed-down version ($\alpha=0.8$) and a sped-up version ($\alpha=1.2$). These are concatenated with the original signal to create a longer, more diverse input sequence for the classifier.

Spectrograms of the original speech compared with time-scaled versions ($\alpha=0.8$ and $\alpha=1.2$).

LSTM Classification

The concatenated BN features are fed into a two-layer Long Short-Term Memory (LSTM) network. LSTMs are well-suited for this task as they can model the long-term temporal dependencies and phonetic "flow" characteristic of different languages.

Experimental Results

The system was evaluated on the AP17-OLR database, covering 10 languages including Kazakh, Tibetan, Uyghur, Cantonese, Mandarin, Indonesian, Japanese, Russian, Korean, and Vietnamese. The training data consisted of approximately 10 hours of recording per language.

The proposed TSM-BN-LSTM model demonstrated superior performance, particularly on the most challenging short-duration subsets.

Model Test-All (EER %) Test-3s (EER %) Test-1s (EER %)
i-Vector (Baseline) 6.94 8.67 17.24
Standard LSTM 9.64 11.05 14.30
Proposed (TSM-BN-LSTM) 0.08 2.62 6.76

On 1-second clips, the proposed method reduced the Equal Error Rate (EER) from 17.24% (i-vector) to 6.76%. For full-length utterances, the EER reached a near-perfect 0.08%.

Discussion and Conclusion

The results highlight two key insights. First, phonetic features learned from one language (Mandarin) generalize effectively to others, suggesting a universal utility for bottleneck features in SLD. Second, artificial length extension via TSM provides the LSTM with a richer temporal context, significantly boosting robustness for short commands.

Future work will explore the impact of multilingual phonetic extractors and the system's robustness in high-noise automotive environments. The compact model size (~20MB) makes it a viable candidate for deployment on edge devices in intelligent vehicles.

Summary updated

Abstract

In this paper we proposed an end-to-end short utterances speech language identification(SLD) approach based on a Long Short Term Memory (LSTM) neural network which is special suitable for SLD application in intelligent vehicles. Features used for LSTM learning are generated by a transfer learning method. Bottle-neck features of a deep neural network (DNN) which are trained for mandarin acoustic-phonetic classification are used for LSTM training. In order to improve the SLD accuracy of short utterances a phase vocoder based time-scale modification(TSM) method is used to reduce and increase speech rated of the test utterance. By splicing the normal, speech rate reduced and increased utterances, we can extend length of test utterances so as to improved improved the performance of the SLD system. The experimental results on AP17-OLR database shows that the proposed methods can improve the performance of SLD, especially on short utterance with 1s and 3s durations.
BibTeX
@misc{ma2018language,
  author = {Zhanyu Ma and Hong Yu},
  title = {{Language Identification with Deep Bottleneck Features}},
  year = {2018},
  eprint = {1809.08909},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1809.08909},
}
Figure