TL;DR
This paper proposes an improved speech/music classification method for the 3GPP EVS codec by integrating Long Short-Term Memory (LSTM) networks. By leveraging LSTM's ability to handle sequential data and using features extracted from the EVS codec, the proposed method achieves better classification accuracy, especially in noisy conditions with low signal-to-noise ratios, compared to the conventional EVS approach.
What Problem Does This Paper Address?
In modern communication systems, especially those supporting Voice over LTE (VoLTE), efficiently processing audio streams is crucial. A key component of this is accurately distinguishing between speech and music. This distinction allows for optimized audio coding, leading to better sound quality and more efficient use of bandwidth. The 3GPP Enhanced Voice Services (EVS) codec, a standard for VoLTE, uses a Gaussian Mixture Model (GMM) for speech/music classification. However, GMMs can struggle with the inherent diversity and complexity of music data, and their performance can degrade significantly in noisy environments, particularly at low signal-to-noise ratios (SNRs). This paper aims to address this limitation by proposing a more robust speech/music classification technique for the EVS codec.
Methods
The core of the proposed method is the integration of Long Short-Term Memory (LSTM) networks into the EVS speech/music classification framework. LSTMs are a type of recurrent neural network (RNN) specifically designed to learn long-term dependencies in sequential data, making them well-suited for audio signals where temporal correlations are important.
The authors leverage the existing feature extraction mechanisms within the EVS codec, which already compute 68 parameters. They then select a subset of 12 discriminative features, similar to the conventional EVS approach. These features are fed into an LSTM network. The LSTM is trained on a large dataset of speech and music.
The paper contrasts their proposed LSTM-based approach with two baselines:
- Conventional EVS: This method uses a GMM with context-based post-processing to classify speech/music.
- RNN-based: This is a baseline using a standard Recurrent Neural Network (RNN) for comparison.
The LSTM architecture employed consists of an input activation function, a memory cell, and three gates (input, forget, and output) to control information flow and mitigate the vanishing gradient problem common in traditional RNNs.
Datasets
The experiments were conducted using a combination of publicly available and collected datasets:
- TIMIT speech database: A standard dataset for speech research.
- Billboard year-end CDs (1980-2013): Used to collect a diverse range of music.
- Classical music CDs: Further expanding the music data variety.
- NOISEX-92 database: Used to generate noisy audio by adding various noise types (babble, car, white, pink, factory1, factory2) to clean speech data at different SNRs.
For training, 60 hours of speech and 160 hours of music were used. Testing involved 20 hours of speech and 61 hours of music, all sampled at 16 kHz with a 20 ms frame size.
Key Insights and Ideas
- LSTM's Superiority for Sequential Audio Data: The paper strongly suggests that LSTMs, with their inherent ability to model temporal dependencies, are a more effective choice for speech/music classification than GMMs, especially when dealing with the complex and time-varying nature of music.
- Leveraging Existing EVS Features: By utilizing the feature extraction already present in the EVS codec, the proposed LSTM approach avoids the need for entirely new feature engineering, simplifying integration and reducing computational overhead.
- Robustness in Low SNR: A critical finding is the significant improvement in classification accuracy at low SNRs. This indicates that the LSTM-based method is more resilient to noise, a common challenge in real-world audio processing.
- "Big Data" for Music Diversity: The authors emphasize the use of a large and diverse music dataset for learning, which is crucial for overcoming the inherent variability and complexity of music across different genres.
Key Results
The proposed LSTM-based speech/music classification method demonstrated superior performance compared to the conventional EVS and RNN-based methods across various conditions.
- Overall Accuracy: The LSTM method consistently achieved higher accuracy.
- Low SNR Performance: In noisy environments, particularly at 5 dB SNR, the LSTM method showed substantial improvements. For instance, in pink noise at 5 dB SNR, the accuracy increased from 0.3752 (EVS) to 0.6312 (Proposed LSTM).
- Comparison with RNN: The LSTM consistently outperformed the standard RNN across all tested conditions, highlighting the benefit of its specialized architecture for this task.
- Clean Conditions: Even in clean conditions, the LSTM achieved a slightly higher accuracy (0.9985) compared to EVS (0.9980) and RNN (0.9981).
Limitations and Future Work
- Computational Complexity: While the paper focuses on improving accuracy, the computational cost of LSTMs compared to GMMs is not explicitly detailed, which could be a consideration for real-time embedded applications.
- Feature Set Optimization: The paper uses the 12 features from the conventional EVS. Future work could explore if a different or expanded set of features might further enhance LSTM performance.
- Real-World Deployment: The experiments were conducted under simulated noisy conditions. Testing on real-world, unscripted noisy audio would be a valuable next step.
The authors' primary contribution is demonstrating the effectiveness of integrating LSTMs into existing speech/music classification frameworks for enhanced robustness, particularly in challenging acoustic environments.
Summary updated