Abstract
This paper presents the DKU-LENOVO submission for the INTERSPEECH 2019 Computational Paralinguistic Challenge (ComParE), focusing on the Orca Activity and Continuous Sleepiness sub-challenges. The proposed system utilizes deep convolutional neural networks (CNNs) for feature extraction, combined with Support Vector Machine (SVM) classifiers and Fisher Vector (FV) encoding. By leveraging robust data augmentation and multi-channel fusion strategies, the systems significantly outperformed the official baselines, achieving state-of-the-art performance in both tasks.
Introduction
Computational paralinguistics involves the automatic extraction of non-verbal information from audio signals, such as emotional states, health conditions, or specific animal vocalizations. This study addresses two tasks from the 2019 ComParE Challenge: Orca Activity Detection, a binary classification task in noisy underwater environments, and Continuous Sleepiness Estimation, a regression task based on human speech. The primary hurdles in these tasks include significant data imbalance and limited dataset sizes, which often lead to model overfitting.
Methodology
The authors developed two distinct strategies tailored to the specific requirements of each task.
Deep Embedding System
The primary system for the Orca task utilizes a deep embedding approach rather than a standard end-to-end CNN. Audio signals are first converted into log mel-spectrograms. A Global Average Pooling (GAP) layer is integrated into the CNN architectures (ResNet, Inception, and DenseNet) to handle variable-length audio clips without requiring resizing.
Instead of using the final softmax layer for classification, the system extracts high-level features (embeddings) from the penultimate layer. These embeddings are then fed into a Support Vector Machine (SVM) classifier. This hybrid structure leverages the powerful feature extraction of deep learning while utilizing the SVM's regularization properties to prevent overfitting on small datasets.
Fisher Vector (FV) Encoding
For the sleepiness task, the authors employed Fisher Vector encoding to represent various acoustic features such as MFCCs and LFCCs. FV encoding captures first- and second-order statistics relative to a Gaussian Mixture Model (GMM), providing a stable representation that is often more effective for smaller paralinguistic datasets than deep learning alone.
Data Augmentation and Fusion
To address data scarcity in the Orca task, four augmentation techniques were applied: adding noise to orca samples, mixing orca samples, mixing noise samples, and speed perturbation. Furthermore, since the orca data was recorded via multiple hydrophones, multi-channel information was fused at the embedding level by averaging features from different channels, providing an optimal balance between performance and efficiency.
Experimental Results
The proposed systems achieved substantial performance gains over the official challenge baselines. In the Orca Activity task, the DenseNet-based deep embedding system reached an AUC of 0.948 on the test set, a significant improvement over the 0.866 baseline. For Continuous Sleepiness, the fusion of the baseline system with FV encoding achieved a Spearman’s correlation of 0.365.
Key Insights
- SVM as a Robust Back-end: A critical finding was that SVMs outperformed fully connected layers as classifiers for deep embeddings. The SVM acts as an effective regularizer, mitigating the tendency of deep models to overfit small training sets.
- Effectiveness of Log Mel-Spectrograms: While standard spectrograms are common in bioacoustics, log mel-spectrograms proved superior for orca calls. The structural similarities between animal vocalizations and human speech make mel-filters surprisingly effective.
- Target-Specific Augmentation: Increasing the variety of the target signal (e.g., Orca + Orca mixing) was found to be more beneficial than simply adding noise, highlighting the importance of signal diversity in detection tasks.
Conclusion and Future Work
The DKU-LENOVO systems demonstrate the effectiveness of combining deep feature extraction with robust classical classifiers and strategic data augmentation. While highly successful, the authors note that multi-channel fusion increases computational complexity. Future research will focus on more efficient fusion strategies and advanced augmentation techniques to further improve performance in high-noise environments.
Summary updated