The DKU-LENOVO Systems for the INTERSPEECH 2019 Computational Paralinguistic Challenge

Haiwei Wu, Weiqing Wang, Ming LiInterspeech 2019 · DOI

Abstract

This paper presents the DKU-LENOVO submission for the INTERSPEECH 2019 Computational Paralinguistic Challenge (ComParE), focusing on the Orca Activity and Continuous Sleepiness sub-challenges. The proposed system utilizes deep convolutional neural networks (CNNs) for feature extraction, combined with Support Vector Machine (SVM) classifiers and Fisher Vector (FV) encoding. By leveraging robust data augmentation and multi-channel fusion strategies, the systems significantly outperformed the official baselines, achieving state-of-the-art performance in both tasks.

Introduction

Computational paralinguistics involves the automatic extraction of non-verbal information from audio signals, such as emotional states, health conditions, or specific animal vocalizations. This study addresses two tasks from the 2019 ComParE Challenge: Orca Activity Detection, a binary classification task in noisy underwater environments, and Continuous Sleepiness Estimation, a regression task based on human speech. The primary hurdles in these tasks include significant data imbalance and limited dataset sizes, which often lead to model overfitting.

Methodology

The authors developed two distinct strategies tailored to the specific requirements of each task.

Deep Embedding System

The primary system for the Orca task utilizes a deep embedding approach rather than a standard end-to-end CNN. Audio signals are first converted into log mel-spectrograms. A Global Average Pooling (GAP) layer is integrated into the CNN architectures (ResNet, Inception, and DenseNet) to handle variable-length audio clips without requiring resizing.

Instead of using the final softmax layer for classification, the system extracts high-level features (embeddings) from the penultimate layer. These embeddings are then fed into a Support Vector Machine (SVM) classifier. This hybrid structure leverages the powerful feature extraction of deep learning while utilizing the SVM's regularization properties to prevent overfitting on small datasets.

System architecture for the Orca activity detection task, showing the pipeline from raw utterances to final classification.

Fisher Vector (FV) Encoding

For the sleepiness task, the authors employed Fisher Vector encoding to represent various acoustic features such as MFCCs and LFCCs. FV encoding captures first- and second-order statistics relative to a Gaussian Mixture Model (GMM), providing a stable representation that is often more effective for smaller paralinguistic datasets than deep learning alone.

Data Augmentation and Fusion

To address data scarcity in the Orca task, four augmentation techniques were applied: adding noise to orca samples, mixing orca samples, mixing noise samples, and speed perturbation. Furthermore, since the orca data was recorded via multiple hydrophones, multi-channel information was fused at the embedding level by averaging features from different channels, providing an optimal balance between performance and efficiency.

Experimental Results

The proposed systems achieved substantial performance gains over the official challenge baselines. In the Orca Activity task, the DenseNet-based deep embedding system reached an AUC of 0.948 on the test set, a significant improvement over the 0.866 baseline. For Continuous Sleepiness, the fusion of the baseline system with FV encoding achieved a Spearman’s correlation of 0.365.

Performance comparison of the proposed systems against official baselines for both Orca Activity and Continuous Sleepiness tasks.

Key Insights

  • SVM as a Robust Back-end: A critical finding was that SVMs outperformed fully connected layers as classifiers for deep embeddings. The SVM acts as an effective regularizer, mitigating the tendency of deep models to overfit small training sets.
  • Effectiveness of Log Mel-Spectrograms: While standard spectrograms are common in bioacoustics, log mel-spectrograms proved superior for orca calls. The structural similarities between animal vocalizations and human speech make mel-filters surprisingly effective.
  • Target-Specific Augmentation: Increasing the variety of the target signal (e.g., Orca + Orca mixing) was found to be more beneficial than simply adding noise, highlighting the importance of signal diversity in detection tasks.

Conclusion and Future Work

The DKU-LENOVO systems demonstrate the effectiveness of combining deep feature extraction with robust classical classifiers and strategic data augmentation. While highly successful, the authors note that multi-channel fusion increases computational complexity. Future research will focus on more efficient fusion strategies and advanced augmentation techniques to further improve performance in high-noise environments.

Summary updated

BibTeX
@inproceedings{wu2019dku,
  author = {Haiwei Wu and Weiqing Wang and Ming Li},
  title = {{The DKU-LENOVO Systems for the INTERSPEECH 2019 Computational Paralinguistic Challenge}},
  booktitle = {Interspeech},
  year = {2019},
  doi = {10.21437/interspeech.2019-1386},
}
Figure