Automated detection of dolphin whistles with convolutional networks and transfer learning

Burla Nur Korkmaz, Roee Diamant, Gil Danino, Alberto Testolin

TL;DR

This paper introduces a deep learning approach using Convolutional Neural Networks (CNNs) for automated detection of dolphin whistles in underwater audio recordings. By leveraging transfer learning with a pre-trained VGG16 model, they achieve significantly higher accuracy than traditional methods, paving the way for more efficient and scalable marine ecosystem monitoring.

What Problem Does This Paper Address?

Effective conservation of marine environments and management of endangered species hinges on accurate and scalable methods for monitoring biodiversity. Ecoacoustics, using passive acoustic sensors like hydrophones, offers a non-invasive way to continuously sample underwater sounds. However, analyzing these vast amounts of acoustic data is a time-consuming process that traditionally requires extensive human supervision. This manual effort limits the feasibility of long-term, large-scale surveys, especially in noisy underwater environments. The paper specifically targets the challenge of reliably detecting dolphin whistles, which are crucial for communication, foraging, and navigation in species like bottlenose dolphins, amidst significant background noise.

Methods

The researchers explored two main deep learning architectures:

  • Vanilla CNN: A custom-built CNN with two convolutional layers, max pooling, dropout, and fully connected layers. This served as a baseline deep learning model.
  • Transfer Learning with VGG16: They utilized a pre-trained VGG16 architecture (originally designed for image recognition) and replaced its top layers with new fully connected layers. This approach leverages the feature extraction capabilities learned from a massive image dataset and fine-tunes them for the acoustic whistle detection task.

Both models were trained using the Adam optimizer and binary cross-entropy loss. Overfitting was managed with an early stopping criterion. For comparison, they also implemented a traditional method using PamGuard, a popular software for automated marine mammal vocalization detection.

Datasets

The study created a large-scale dataset of underwater sound recordings collected over 27 days in Eilat, Israel, using a custom-built acoustic recorder deployed at a depth of 50 meters.

  • Data Collection: Recordings were made using Geospectrum M18 hydrophones.
  • Pre-processing: Audio data underwent bandpass filtering (5-20 kHz), whitening, averaging of channels, and outlier removal based on signal length. Spectrograms were generated using Short-Time Fourier Transform (STFT).
  • Manual Tagging: Spectrograms were manually tagged by a human expert in two phases: initial marking to train a preliminary classifier and subsequent validation tagging. This process identified whistles and marked their contours.
  • Dataset Split: The dataset was split into training and testing sets based on recording dates to ensure robust evaluation of generalization performance under varying sea conditions. The training set contained approximately 108,000 spectrograms (58,510 whistles, 49,807 noise), and the test set contained about 6,800 spectrograms (2,657 whistles, 4,212 noise). 5-fold cross-validation was used on the training set.

Key Insights and Ideas

  • Deep Learning's Superiority: Deep learning models, particularly those leveraging transfer learning, significantly outperform traditional methods like PamGuard in detecting dolphin whistles. This highlights the power of modern AI for complex acoustic analysis.
  • Transfer Learning Advantage: Using a pre-trained VGG16 model for transfer learning yielded the most impressive results. This suggests that features learned from general image recognition tasks can be effectively adapted to acoustic signal processing, reducing the need for massive, task-specific datasets from scratch.
  • Handling Noise: The deep learning models demonstrated a remarkable ability to reduce false positives and false negatives, even in the presence of significant ambient noise, a critical challenge for underwater acoustic monitoring.
  • Scalability Potential: The success of these automated methods points towards the development of cost-effective and scalable platforms for continuous marine ecosystem monitoring, a crucial step for conservation efforts.

Key Results

  • Vanilla CNN: Achieved a mean detection accuracy of 80.6%.
  • PamGuard (Baseline): Achieved a mean detection accuracy of 66.4%.
  • VGG16 with Transfer Learning: Achieved a highly impressive mean detection accuracy of 92.3%.
  • Confusion Matrices: The VGG model showed a significantly higher number of true positives compared to both the vanilla CNN and PamGuard. PamGuard exhibited a particularly high number of false negatives, indicating its limitations in challenging detection scenarios.
  • ROC Curves: The VGG architecture's ROC curve approached the performance of an ideal classifier, demonstrating its superior ability to balance true positive and false positive rates.

Limitations and Future Work

  • Generalization to Other Environments/Species: The authors acknowledge that further research is needed to validate these methods in different marine environments and with different animal species.
  • Further Research: They believe that deep learning will enable the creation and deployment of cost-effective monitoring platforms, implying future work in optimizing these systems for real-world deployment.
  • Dataset Availability: The complete dataset of dolphin recordings has been made publicly available, encouraging further research and development in this area.

Summary updated

Abstract

Effective conservation of maritime environments and wildlife management of endangered species require the implementation of efficient, accurate and scalable solutions for environmental monitoring. Ecoacoustics offers the advantages of non-invasive, long-duration sampling of environmental sounds and has the potential to become the reference tool for biodiversity surveying. However, the analysis and interpretation of acoustic data is a time-consuming process that often requires a great amount of human supervision. This issue might be tackled by exploiting modern techniques for automatic audio signal analysis, which have recently achieved impressive performance thanks to the advances in deep learning research. In this paper we show that convolutional neural networks can indeed significantly outperform traditional automatic methods in a challenging detection task: identification of dolphin whistles from underwater audio recordings. The proposed system can detect signals even in the presence of ambient noise, at the same time consistently reducing the likelihood of producing false positives and false negatives. Our results further support the adoption of artificial intelligence technology to improve the automatic monitoring of marine ecosystems.
BibTeX
@article{korkmaz2023automated,
  author = {Burla Nur Korkmaz and Roee Diamant and Gil Danino and Alberto Testolin},
  title = {{Automated detection of dolphin whistles with convolutional networks and transfer learning}},
  journal = {Frontiers Artif. Intell},
  year = {2023},
  doi = {10.3389/frai.2023.1099022},
}
Figure