Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook

Lei Jiao, Morten Goodwin, Kim Tallaksen Halvorsen, Lei Jiao 0001, Kristian Muri Knausgård, Angela Helen Martin, Marta Moyano, Rebekah A. Oomen, Jeppe Have Rasmussen, Tonje Knutsen Sørdalen, Susanna Huneide ThorbjørnsenarXiv 2021 · arXiv

Introduction: The Marine Data Deluge

Marine ecosystems are currently facing unprecedented stress from human activities, making Ecosystem-Based Management (EBM) more critical than ever. While modern technology—ranging from high-resolution underwater cameras to autonomous vehicles—allows for the collection of vast amounts of data, the field is currently facing a "data deluge." The primary bottleneck is manual processing; relying on human experts to count fish or identify whale calls is slow, expensive, and prone to subjective bias. This paper serves as a strategic bridge between marine ecology and computer science, demonstrating how deep learning (DL) can transform raw data into actionable ecological insights.

The Supervised Learning Workflow

To bridge the knowledge gap between ecologists and computer scientists, it is essential to understand the standard pipeline for developing AI models. Most applications in marine ecology rely on supervised learning, where a model learns from labeled examples. This process is typically divided into three distinct phases: training, validation, and testing.

The standard workflow for training, validating, and testing deep learning models in marine ecology.

In the Training phase, the AI is exposed to labeled data to learn patterns. During Validation, the model's performance is evaluated on a separate dataset to tune hyperparameters and prevent overfitting. Finally, Testing involves running the trained and validated AI on completely unseen data to assess its real-world accuracy and generalizability.

Deep Learning Architectures and Applications

The paper outlines several key Deep Neural Network (DNN) architectures tailored for specific ecological tasks. These range from simple classification to complex semantic segmentation.

  • Convolutional Neural Networks (CNNs): The gold standard for image-based tasks, such as identifying species or detecting objects like fishing gear.
  • Recurrent Neural Networks (RNNs) & Transformers: Specialized for time-series data, these are ideal for analyzing acoustic signals like fish drumming or whale vocalizations.
  • Siamese Neural Networks (SNNs): Used for "one-shot" learning, enabling the re-identification of individual organisms based on unique physical patterns.

The choice of architecture depends heavily on the specific case study and the nature of the input data, as summarized in the following table:

Mapping marine ecology case studies to specific deep learning tasks and architectures.

Key Results and Performance Benchmarks

Deep learning models have demonstrated the potential to significantly outperform manual human analysis in both speed and consistency. While human experts often achieve only 67–83% self-consistency when classifying difficult plankton samples, DL models can exceed 90% accuracy.

Specific performance highlights include:

  • Fish Identification: Squeeze-and-excitation CNNs achieved 99.27% accuracy on the Fish4Knowledge dataset.
  • Whale Detection: Models for detecting sperm whale clicks reached 99.5% accuracy, while Beluga whale call detection achieved an Area Under the Curve (AUC) of 0.9906.
  • Efficiency Gains: A task that would take a human analyst up to 350 days to complete can be processed by a trained neural network in approximately four days.

Challenges and Future Outlook

Despite these successes, the "Achilles' heel" of deep learning remains the requirement for high-quality labeled data. Creating these training sets still demands significant expert labor. Furthermore, models often struggle with "dataset shift" when moved to new locations with different lighting or water turbidity.

To overcome these hurdles, the authors advocate for a shift toward an Open Science Culture, where the marine science community shares models and labeled datasets (such as EcoTaxa and BIIGLE) to accelerate progress. Additionally, there is a pressing need for "biologist-coders"—a new generation of researchers capable of navigating both the nuances of marine biology and the complexities of neural network architecture. By synchronizing these fields, deep learning can enable real-time, global monitoring of our oceans' health.

Summary updated

Abstract

The deep learning revolution is touching all scientific disciplines and corners of our lives as a means of harnessing the power of big data. Marine ecology is no exception. These new methods provide analysis of data from sensors, cameras, and acoustic recorders, even in real time, in ways that are reproducible and rapid. Off-the-shelf algorithms can find, count, and classify species from digital images or video and detect cryptic patterns in noisy data. Using these opportunities requires collaboration across ecological and data science disciplines, which can be challenging to initiate. To facilitate these collaborations and promote the use of deep learning towards ecosystem-based management of the sea, this paper aims to bridge the gap between marine ecologists and computer scientists. We provide insight into popular deep learning approaches for ecological data analysis in plain language, focusing on the techniques of supervised learning with deep neural networks, and illustrate challenges and opportunities through established and emerging applications of deep learning to marine ecology. We use established and future-looking case studies on plankton, fishes, marine mammals, pollution, and nutrient cycling that involve object detection, classification, tracking, and segmentation of visualized data. We conclude with a broad outlook of the field's opportunities and challenges, including potential technological advances and issues with managing complex data sets.
BibTeX
@misc{jiao2021unlocking,
  author = {Lei Jiao and Morten Goodwin and Kim Tallaksen Halvorsen and Lei Jiao and Kristian Muri Knausgård and Angela Helen Martin and Marta Moyano and Rebekah A. Oomen and Jeppe Have Rasmussen and Tonje Knutsen Sørdalen and Susanna Huneide Thorbjørnsen},
  title = {{Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook}},
  year = {2021},
  eprint = {2109.14737},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.14737},
}
Figure