Detection of Marine Animals in a New Underwater Dataset with Varying Visibility

Malte Pedersen, Joakim Bruslund Haurum, Rikke Gade, Thomas B. MoeslundWorkshops @ CVPR 2019 · OpenReview

Abstract

Automated monitoring of marine life in temperate coastal regions is often hindered by high turbidity and variable lighting. This paper presents the Brackish Dataset, the first publicly available annotated underwater image collection specifically targeting these challenging environments. By fine-tuning YOLOv2 and YOLOv3 architectures, we establish performance baselines for six marine species, demonstrating that while large or slow-moving organisms like starfish are easily detected, small and fast-moving species remain a significant computer vision challenge in murky waters.

Introduction

Most underwater computer vision research has historically focused on clear-water environments, such as tropical coral reefs or the deep sea. However, temperate coastal areas—often characterized by "brackish" water—are critical for biodiversity and sustainable fisheries management. These regions suffer from high turbidity caused by suspended particles, which leads to significant light scattering and absorption. Standard object detection models trained on high-visibility datasets frequently fail when applied to these noisy, real-world conditions. This study addresses this gap by providing a specialized dataset and evaluating state-of-the-art detectors in low-visibility scenarios.

Experimental Setup

To capture realistic data, a custom-built rig was deployed in Limfjorden, Denmark. The setup consisted of three cameras and three LED lights mounted on a bridge pillar, approximately nine meters below the water surface.

Deployment of the camera setup on a bridge pillar in Limfjorden, Denmark.

The data was collected over several weeks to ensure a wide range of natural variations in turbidity and lighting conditions, reflecting the dynamic nature of coastal ecosystems.

The Brackish Dataset

The core contribution of this work is the Brackish Dataset, which comprises 14,518 frames and 25,613 manual annotations. The dataset includes six distinct classes: big fish, small fish, crabs, shrimp, jellyfish, and starfish.

Examples of the six marine animal classes in the Brackish Dataset under varying visibility conditions.

Unlike existing datasets such as F4K or HabCam, the Brackish Dataset focuses specifically on temperate, variable-visibility environments. The varying concentrations of dissolved matter in the water act as a form of natural image noise, requiring models to learn robust feature representations that are invariant to these distortions.

Methodology and Results

We evaluated two "single-shot" object detectors: YOLOv2 (pre-trained on fish-specific data) and YOLOv3 (pre-trained on Open Images). Both models were fine-tuned using the Darknet framework. Performance was measured using Average Precision (AP) and $AP_{50}$.

The fine-tuned YOLOv3 model emerged as the superior detector, achieving an overall $AP_{50}$ of approximately 84%. Key findings include:

  • The "Starfish Advantage": Starfish were detected with near-perfect accuracy ($AP_{50}$ of 98.7%), likely due to their distinct shape and slow movement, which provides consistent training samples.
  • The Speed-Size Penalty: Small fish ($AP_{50}$ of 62.3%) and shrimp proved the most difficult to detect. Their small size and rapid movement often result in motion blur, which, when combined with turbidity, obscures critical identification features.
  • Domain Adaptation: Models trained on general datasets performed poorly until fine-tuned on the brackish data, highlighting the necessity of domain-specific training for underwater applications.

Conclusion and Future Work

The Brackish Dataset provides a vital resource for developing robust marine monitoring systems. While current models are highly reliable for larger organisms, improving the detection of small, fast-moving species in murky water remains an open challenge. Future work will focus on utilizing uncompressed video data to preserve fine details and exploring stereo vision setups to incorporate depth information for more accurate species classification.

The dataset is publicly available on Kaggle to encourage further research in underwater computer vision.

Summary updated

BibTeX
@inproceedings{pedersen2019detection,
  author = {Malte Pedersen and Joakim Bruslund Haurum and Rikke Gade and Thomas B. Moeslund},
  title = {{Detection of Marine Animals in a New Underwater Dataset with Varying Visibility}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=izv1Ro1QPpd},
}
Figure