Deep learning‐based seabird detection in fisheries for seabird protection

J. C. Y. Leong, Junhong Zhao, Bing Xue, William Gibson, Mengjie ZhangJournal of the Royal Society of New Zealand 2025 · DOI

TL;DR

This paper tackles the critical issue of seabird bycatch in New Zealand fisheries by developing a deep learning model to automatically detect seabirds in real-world fishing footage. Their fine-tuned YOLOv8 model achieves impressive accuracy (mAP@50 of 0.9926), paving the way for automated monitoring and conservation efforts.

What Problem Does This Paper Address?

New Zealand is a global hotspot for seabirds, but commercial fishing operations unfortunately lead to accidental bycatch, harming these populations and impacting their long-term sustainability. Manually reviewing vast amounts of video footage from fishing vessels to identify these interactions is impractical and time-consuming. This paper aims to automate this process by creating a robust object detection system capable of identifying seabirds in the challenging, unconstrained environment of active fishing operations.

Methods

The researchers employed a deep learning approach using the YOLO (You Only Look Once) object detection framework, specifically focusing on YOLOv8 and YOLOv9 architectures. Their methodology involved:

  • Dataset Creation: A new dataset was curated from video footage provided by Fisheries New Zealand, featuring raw still images from onboard cameras. This dataset was manually annotated with bounding boxes around seabirds.
  • Transfer Learning: Pre-trained YOLO models (YOLOv8 and YOLOv9) were fine-tuned on the newly created seabird dataset. This process adapts the models, originally trained on general object detection (COCO dataset), to the specific task of seabird identification.
  • Hyperparameter Tuning: A grid search method was used to systematically tune key hyperparameters (learning rate and momentum) for the top-performing YOLO models to optimize their performance.
  • Model Selection: The best-performing YOLOv8 and YOLOv9 models were identified and compared.

Datasets

  • FNZ Dataset: A newly created dataset comprising 8402 annotated images extracted from video footage captured by onboard cameras on commercial fishing vessels. This dataset is unique as it represents real-world marine interactions, unlike many existing datasets focused on controlled environments.
  • COCO 2017 Dataset: Used for pre-training the YOLOv8 and YOLOv9 models. This dataset contains images from 80 different object classes.

Key Insights and Ideas

  • Real-world Marine Scenarios are Crucial: The paper highlights that models trained on controlled datasets struggle in the dynamic and unconstrained environment of fishing vessels. Their FNZ dataset, capturing these real-world conditions, is key to developing a practical solution.
  • YOLOv8's Efficiency: The YOLOv8 architecture, particularly the "large" variant, demonstrated a strong balance between performance and computational efficiency, making it a suitable choice for potential real-time applications.
  • Fine-tuning is Essential: Simply using pre-trained models on the FNZ dataset was insufficient. Transfer learning and subsequent hyperparameter tuning were critical to achieving high accuracy.
  • mAP@50-95 as a Robust Metric: The paper emphasizes the importance of the mAP@50-95 metric, which considers a range of Intersection over Union (IoU) thresholds, for evaluating performance in more challenging detection scenarios.

Key Results

The fine-tuned YOLOv8 large model achieved state-of-the-art performance on the FNZ test set:

  • mAP@50: 0.9926 (exceeding their 90% target)
  • mAP@50-95: 0.9147
  • The model demonstrated effectiveness in detecting seabirds at varying distances and in different lighting conditions.
  • A basic interactive dashboard was developed to visualize detection results, serving as a proof-of-concept for real-world application.

Limitations and Future Work

  • Model Uncertainty: The current model struggles with detecting smaller seabirds further away from the camera, especially under poor lighting conditions.
  • Dataset Diversity: The FNZ dataset lacks diversity in terms of time of day, weather conditions, and flock sizes, which limits the model's robustness.
  • Single-Class Detection: The model is currently limited to detecting a single class ("Seabird") and cannot distinguish between different species.

Future work suggested includes:

  • Modifying the YOLO architecture to better detect smaller seabirds and improve performance under various weather and lighting conditions.
  • Expanding the FNZ dataset to include more diverse conditions and different seabird species.
  • Developing a multi-species seabird detection model.
  • Integrating object tracking algorithms (like ByteTrack or DeepSORT) to track and count individual seabirds in video clips.

Summary updated

Abstract

New Zealand is considered to be the 'seabird capital' of the world. As part of the harvesting process, some commercial fishers accidentally bycatch seabirds during fishing operations, which can result in accidental deaths and injuries. The accidental bycatch is impacting the long-term sustainability of New Zealand seabird populations. To address this, we developed a YOLO model that can be used to automatically detect seabirds that interact with the fishing vessels. The model development process involved gathering, annotating and preprocessing a new image dataset, conducting transfer learning across YOLO benchmark models, and performing hyperparameter tuning on the top YOLO models to further improve the model's performance. We evaluate the performance and effectiveness of our developed model under diverse data conditions, with it achieving a mAP@50 score of 0.9926 and a mAP@50-95 score of 0.9147 on the test data. The results demonstrate that the developed model performs effectively in unconstrained real-world marine scenarios, addressing the limitations of previous models primarily evaluated in controlled settings. This automation could help to reduce or even eliminate manual inspection of footages by reviewers and will help to quantify seabird interactions with commercial fishing vessels. Our contributions represent a significant first step in automated seabird detection, mitigating the gap between constrained and unconstrained real-world maritime scenarios.
BibTeX
@article{leong2025deep,
  author = {J. C. Y. Leong and Junhong Zhao and Bing Xue and William Gibson and Mengjie Zhang},
  title = {{Deep learning‐based seabird detection in fisheries for seabird protection}},
  journal = {Journal of the Royal Society of New Zealand},
  year = {2025},
  doi = {10.1080/03036758.2025.2500998},
}
Figure