Image-Based Seal Recognition: Approaches and Challenges in Current Automated Systems

Jorge Yero, Renato Borras‐Chavez, Sarah S. Kienle, Pablo RivasCommunications 2025 · DOI · OpenAlex

Monitoring leopard seals (Hydrurga leptonyx) in their natural habitat is a significant logistical challenge. As apex predators residing in remote Antarctic regions, they are elusive and difficult to study. Traditionally, ecological monitoring has relied on researchers manually identifying individuals from vast photographic catalogs—a process that is labor-intensive, prone to human error, and fundamentally unscalable.

Recent advancements in computer vision (CV) offer a path forward, yet "wildlife in the wild" presents unique hurdles. Seals frequently change poses, blend into complex icy or rocky backgrounds (occlusion), and are subject to the volatile lighting conditions of the Antarctic. This paper synthesizes the current state-of-the-art in automated recognition, evaluating how modern machine learning architectures can bridge the gap between raw field data and actionable ecological insights.

Methodology: From Detection to Recognition

The research evaluates several cutting-edge pipelines across three primary tasks: object detection, image segmentation, and individual recognition.

1. Object Detection

The study contrasts YOLOv5, optimized for real-time speed and accuracy, with Detic, which offers a broader vocabulary for object recognition. These automated approaches aim to replace older CNN-based facial recognition models that required manual landmarking of features like eyes and noses.

2. Image Segmentation

Segmentation is critical for isolating the subject from its environment. The paper compares traditional unsupervised methods—which rely on superpixels and Support Vector Machines (SVMs)—against the Segment Anything Model (SAM). SAM, a "foundation model," utilizes a Vision Transformer (ViT) backbone to perform zero-shot segmentation on previously unseen objects.

Traditional unsupervised segmentation pipeline using superpixel classification.

3. Species and Individual Recognition

A core argument of the paper is the superiority of Vision Transformers (ViT) over traditional Convolutional Neural Networks (CNNs). While CNNs process images through local pixel neighborhoods, ViTs utilize "self-attention" mechanisms to analyze the entire image simultaneously. This allows the model to capture global relationships and subtle textures that are often lost in local processing.

Key Insights: The Power of Foundation Models

A pivotal finding in the study is the "prompting advantage" offered by SAM. While the model is highly capable, it can occasionally "over-segment" seals by focusing too closely on intricate skin patterns. However, by providing a simple bounding box or point prompt, researchers can guide the model to produce a singular, highly accurate mask of the animal.

The Segment Anything Model (SAM) architecture, utilizing image embeddings and prompt encoders for zero-shot segmentation.

Furthermore, the transition to ViTs addresses the issue of environmental variability. By treating image patches similarly to words in a sentence, ViTs better understand the context of a seal against complex backgrounds of ice and water. Given the scarcity of annotated leopard seal data, the authors emphasize transfer learning—fine-tuning models pre-trained on massive general datasets (like COCO or LVIS) to achieve high accuracy with limited species-specific labels.

Results and Future Directions

Qualitative results indicate that SAM provides more generalized and accurate segmentation than traditional superpixel methods. Additionally, ViTs demonstrate exceptional scalability, making them the preferred option for processing the large volumes of data generated by field research. The shift from manual landmarking to automated full-body detection via YOLOv5 represents a significant leap in research scalability.

However, several bottlenecks remain:

  • Data Scarcity: The lack of high-quality annotated wildlife data is a primary challenge. Future work may involve using Generative Adversarial Networks (GANs) or Diffusion Models to create synthetic training data.
  • Real-time Processing: Heavy encoders like those in SAM are computationally expensive. Optimizing these models for deployment on field hardware, such as drones, is a priority.
  • Pose and Occlusion: Because seals are highly flexible, developing models that integrate pose estimation with feature extraction will be essential for improving individual identification accuracy.

Summary updated

BibTeX
@inproceedings{yero2025image,
  author = {Jorge Yero and Renato Borras‐Chavez and Sarah S. Kienle and Pablo Rivas},
  title = {{Image-Based Seal Recognition: Approaches and Challenges in Current Automated Systems}},
  booktitle = {Communications},
  year = {2025},
  doi = {10.1007/978-3-031-85933-5\_3},
}

Loading related content...

Figure