Improved YOLOv8-Based Method for the Carapace Keypoint Detection and Size Measurement of Chinese Mitten Crabs

Ke Chen, Z. Chen, Changbo Wang, Zhifan Zhou, Maohua Xiao, Hong Zhu, Dongfang Li, Weimin LiuAnimals 2025 · DOI

TL;DR

This paper introduces an improved YOLOv8-based method for automatically measuring the carapace size of Chinese mitten crabs, a crucial metric for aquaculture. By integrating Swin Transformer into the YOLOv8l-pose architecture and refining the loss function, the system achieves high accuracy in keypoint detection and non-contact size measurement, offering a significant advancement over traditional manual methods.

What Problem Does This Paper Address?

The aquaculture of Chinese mitten crabs is a significant industry, but precise growth monitoring is hampered by traditional methods. Manually capturing and measuring crabs is time-consuming, labor-intensive, and can stress the animals. Furthermore, the complex underwater environment and the crabs' behavior make automated measurement challenging. Existing computer vision approaches often struggle with accurate keypoint detection and size estimation in such dynamic conditions. This paper aims to bridge this gap by developing an automated, non-contact system for carapace size measurement, which is a key indicator of crab growth and health.

Methods

The core of this work is an enhanced YOLOv8l-pose algorithm. The authors made two primary modifications:

  • Backbone Network Integration: They replaced the original YOLOv8 backbone with a Swin Transformer. This hierarchical attention mechanism is designed to efficiently capture global contextual information by processing images in shifted windows, leading to improved feature representation and reduced computational complexity.
  • Loss Function Refinement: The loss function was modified to better suit the keypoint detection task. Specifically, they replaced the CIoU loss with EIoU for bounding box regression and introduced a new keypoint loss function incorporating Object Keypoint Similarity (OKS) to accurately model spatial correlations between keypoints and improve detection accuracy.

The system then uses these detected keypoints, along with pre-calibrated background markers, to perform non-contact size measurements based on proportional relationships.

Datasets

The study utilized a dataset of 1256 images of Chinese mitten crabs. To address the small dataset size for deep learning, they employed data augmentation techniques, including scale resizing, flipping, and color transformation, expanding the dataset to 5024 images. The images were labeled using Labelme software, with six keypoints defined for each crab: left eye, right eye, and front, back, left, and right sides of the dorsal armor. The dataset was split into training (90%) and validation (10%) sets.

Key Insights and Ideas

  • Swin Transformer's Efficacy: Integrating Swin Transformer as the backbone significantly boosts feature extraction capabilities, especially for complex tasks like multi-keypoint detection on irregularly shaped objects like crab carapaces. This highlights the power of hierarchical attention for capturing both local details and global context.
  • Tailored Loss Functions Matter: Simply using a standard object detection loss function isn't enough for precise keypoint localization. The authors demonstrate that refining the loss function to specifically account for keypoint spatial relationships (via OKS) and improving bounding box regression (via EIoU) leads to more accurate and reliable keypoint detection.
  • Non-Contact Measurement is Feasible: By combining accurate keypoint detection with a known reference scale (the background plate), the system can achieve reliable size measurements without physically handling the crabs. This is a critical step towards intelligent aquaculture.

Key Results

The enhanced YOLOv8l-pose model demonstrated significant improvements over the baseline:

  • Object Detection: Achieved a mean average precision (mAP) of 95.88%, a 2.61% improvement over the baseline.
  • Keypoint Detection: Reached an overall Object Keypoint Similarity (OKS) of 91.32%, indicating high reliability in keypoint localization.
  • Size Measurement Accuracy: The system achieved a maximum dimensional measurement error of 4.8% and a mean absolute error of 2.34%, which the authors state meets practical aquaculture requirements.

Limitations and Future Work

The authors acknowledge that the proposed method is currently tied to specific image-capturing conditions and requires a dedicated data acquisition terminal. They also note that creating user-friendly application software based on this model is a future development. While the model shows good robustness and generalization ability across different crab postures and growth stages, further validation with larger and more diverse datasets in real-world aquaculture environments would be beneficial. Future work could focus on integrating this system into a comprehensive intelligent farming platform.

Summary updated

Abstract

The carapace size of the Chinese mitten crab (<i>Eriocheir sinensis</i>) is a vital indicator for assessing the growth performance of crabs. However, measuring the carapace sizes of Chinese mitten crabs remains challenging due to environmental complexity, species-specific behavioral patterns, and the current limitations of data acquisition methods characterized by labor-intensive manual measurements and subjective empirical judgments. Our study proposes an automated carapace dimension-measuring method integrating enhanced computer vision techniques to address the above challenges. Specifically, we used the YOLOv8 algorithm combined with the pose keypoint detection algorithm to process Chinese mitten crab images to acquire carapace sizes. We redesigned the YOLOv8l-pose architecture by incorporating Swin Transformer as the backbone network to improve feature representation for multikeypoint detection on crab carapaces, significantly enhancing global contextual feature extraction capabilities. Furthermore, we refined the loss function to model spatial correlations between keypoint locations accurately and thus improve detection accuracy for dorsal carapace dimension key points in Chinese mitten crabs. Our system enabled noncontact size measurement to leverage the proportional relationship between precalibrated background markers and detected carapace keypoints. Experimental results demonstrated that our enhanced model achieved a mean average precision of 95.88%, representing a 2.61% improvement over the baseline. The overall object keypoint similarity reached 91.32%, with maximum and mean dimensional measurement errors of 4.8% and 2.34%, respectively, validating our method's reliability for aquaculture applications.
BibTeX
@inproceedings{chen2025improved,
  author = {Ke Chen and Z. Chen and Changbo Wang and Zhifan Zhou and Maohua Xiao and Hong Zhu and Dongfang Li and Weimin Liu},
  title = {{Improved YOLOv8-Based Method for the Carapace Keypoint Detection and Size Measurement of Chinese Mitten Crabs}},
  booktitle = {Animals},
  year = {2025},
  doi = {10.3390/ani15070941},
}
Figure