Abstract
Image processing and analysis techniques have drawn increasing attention since they enable a non-extractive and non-lethal approach to collecting fisheries data, such as fish size measurement, catch estimation, regulatory compliance, species recognition, and population counting. Measuring fish size accurately requires reliable image segmentation. Major challenges that can easily affect the segmentation include blurring of image areas due to water drops on the camera lens and parts of a fish body being out of the camera view. In this paper, we address each of these issues with an innovative and effective contour-based segmentation and a missing shape recovery method from an arbitrary initial segmentation. The refinement is processed from the coarse level to the fine level. At the coarse level, we align the entire fish contour of the initial segmentation with trained representative contours by using iteratively reweighted least squares (IRLS). At finer levels, we iteratively refine contour segments to represent poorly segmented or missing shape parts. This method addresses the problems listed above and generates promising results with highly robust segmentation performance and length measurement.
Introduction
Automated fisheries monitoring is essential for sustainable management, aiming to replace labor-intensive manual measurements with efficient electronic monitoring. Systems like the "Camera Chute" are designed to capture images of fish as they pass through a controlled environment on fishing vessels. However, the maritime environment presents significant challenges for standard image segmentation algorithms. Specifically, water splashes on the camera lens create localized blur, and fish often trigger sensors while only partially within the camera's field of view.
Standard deep learning models, such as Fully Convolutional Networks (FCNs), often struggle with these artifacts, producing "blobby" results or failing to reconstruct missing anatomical features. This research proposes a multi-scale contour-based refinement method to enforce anatomical constraints and recover missing shapes.
Methodology
The proposed approach utilizes a coarse-to-fine strategy to refine initial segmentations.
Shape Modeling and Alignment
The system identifies 20 representative contours from training data using k-means clustering to capture common poses. Seven key landmarks (e.g., mouth, tail tips) are defined, and a linear regression model is trained to predict missing landmark locations based on available ones. For coarse alignment, the system uses Iteratively Reweighted Least Squares (IRLS) to fit the initial segmentation to the best representative contour. IRLS is particularly effective here as it treats segmentation errors caused by water drops as outliers, assigning them lower weights and relying on the global shape prior to bridge gaps.
Fine Refinement and Replacement
The contour is divided into segments between landmarks. These segments are iteratively adjusted to align with local image gradients. To ensure realistic results, generated segments are replaced with the most similar actual segments from the training database, recovering realistic textures and curves.
Dataset and Evaluation
The study used a dataset of 583 manually labeled segmentation masks of flat fish (halibut and flounder). 489 masks were used for training shape models, and 94 for testing. To evaluate "missing shape recovery," the authors simulated challenges by randomly removing approximately one-third of the fish body in test images.
Results and Discussion
The refinement method significantly improved the performance of various initial segmentation techniques. For Double Local Thresholding (DLT), the Average Intersection over Union (IoU) rose from 89.65% to 95.59%. For FCN-based segmentation, it improved from 92.35% to 95.47%.
Even when 33% of the fish body was missing, the system maintained an average IoU of nearly 90% and kept mean length errors between 3.9% and 4.6%. This robustness stems from the integration of anatomical constraints; unlike pixel-wise classification, this method "knows" the geometry of a fish, allowing it to estimate where a missing tail should be based on the rest of the body.
Conclusion
This contour-based refinement provides a robust solution for fisheries monitoring in harsh environments. While currently optimized for flat fish, future work will involve expanding the library of representative contours to accommodate more diverse species. Additionally, integrating a species classifier could allow the system to select species-specific shape models, further enhancing precision and measurement accuracy.
Summary updated