Identification of Large Yellow Croaker under Variable Conditions Based on the Cycle Generative Adversarial Network and Transfer Learning

Shijing Liu, Cheng Qian, Xueying Tu, Haojun Zheng, Lin Zhu, Huang Liu, Jun ChenJournal of Marine Science and Engineering 2023 · DOI

TL;DR

This paper tackles the challenge of identifying large yellow croakers in diverse aquatic environments by proposing a novel approach that combines CycleGAN for domain adaptation and transfer learning for improved recognition. Their method effectively bridges the gap between controlled lab settings and real-world aquaculture, achieving high identification accuracy.

What Problem Does This Paper Address?

Accurately identifying fish, particularly commercially valuable species like the large yellow croaker, is crucial for advancements in aquaculture and genetic breeding. However, a significant hurdle is the "domain shift" problem: images captured under different conditions (e.g., varying lighting, water clarity, camera angles, and farming environments) have distinct visual characteristics. This makes it difficult to train a single model that performs well across all scenarios, as data collected in one environment often doesn't generalize effectively to another. Existing methods struggle with this variability, limiting their practical application in real-world fish farming.

Methods

The core of the proposed solution lies in a two-pronged approach:

  • CycleGAN for Domain Adaptation: The authors leverage CycleGAN, a powerful image-to-image translation framework, to bridge the domain gap between a "source domain" (controlled breeding scenarios) and a "target domain" (actual farming conditions). CycleGAN learns to translate images from one domain to another without requiring paired examples.
  • Enhanced CycleGAN with Identity and Distribution Losses: To improve the quality and relevance of the translated images, they introduce two key modifications to the standard CycleGAN:
  • Identity Foreground Loss (IDF): This loss function ensures that the identity of the fish is preserved during the translation process. It focuses on the foreground (the fish itself) and minimizes noise or background interference that could lead to misidentification.
  • Maximum Mean Discrepancy (MMD): MMD is employed to minimize the statistical difference between the distributions of the source and target domains. This helps to make the translated images more representative of the target domain's characteristics.
  • Transfer Learning: After augmenting the target domain dataset with CycleGAN-generated images, the authors apply transfer learning. They pre-train a VGG-16 model on the CIFAR-10 dataset and then fine-tune it using the combined original and augmented data. This leverages existing knowledge and adapts it to the specific task of large yellow croaker identification.

Datasets

The study utilizes two primary datasets:

  • Source Domain Dataset: Collected from a recirculating aquaculture system under controlled conditions. It comprises 600 images of large yellow croakers, with 480 for training and 120 for testing.
  • Target Domain Dataset: Captured from an actual aquaculture ship ("Guoxin 1"). This dataset represents the challenging real-world farming environment and includes approximately 10,000 large yellow croakers. The paper uses 300 images from this, with 240 for training and 60 for testing.

The CIFAR-10 dataset was used for pre-training the VGG-16 model in the transfer learning phase.

Key Insights and Ideas

  • Domain Adaptation is Key for Variable Environments: The paper strongly demonstrates that simply training on one domain and testing on another is insufficient. Effective domain adaptation techniques are crucial for achieving robust performance in real-world scenarios with inherent variability.
  • Preserving Identity is Paramount: Standard image translation might alter crucial identity features. The introduction of IDF loss is a critical insight, showing that explicitly enforcing identity preservation during translation significantly boosts recognition accuracy.
  • Bridging Distribution Gaps Matters: MMD loss effectively addresses the subtle statistical differences between domains, leading to more realistic and useful synthetic data for training. This is particularly important when the target domain has lower image quality or more complex backgrounds.
  • Transfer Learning Amplifies Gains: The combination of domain adaptation (data amplification) and transfer learning proves to be a powerful synergy, allowing the model to learn from a richer, more representative dataset and achieve higher accuracy.

Key Results

The proposed method achieved impressive results:

  • Controlled Scene (Source Domain): Average recognition accuracy of 96.9%.
  • Actual Culture Scene (Target Domain): Average recognition accuracy of 94%.
  • Ablation Studies: The combination of CycleGAN, IDF, and MMD (the full method) significantly outperformed individual components and direct transfer methods. For instance, the full method achieved a recall of 77.5% and mAP of 88.7% for target-to-source transfer, a substantial improvement over direct transfer.
  • Qualitative Results: Figure 2 showcases the visual improvements, with the proposed method generating more realistic and identity-preserving images compared to simpler CycleGAN applications.

Limitations and Future Work

  • Data Quality in Target Domain: The paper acknowledges that the lower image quality in the actual farming environment (target domain) leads to more loss of identity information during migration. This suggests that further improvements in image enhancement or more robust feature extraction might be beneficial.
  • Parameter Sensitivity: The authors highlight that the optimal weights for IDF (λ₂) and MMD (λ₃) can vary depending on data quality and domain distribution, suggesting that careful selection or adaptive tuning of these parameters is necessary for different applications.
  • Future Work: The paper implies that this work provides a strong foundation for future research in fish behavior tracking and phenotype measurement, suggesting that the developed identification technology can be integrated into more complex systems.

Summary updated

Abstract

Variable-condition fish recognition is a type of cross-scene and cross-camera fish re-identification (re-ID) technology. Due to the difference in the domain distribution of fish images collected under different culture conditions, the available training data cannot be effectively used for the new identification method. To solve these problems, we proposed a new method for identifying large yellow croaker based on the CycleGAN (cycle generative adversarial network) and transfer learning. This method constructs source sample sets and target sample sets by acquiring large yellow croaker images in controllable scenes and actual farming conditions, respectively. The CycleGAN was used as the basic framework for image transformation from the source domain to the target domain to realize data amplification in the target domain. In particular, IDF (identity foreground loss) was used to optimize identity loss judgment criteria, and MMD (maximum mean discrepancy) was used to narrow the distribution between the source domain and target domain. Finally, transfer learning was carried out with the expanded samples to realize the identification of large yellow croaker under varying conditions. The experimental results showed that the proposed method achieved good identification results in both the controlled scene and the actual culture scene, with an average recognition accuracy of 96.9% and 94%, respectively. These provide effective technical support for the next steps in fish behavior tracking and phenotype measurement.
BibTeX
@article{liu2023identification,
  author = {Shijing Liu and Cheng Qian and Xueying Tu and Haojun Zheng and Lin Zhu and Huang Liu and Jun Chen},
  title = {{Identification of Large Yellow Croaker under Variable Conditions Based on the Cycle Generative Adversarial Network and Transfer Learning}},
  journal = {Journal of Marine Science and Engineering},
  year = {2023},
  doi = {10.3390/jmse11071461},
}
Figure