Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Dongliang Ma, Jine Wei, Likai Zhu, Fang Zhao, Hao Wu, Xi Chen, Ye Li, Min LiuFrontiers in Marine Science 2024 · DOI

Abstract

Aquatic biodiversity monitoring relies on species recognition from images. While deep learning (DL) streamlines the recognition process, the performance of these methods is closely linked to large-scale labeled datasets, necessitating manual processing with expert knowledge and consuming substantial resources. Semi-supervised learning (SSL) offers a promising avenue to improve DL models by utilizing extensive unlabeled samples. However, complex collection environments and long-tailed class imbalance make SSL difficult to implement effectively. To address these challenges, we propose a Wavelet Fusion Network (WFN) and a Consistency Equilibrium Loss (CEL) function. WFN mitigates environmental noise by fusing image information at different frequencies, while CEL refines the consistency loss and adaptively adjusts margins for each class. Extensive experiments on the FishNet dataset show our method improves existing SSL schemes by up to 9.34% in overall accuracy, demonstrating the potential of SSL with limited labeled data for advancing aquatic conservation.

Introduction

Monitoring aquatic biodiversity is essential for maintaining ecosystem health, yet the manual processing of underwater imagery remains a significant bottleneck. While deep learning offers a path toward automation, its reliance on massive, expertly labeled datasets creates a barrier in terms of time, labor, and financial cost. Semi-supervised learning (SSL) presents a solution by leveraging a small amount of labeled data alongside a large volume of unlabeled data. However, applying SSL to marine environments introduces two unique difficulties: environmental noise and extreme class imbalance.

Challenges in Aquatic Species Recognition

Underwater imagery is notoriously difficult to process due to variable lighting, high turbidity, and complex backgrounds that obscure biological features. Furthermore, aquatic datasets typically follow a "long-tailed" distribution. A few common "head" classes have abundant samples, while the vast majority of rare "tail" classes have very few.

The long-tailed distribution of aquatic species data (A) and the corresponding drop in mean classification scores for rare species (B).

As shown in Figure 1, standard models often become biased toward common species, leading to a significant drop in classification performance for rare species—which are often the most critical for conservation efforts.

Methodology: WFN and CEL

To overcome these hurdles, we introduce two key architectural innovations integrated into the FixMatch SSL framework:

Wavelet Fusion Network (WFN)

Instead of processing raw images, the WFN uses a discrete wavelet transform to decompose images into low-frequency (LF) and high-frequency (HF) components. The LF stream captures abstract semantics (the "what"), while the HF stream focuses on fine details and textures (the "how"). These streams are integrated via a FusionBlock using cross-stream attention, allowing the model to prioritize biological features over environmental noise like backscatter or turbidity.

Consistency Equilibrium Loss (CEL)

Standard SSL uses a fixed confidence threshold for generating pseudo-labels, which often excludes rare species because the model lacks confidence in their classification. CEL adaptively adjusts the margin for each class based on its mean classification score. By lowering the barrier for rare classes and refining pseudo-labels to match the true data distribution, the model achieves a "classification equilibrium."

Overview of the proposed framework, integrating the Wavelet Fusion Network and Consistency Equilibrium Loss into the FixMatch scheme.

Experimental Results

The framework was evaluated on the FishNet dataset, which contains over 94,000 images across 17,357 species. Focusing on family-level classification (including 6 head, 52 medium, and 405 rare categories), the proposed method demonstrated significant gains:

  • Overall Accuracy: The full framework reached 37.11% accuracy, a 9.34% improvement over the standard FixMatch baseline (27.77%).
  • Rare Species Performance: Accuracy for rare categories improved from 22.24% to 32.61%.
  • Synergy: Ablation studies confirmed that WFN provides superior feature representations, while CEL ensures those features are utilized effectively across all classes.

Conclusion and Future Work

The integration of wavelet-based frequency decomposition and adaptive loss functions significantly advances the state-of-the-art for semi-supervised aquatic species recognition. While the WFN introduces some computational overhead due to its parallel encoder structure, the performance gains—particularly for rare species—justify the cost for biodiversity monitoring. Future work will explore multi-level wavelet decomposition and the extension of this framework to real-time object detection and tracking in the wild.

Summary updated

Abstract

Aquatic biodiversity monitoring relies on species recognition from images. While deep learning (DL) streamlines the recognition process, the performance of these method is closely linked to the large-scale labeled datasets, necessitating manual processing with expert knowledge and consume substantial time, labor, and financial resources. Semi-supervised learning (SSL) offers a promising avenue to improve the performance of DL models by utilizing the extensive unlabeled samples. However, the complex collection environments and the long-tailed class imbalance of aquatic species make SSL difficult to implement effectively. To address these challenges in aquatic species recognition within the SSL scheme, we propose a Wavelet Fusion Network and the Consistency Equilibrium Loss function. The former mitigates the influence of data collection environment by fusing image information at different frequencies decomposed through wavelet transform. The latter improves the SSL scheme by refining the consistency loss function and adaptively adjusting the margin for each class. Extensive experiments are conducted on the large-scale FishNet dataset. As expected, our method improves the existing SSL scheme by up to 9.34% in overall classification accuracy. With the accumulation of image data, the improved SSL method with limited labeled data, shows the potential to advance species recognition for aquatic biodiversity monitoring and conservation.
BibTeX
@inproceedings{ma2024semi,
  author = {Dongliang Ma and Jine Wei and Likai Zhu and Fang Zhao and Hao Wu and Xi Chen and Ye Li and Min Liu},
  title = {{Semi-supervised learning advances species recognition for aquatic biodiversity monitoring}},
  booktitle = {Frontiers in Marine Science},
  year = {2024},
  doi = {10.3389/fmars.2024.1373755},
}
Figure