TL;DR
This paper introduces FishViT, a novel, lightweight vision Transformer designed for high-speed, accurate fish detection in challenging water-land transfer scenarios. By employing a similarity-aware multi-level encoder and a soft-threshold attention mechanism, FishViT effectively handles dense, similar fish and noisy backgrounds, achieving over 80 FPS with state-of-the-art accuracy.
What Problem Does This Paper Address?
The fishery industry relies on efficient and accurate fish detection, particularly during the water-land transfer process where fresh fish are moved from culture vessels to markets. Traditional manual detection methods are inefficient, costly, prone to errors due to fatigue and individual inconsistencies, and can even harm the fish. While deep learning has shown promise, existing object detection models, especially Transformer-based ones, often struggle with the high computational complexity required for real-time applications and the specific challenges of this domain: dense groups of highly similar fish, occlusions, and cluttered backgrounds. This paper aims to bridge this gap by developing an intelligent, edge-deployable system that can perform fast and accurate fish detection in these demanding conditions.
Methods
The core of the proposed solution is the FishViT (Fish Vision Transformer) model, built upon a lightweight ResNet18 backbone for efficient feature extraction. FishViT introduces two key novel components:
- Similarity-Aware Multi-Level Encoder: This component enhances multi-scale feature representations in parallel. It processes features from different levels of the backbone through multiple branches, each employing a self-attention mechanism for global modeling. This parallel structure helps in generating discriminative representations for fish of varying sizes and addresses the issue of similar appearances.
- Soft-Threshold Attention (STAttention): This novel attention mechanism is designed to suppress background noise and accurately identify edge details of similar fish. It combines a factorized attention approach for computational efficiency with a soft-thresholding function inspired by signal processing. This helps in filtering out irrelevant information and focusing on crucial fish features, thereby improving robustness in noisy environments.
The system also incorporates Pooling Positional Encoding (PPE), a method to speed up computation while maintaining the effectiveness of positional encoding in Transformer models.
Datasets
The authors collected 85 challenging video sequences from real fish water-land transfer scenarios. These sequences feature high frame rates (90 FPS) and high resolution, capturing two fish species ('Black carp' and 'Silver carp'). The dataset is divided into:
- 50 sequences for training
- 5 sequences for validation
- 30 sequences for testing
These sequences are characterized by six challenging attributes: 'spume', 'overexposure', 'tumbling', 'dense', 'flowing', and 'underexposure'. This newly curated dataset serves as a benchmark for evaluating fish detection in this specific domain.
Key Insights and Ideas
- The "Similarity" Challenge: The paper highlights that the high visual similarity between individual fish is a primary obstacle in accurate detection, leading to missed detections and false positives. FishViT's design directly targets this by enhancing feature discriminability.
- Noise as Information: Instead of just removing noise, the STAttention mechanism cleverly uses a soft-threshold to convert noise into active features or suppress it, thereby improving the detection of subtle edge details crucial for distinguishing similar fish.
- Edge-Awareness for Similar Objects: The combination of multi-scale feature enhancement and noise suppression through STAttention allows the model to focus on the precise boundaries of fish, which is critical when dealing with visually homogeneous groups.
- Lightweight for Edge Deployment: The choice of a ResNet18 backbone and the efficient STAttention mechanism are crucial for enabling the system to run on edge devices, making it practical for real-world deployment.
Key Results
- State-of-the-Art Performance: FishViT achieved the best performance across all evaluated metrics, including AP50 (94.7%), Ecls (0.11), Eloc (1.65), and Emiss (0.28), outperforming other real-time and end-to-end detectors like YOLOv8, DETR, Anchor-DETR, Conditional-DETR, and RT-DETR.
- High Speed: The system achieves a real-time inference speed of over 80 FPS, meeting the demands of high-speed water-land transfer.
- Robustness to Challenging Scenarios: FishViT demonstrated superior performance across all six challenging attributes (spume, overexposure, tumbling, dense, flowing, underexposure), particularly excelling in the "dense" scenario.
- Module Effectiveness: Ablation studies showed significant improvements from both the STAttention (AP50 +3.5%) and the Similarity-Aware Multi-Level Encoder (AP50 +4.1%). The combined system (FishViT) achieved a substantial AP50 increase of 5.6% over the baseline.
Limitations and Future Work
- Limitations: While the paper demonstrates strong performance, the primary focus is on fish detection. The authors acknowledge that subsequent tasks like tracking, segmentation, and sizing are critical for a complete system. The current work provides a robust foundation for these future steps.
- Future Work: The authors suggest that FishViT can significantly enhance subsequent critical tasks in the field of water-land transfer, including tracking, segmentation, sizing, and counting of fish. The intelligent vision system with FishViT is positioned to aid in the advancement of fish detection in water-land transfer.
Summary updated