TL;DR
Researchers have developed FCFormer, a Transformer-based deep learning framework designed to accurately count fish and estimate density in crowded aquaculture tanks. By combining global feature extraction with adaptive feature fusion, the model achieves over 97% accuracy, significantly outperforming traditional CNN-based methods in handling occlusion and small, distant targets.
What Problem Does This Paper Address?
In recirculating aquaculture systems (RAS), knowing exactly how many fish are in a tank is vital for managing feed, monitoring health, and predicting survival rates. However, counting fish is a nightmare for computer vision.
Traditional Convolutional Neural Networks (CNNs) often struggle in these environments for three main reasons:
- Mutual Occlusion: Fish swim over and under each other, hiding parts of their bodies.
- Rapid Movement: Fast-moving fish create motion blur.
- Scale Variation: Fish near the camera look huge, while those further away are tiny, blurry dots.
CNNs typically have a limited "receptive field," meaning they are great at seeing local textures but poor at understanding the global context of a crowded school. This paper introduces a Transformer-based approach to capture that missing global information.
Methods
The authors propose a framework called FCFormer (Fish Count Transformer), which moves away from simple object detection toward density map regression. Instead of drawing a box around every fish, the model predicts a map where the intensity of pixels represents fish density.
The architecture consists of three novel components:
- Twins-SVT Backbone: Unlike standard Transformers that can be computationally heavy, the Twins-SVT uses a hierarchical structure to extract global features efficiently, allowing the model to "see" the entire tank at once.
- BiCC (Bi-FPN + CAM Count Module): This is the "aggregator." It uses a Bidirectional Feature Pyramid Network (Bi-FPN) to pass information up and down between high-level and low-level features. It’s paired with a Class Activation Map (CAM) module that forces the network to focus on critical regions where fish are actually present.
- CRMHead: An adaptive regression head that intelligently fuses features. It recognizes that different layers of the network are better at detecting different things—low-level layers for small, distant fish and high-level layers for large, nearby fish.
Datasets
The researchers utilized two primary data sources:
- Lab Tank Dataset: A newly curated dataset consisting of 842 high-resolution images from four different aquaculture tanks. This was a massive manual undertaking: three researchers spent three weeks annotating approximately 205,806 individual fish.
- Deep Blue No. 1: To test generalization, the authors used real-world underwater footage from an offshore aquaculture cage in the Yellow Sea. This dataset features Atlantic salmon in much more complex, low-visibility environments compared to the lab tanks.
Key Insights and Ideas
- Global Context is King: The move to a Transformer backbone (Twins-SVT) was the "aha moment." By capturing global dependencies, the model avoids the "double counting" or "missing" errors common in CNNs when fish are tightly packed.
- Attention to Detail: The inclusion of the CAM module acts like a spotlight, telling the model which parts of the image are noise (like bubbles or reflections) and which are fish.
- Scale Agnosticism: By using the CRMHead to adaptively select feature layers, the model doesn't need to be told whether it's looking at a zoomed-in or zoomed-out shot; it automatically weights the features that best match the fish sizes in the frame.
Key Results
FCFormer sets a new benchmark for fish counting in aquaculture:
- Accuracy: Achieved a peak accuracy of 97.06%.
- Error Metrics: Recorded a Mean Absolute Error (MAE) of 6.37 and a Root Mean Square Error (MSE) of 8.69.
- Comparison: It outperformed other state-of-the-art architectures, including CCTrans and DM_Count, showing a 2.02% improvement over the baseline Twins transformer.
- Real-World Performance: On the "Deep Blue No. 1" ocean dataset, the model maintained a high accuracy of 97.03%, proving it isn't just a "lab-only" solution.
Limitations and Future Work
While FCFormer is highly effective, the authors highlight a few hurdles for the future:
- Background Diversity: The primary training set had relatively uniform backgrounds. To make the model truly robust, it needs to be exposed to a wider variety of tank materials and water turbidities.
- Extreme Occlusion: While better than CNNs, the model still faces challenges when fish are so densely packed that they form a solid mass.
- Edge Deployment: The current model is powerful but computationally demanding. The authors suggest that model compression and acceleration (like pruning or quantization) will be necessary to run FCFormer on resource-constrained edge devices directly at the fish farm.
Summary updated