TL;DR
This paper introduces an automated pig counting system for large-scale farming using a single fisheye camera on an inspection robot. It tackles challenges like pig occlusion and movement with a novel bottom-up keypoint detection and tracking approach, coupled with a spatial-aware temporal filtering method to achieve highly accurate counts, outperforming human readers.
What Problem Does This Paper Address?
Accurately counting pigs in large, crowded housing facilities is a critical but labor-intensive and error-prone task for farmers. Existing automated methods often rely on single images, which have limited fields of view and struggle with the dynamic nature of pigs (movement, occlusion, and overlap). This makes it difficult to monitor large groups effectively. The authors aim to bridge this gap by developing a robust, real-time system that can handle these challenges in a practical, large-scale farming environment.
Methods
The proposed system employs a multi-stage approach:
- Sensor Configuration: A monocular fisheye camera mounted on an inspection robot that moves along a rail provides a top-down view of the entire pig house. This setup is crucial for capturing large areas.
- Bottom-Up Keypoint Detection: Instead of traditional bounding boxes, the system detects keypoints (e.g., body mid-point, ends, quarters) on individual pigs. This "bottom-up" approach is more robust to overlapping and occluded pigs, as it focuses on identifying parts rather than whole objects. A deep convolutional neural network (CNN) is used for this detection.
- Keypoint Association and Tracking: An online tracking algorithm associates these detected keypoints across video frames, assigning unique IDs to individual pigs. This is framed as a bipartite graph matching problem to link pig candidates between consecutive frames.
- Spatial-Aware Temporal Response Filtering (STRF): This novel method is the core of the counting mechanism. It uses spatial encoding to define "activated" and "deactivated" zones based on an activity scanning line. By analyzing the temporal sequence of a pig's presence in these zones, the STRF method can infer whether a pig should be counted (entering the activated zone) or not (e.g., if it re-enters the activated zone after being counted, or if tracking fails within a single zone). This effectively filters out false positives caused by tracking errors or pigs moving in and out of view.
Datasets
The authors collected 51 videos from two pig farming corporations. These videos were recorded at 1280x720 resolution and then resized and cropped to 352x640 for experiments. The dataset was split into 21 videos for training, 5 for validation, and 25 for testing. Ground truth counts were provided by human workers who observed the pigs during video recording. No new benchmark dataset was introduced; rather, existing videos were used for evaluation.
Key Insights and Ideas
- Keypoints over Bounding Boxes: The paper strongly advocates for using keypoints instead of bounding boxes for pig detection in crowded scenarios. This is because keypoints are less susceptible to the high overlap ratios that can lead to false negatives with bounding boxes.
- Bottom-Up for Robustness: The bottom-up detection strategy is key to handling the inherent challenges of pig farming environments, where pigs are rarely perfectly visible or distinct.
- STRF for Temporal Consistency: The STRF method is a clever way to leverage temporal information and spatial context to refine counts. By defining "activated" zones and analyzing transitions, it effectively mitigates the problem of double-counting or missing pigs due to movement and tracking issues.
- Edge Deployment for Practicality: Deploying the entire pipeline on an edge computing device highlights the system's real-world applicability and efficiency for on-site monitoring.
Key Results
- Superior Accuracy: The proposed system achieved a Mean Absolute Percentage Error (MAPE) of 2.67% and a Mean Absolute Error (MAE) of 3.32%. This significantly outperformed human readers, who had MAPE values ranging from 11.0% to 17.4% and MAE values from 12.6 to 26.3.
- Efficient Keypoint Detection: The proposed keypoint detection method achieved better accuracy than UNet and Stacked Hourglass networks, with significantly less computation cost and fewer parameters.
- Runtime Performance: The system achieved 82.6 FPS on a desktop with a GPU and was successfully deployed on edge devices, achieving 0.625 FPS on a Firefly-RK3399 and 3.19 FPS on a NVIDIA Jetson Nano.
Limitations and Future Work
- Image Boundaries: The paper notes that detection failures can occur around image boundaries due to fisheye distortion and image cutoff. However, the STRF method is designed to minimize the impact of these failures on the final count.
- Computational Cost: While the system is efficient, achieving higher FPS on edge devices might still be a target for future improvements, especially for real-time applications requiring very high frame rates.
- Generalizability: The current system is tailored for pig counting. Future work could explore its adaptability to other livestock or similar crowd-counting scenarios with dynamic objects.
- Further STRF Refinements: While STRF is effective, exploring more sophisticated temporal filtering or spatial encoding strategies could potentially further enhance robustness.
Summary updated