TL;DR
This paper introduces SideCow-VSS, a novel video dataset for semantic segmentation of dairy cows, addressing the critical need for realistic data in precision livestock farming. They also provide a benchmark of eight deep learning models, revealing a clear trade-off between segmentation accuracy and inference speed, with Mask2Former excelling in precision and PIDNet-s in speed.
What Problem Does This Paper Address?
Modern dairy farming is increasingly adopting Precision Livestock Farming (PLF) techniques, leveraging AI and computer vision for automated health monitoring. This allows for early detection of diseases and improved animal welfare. However, a significant bottleneck has been the lack of high-quality, realistic video datasets specifically for dairy cows in their natural farm environments. Existing datasets are often static, lack dense pixel-level annotations, or don't capture the temporal dynamics crucial for monitoring health indicators like gait and body condition. Furthermore, there's a need for standardized benchmarks to evaluate and compare different AI models for this specific task, helping researchers and practitioners choose the right tools for their needs.
Methods
The authors developed a semi-automated pipeline for creating their dataset. This involved:
- Initial Object Detection: Using YOLOv11 to generate bounding box proposals for cows in each frame.
- Initial Segmentation: Employing the Segment Anything Model 2 (SAM2) to create preliminary segmentation masks based on the bounding boxes.
- Manual Verification and Curation: A team of veterinarians and computer vision engineers meticulously reviewed and corrected these masks, discarding frames with significant errors or heavy occlusions.
For the benchmarking, they selected eight diverse semantic segmentation models, ranging from established Convolutional Neural Networks (CNNs) like U-Net and DeepLabV3+ to modern Transformer-based architectures such as SegFormer, SegNeXt-L, and Mask2Former. These models were trained and evaluated under consistent experimental conditions.
Datasets
The paper introduces the SideCow-VSS dataset, a new collection of 921 five-second video clips (totaling 46,050 frames) featuring side-view footage of dairy cows in a commercial farm. These clips are densely annotated at the pixel level. The dataset is split into training (70%), validation (15%), and testing (15%) sets, ensuring that all frames from a single 5-second sequence remain within the same split to prevent data leakage.
Key Insights and Ideas
- Side-view is Key: The authors emphasize that side-view perspectives are particularly valuable for assessing health indicators like Body Condition Scoring (BCS) and gait, making their dataset uniquely suited for these applications.
- The "Precision vs. Speed" Dilemma: The benchmark clearly illustrates that achieving the highest segmentation accuracy often comes at the cost of slower inference speeds, and vice-versa. This is a critical consideration for real-world deployment.
- Transformer Dominance in Accuracy: State-of-the-art Transformer models, particularly Mask2Former with a Swin-L backbone, achieve superior segmentation accuracy, highlighting their ability to capture complex visual features.
- Lightweight Models for Real-Time: For applications requiring immediate alerts, lightweight CNN-based models like PIDNet-s offer significantly faster inference, making them practical for real-time monitoring.
- Classic CNNs Remain Relevant: Despite the rise of Transformers, established CNN models like U-Net and DeepLabV3+ still provide robust and competitive accuracy, serving as strong baselines.
Key Results
- Highest Accuracy: Mask2Former with a Swin-L backbone achieved the highest mean Intersection over Union (mIoU) of 97.32%.
- Fastest Inference: PIDNet-s model delivered the fastest inference speed of 59.5 Frames Per Second (FPS).
- Competitive Accuracy: SegNeXt-L achieved an mIoU of 97.00%, while U-Net and DeepLabV3+ scored 96.97% and 96.96% mIoU, respectively.
- Qualitative Differences: Visual analysis showed Mask2Former providing the most precise boundary delineation, especially around legs and tails, while models optimized for speed sometimes struggled with intricate details.
Limitations and Future Work
- Dataset Specificity: The SideCow-VSS dataset was collected from a single farm. The authors acknowledge that model performance might vary across different breeds, housing systems, or lighting conditions, suggesting a need for broader environmental diversity in future datasets.
- Individual Animal Identification: The current semantic segmentation approach cannot distinguish between individual cows when they are in close proximity.
- Future Directions:
- Expand the dataset to include more diverse environments.
- Develop instance segmentation models to enable individual animal tracking for personalized health management.
- Explore leveraging precise 2D contours for inferring 3D shape information, potentially leading to more advanced health monitoring systems.
Summary updated