An Integrated Gather-and-Distribute Mechanism and Attention-Enhanced Deformable Convolution Model for Pig Behavior Recognition

Rui Mao, Dongzhen Shen, Ruiqi Wang, Yiming Cui, Yufan Hu, Mei Li, Meili WangAnimals 2024 · DOI

TL;DR

This paper introduces DM-GD-YOLO, a novel model for recognizing pig behaviors, including critical abnormal ones, by enhancing YOLOv8 with attention-guided deformable convolutions and a gather-and-distribute mechanism. This approach significantly improves accuracy in complex environments, offering a practical solution for better pig welfare and farm management.

What Problem Does This Paper Address?

Accurate and timely recognition of pig behavior is crucial for monitoring their health and welfare, especially in intensive farming environments. Abnormal behaviors like fighting or fence-climbing can indicate stress, illness, or environmental issues, leading to economic losses and animal suffering. Traditional manual monitoring is labor-intensive and prone to errors. While computer vision and deep learning have shown promise, existing models struggle with:

  • Non-rigid deformations: Pigs' postures change rapidly and unpredictably, making it hard for standard convolutional filters to capture features consistently.
  • Complex environments: Pig pens often have varying lighting, occlusions, and multiple pigs interacting, posing challenges for robust detection.
  • Simultaneous recognition: Effectively identifying a range of common and abnormal behaviors concurrently is difficult.
  • Feature fusion: Integrating information from different scales and layers of a network efficiently is key to comprehensive understanding.

Methods

The core of this work lies in enhancing the YOLOv8 architecture with two key novel components:

  • Attention-Enhanced Deformable Convolution (DCN-MPCA):
    • Deformable Convolutional Networks (DCN): This technique allows convolutional kernels to adapt their sampling locations by learning offsets, making them more flexible in capturing non-rigid deformations.
  • Multi-Path Coordinate Attention (MPCA): To overcome limitations in how DCNs learn these offsets, MPCA is introduced. It uses multiple attention branches to better capture global and local feature information, leading to more optimized offset calculations for the DCN. This DCN-MPCA module is integrated into the backbone's C2f module, creating the "C2f-DM" module, which significantly boosts feature extraction.
  • Gather-and-Distribute (GD) Mechanism:
    • This mechanism is implemented in the neck of the YOLOv8 network to improve how features from different layers are fused.
  • It addresses the issue of non-adjacent layer feature fusion, which can lead to information loss in standard architectures like FPN.
  • The GD mechanism uses two branches (Low-GD and High-GD) to align, fuse, and inject global context information across multiple feature levels, ensuring more efficient information exchange and reducing data loss.

The proposed model, DM-GD-YOLO, combines these enhancements within the YOLOv8 framework.

Datasets

The study utilized a self-built dataset comprising 11,999 images collected from an online monitoring platform. This dataset focuses on pigs aged between 70 and 150 days, housed in pens with approximately 30 pigs. The dataset includes:

  • Seven typical pig behaviors: Sniffing, lying, walking, kneeling (common), and fighting, fence climbing, mounting (abnormal).
  • Data Augmentation: The Mosaic Data Augmentation (MDA) method was used to expand the dataset fivefold, incorporating image stitching, mirroring, clipping, random rotation, and HSV tone enhancement.
  • Data Split: The dataset was divided into training (70%), validation (20%), and testing (10%) sets.

Key Insights and Ideas

  • Deformable convolutions are essential for dynamic animal behavior: The inherent non-rigid nature of pig movements necessitates adaptive feature extraction, which DCNs provide.
  • Attention mechanisms can significantly improve DCNs: Simply using DCNs has limitations; integrating attention (MPCA) to guide offset learning makes them much more effective for complex tasks like pig behavior recognition.
  • Efficient multi-scale feature fusion is critical: The GD mechanism's ability to gather and distribute information across different network layers is key to preventing information loss and achieving a holistic understanding of behavior.
  • Abnormal behavior detection requires robust feature extraction: The model's success in identifying behaviors like fighting and fence-climbing highlights the importance of the DCN-MPCA module in capturing subtle but critical cues.

Key Results

The DM-GD-YOLO model demonstrated strong performance:

  • Recognition Accuracy: Achieved a precision of 88.2%, recall of 92.2%, and a mean Average Precision (mAP) of 95.3%.
  • Efficiency: The model is relatively lightweight, with 6.0 MB of parameters and 10.0 G FLOPs.
  • Outperformance: DM-GD-YOLO surpassed popular models like Faster R-CNN, EfficientDet, YOLOv7, and YOLOv8 in mAP, with a notable 2.6% increase over YOLOv8.
  • Specific Behavior Improvements: Significant gains were observed in recognizing walking (4.9% mAP increase) and fighting behaviors.
  • Robustness: Visualizations showed DM-GD-YOLO's superior ability to handle challenging conditions like low light and similar-looking behaviors (sniffing vs. walking).

Limitations and Future Work

The authors acknowledge several limitations:

  • Occlusion in High-Density Environments: In pens with many pigs, distant animals can be occluded, posing challenges for feature extraction and recognition.
  • Model Complexity: While efficient, the DCN-MPCA and GD mechanisms increase network parameters and complexity, potentially hindering deployment on resource-constrained mobile terminals.
  • Single-Frame Recognition: The current model primarily relies on single frames. While it aims for swift detection, multi-frame analysis could offer greater stability.

Future work will focus on:

  • Enhancing model robustness to environmental variations.
  • Exploring model pruning techniques for greater compactness and efficiency.
  • Investigating multi-frame analysis for more stable behavior recognition.
  • Testing the model in scenarios with even higher pig densities or different pen designs.

Summary updated

Abstract

The behavior of pigs is intricately tied to their health status, highlighting the critical importance of accurately recognizing pig behavior, particularly abnormal behavior, for effective health monitoring and management. This study addresses the challenge of accommodating frequent non-rigid deformations in pig behavior using deformable convolutional networks (DCN) to extract more comprehensive features by incorporating offsets during training. To overcome the inherent limitations of traditional DCN offset weight calculations, the study introduces the multi-path coordinate attention (MPCA) mechanism to enhance the optimization of the DCN offset weight calculation within the designed DCN-MPCA module, further integrated into the cross-scale cross-feature (C2f) module of the backbone network. This optimized C2f-DM module significantly enhances feature extraction capabilities. Additionally, a gather-and-distribute (GD) mechanism is employed in the neck to improve non-adjacent layer feature fusion in the YOLOv8 network. Consequently, the novel DM-GD-YOLO model proposed in this study is evaluated on a self-built dataset comprising 11,999 images obtained from an online monitoring platform focusing on pigs aged between 70 and 150 days. The results show that DM-GD-YOLO can simultaneously recognize four common behaviors and three abnormal behaviors, achieving a precision of 88.2%, recall of 92.2%, and mean average precision (mAP) of 95.3% with 6.0MB Parameters and 10.0G FLOPs. Overall, the model outperforms popular models such as Faster R-CNN, EfficientDet, YOLOv7, and YOLOv8 in monitoring pens with about 30 pigs, providing technical support for the intelligent management and welfare-focused breeding of pigs while advancing the transformation and modernization of the pig industry.
BibTeX
@inproceedings{mao2024integrated,
  author = {Rui Mao and Dongzhen Shen and Ruiqi Wang and Yiming Cui and Yufan Hu and Mei Li and Meili Wang},
  title = {{An Integrated Gather-and-Distribute Mechanism and Attention-Enhanced Deformable Convolution Model for Pig Behavior Recognition}},
  booktitle = {Animals},
  year = {2024},
  doi = {10.3390/ani14091316},
}
Figure