Automatic Detection Method of Dairy Cow Feeding Behaviour Based on YOLO Improved Model and Edge Computing

Zhenwei Yu, Yuehua Liu, Sufang Yu, Ruixue Wang, Zhanhua Song, Yinfa Yan, Fade Li, Zhonghua Wang, Fuyang TianSensors 2022 · DOI

TL;DR

This paper introduces DRN-YOLO, a novel deep learning model for automatically detecting dairy cow feeding behavior in real-time using edge computing. By enhancing the YOLOv4 architecture with a custom DRNet backbone and Spatial Pyramid Pooling, DRN-YOLO significantly improves detection accuracy and efficiency, paving the way for more intelligent and welfare-focused dairy farming.

What Problem Does This Paper Address?

Monitoring dairy cow feeding behavior is crucial for assessing their health, detecting diseases (like mastitis), and evaluating overall welfare. Traditional methods often rely on wearable sensors, which can be cumbersome and have limitations in data richness, or manual observation, which is labor-intensive and prone to error. Existing computer vision approaches struggle with the complexities of open farm environments, leading to low accuracy and sensitivity. The challenge is to develop a robust, accurate, and efficient automated system for real-time feeding behavior detection that can operate effectively in these dynamic conditions.

Methods

The core of this research is the development of the DRN-YOLO model, an improved version of the YOLOv4 object detection framework. Key modifications include:

  • Custom Backbone Network (DRNet): The standard CSPDarknet backbone of YOLOv4 is replaced with a self-designed DRNet. This new backbone incorporates elements from DenseNet and ResNet, utilizing Denseblock and Resblock modules. The DRNet aims to improve feature extraction by allowing direct information flow through shortcut connections while also learning differences, simplifying the learning objective and mitigating gradient disappearance.
  • Enhanced Feature Pyramid: The model incorporates multiple feature scales and a Spatial Pyramid Pooling (SPP) structure. This is designed to enrich the interaction of semantic features across different scales, improving the model's ability to capture both local and global contextual information.
  • CIOU Loss: The Continuous Intersection over Union (CIOU) loss function is employed for bounding box regression. CIOU considers overlap, distance, and aspect ratio, leading to more stable and accurate object localization compared to simpler IoU variants.
  • Edge Computing: The entire system is designed to run on an edge computing device (NVIDIA Jetson TX2). This allows for real-time processing of video feeds directly on the farm, reducing latency and the need for constant data transmission to a central server.

Datasets

The study utilized a custom dataset collected from Taian Jinlan Dairy Farming Company in Shandong Province, China.

  • Data Collection: Video data was captured over 27 days using a ZED2 depth camera, focusing on feeding times. Images were extracted at a resolution of 1280x720 pixels at 30 fps.
  • Dataset Size: A total of 10,288 images were collected and manually labeled using LabelImg.
  • Behavior Categories: The dataset includes labels for "Feeding Behaviour," "Chewing Behaviour," and "Pushing Behaviour."
  • Data Split: The dataset was split into training and testing sets. The paper provides counts for labeled boxes for each behavior category in both training and testing sets.

Key Insights and Ideas

  • DRNet's Superior Feature Extraction: The custom DRNet backbone, by integrating DenseNet and ResNet principles, demonstrably extracts richer and more comprehensive features than the standard YOLOv4 backbone, leading to improved detection performance.
  • SPP for Multi-Scale Understanding: The integration of the Spatial Pyramid Pooling structure is crucial for effectively fusing local and global features, enhancing the model's robustness to variations in cow posture and feeding intensity.
  • Edge Computing for Real-Time Farm Applications: Deploying the detection model on an edge device like the NVIDIA Jetson TX2 is essential for practical, real-time monitoring in a farm setting, enabling immediate insights without network bottlenecks.
  • CIOU Loss for Stable Localization: The choice of CIOU loss function contributes to more precise bounding box predictions, which is vital for accurate behavior classification.

Key Results

The DRN-YOLO model demonstrated significant improvements over the baseline YOLOv4:

  • Accuracy Gains:
    • Precision: Improved by 1.70% (front) and 1.67% (above).
  • Recall: Improved by 1.82% (front) and 1.27% (above).
  • mAP: Improved by 0.97% (front) and 1.48% (above).
  • F1-Score: Improved by 1.76% (front) and 1.48% (above).
  • Speed: DRN-YOLO achieved a detection time of 22.65 ms, which is significantly faster than Faster RCNN (160 ms) and comparable to YOLOv4 (31.17 ms), while offering superior accuracy.
  • Robustness: The model showed good performance even with occlusions and variations in lighting and camera angles, though limitations were noted.
  • Loss Curve Analysis: DRN-YOLO exhibited more stable and faster convergence in loss curves compared to YOLOv4, indicating better training stability and a more robust model.

Limitations and Future Work

  • Limitations:
    • Grass Arching Misjudgment: The model occasionally misjudges "grass arching behavior" as feeding behavior due to their visual similarity.
  • Limited Comparison: The ablation study focused on individual modules, but a broader comparison with other target detection and tracking models was not performed.
  • Environmental Factors: The study did not consider the impact of weather (cloudy, foggy) or night-time conditions on detection performance.
  • Future Work:
    • Behavior Subdivision: Further subdivide feeding behaviors (e.g., swallowing, regurgitation, pushing) for more granular analysis.
  • Feed Depth Analysis: Integrate feed depth information from binocular cameras to analyze feed consumption and calculate real-time feeding rates.
  • Comprehensive System Development: Combine these advancements to create a fully automated monitoring system for long-term dairy cow feeding behavior tracking.
  • Broader Model Comparison: Conduct more extensive comparisons with other state-of-the-art detection and tracking algorithms.

Summary updated

Abstract

The feeding behaviour of cows is an essential sign of their health in dairy farming. For the impression of cow health status, precise and quick assessment of cow feeding behaviour is critical. This research presents a method for monitoring dairy cow feeding behaviour utilizing edge computing and deep learning algorithms based on the characteristics of dairy cow feeding behaviour. Images of cow feeding behaviour were captured and processed in real time using an edge computing device. A DenseResNet-You Only Look Once (DRN-YOLO) deep learning method was presented to address the difficulties of existing cow feeding behaviour detection algorithms' low accuracy and sensitivity to the open farm environment. The deep learning and feature extraction enhancement of the model was improved by replacing the CSPDarknet backbone network with the self-designed DRNet backbone network based on the YOLOv4 algorithm using multiple feature scales and the Spatial Pyramid Pooling (SPP) structure to enrich the scale semantic feature interactions, finally achieving the recognition of cow feeding behaviour in the farm feeding environment. The experimental results showed that DRN-YOLO improved the accuracy, recall, and mAP by 1.70%, 1.82%, and 0.97%, respectively, compared to YOLOv4. The research results can effectively solve the problems of low recognition accuracy and insufficient feature extraction in the analysis of dairy cow feeding behaviour by traditional methods in complex breeding environments, and at the same time provide an important reference for the realization of intelligent animal husbandry and precision breeding.
BibTeX
@inproceedings{yu2022automatic,
  author = {Zhenwei Yu and Yuehua Liu and Sufang Yu and Ruixue Wang and Zhanhua Song and Yinfa Yan and Fade Li and Zhonghua Wang and Fuyang Tian},
  title = {{Automatic Detection Method of Dairy Cow Feeding Behaviour Based on YOLO Improved Model and Edge Computing}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22093271},
}
Figure