Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network

Wangli Hao, Meng Han, Kai Zhang, Li Zhang, Wangbao Hao, Fuzhong Li, Zhenyu Liu, 3. Yongji Puzhou Old City Beimenwai Cattle Farm, Yongji 044500, Shanxi, ChinaInternational Journal of Agricultural and Biological Engineering 2024 · DOI

Abstract

Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle. However, there are still several challenges in the current Jinnan cattle action recognition. Traditional methods are based on manual characteristics and low recognition accuracy. This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding. The acquired data from the previous methods contain a lot of noise, which will cause individual cattle to have excessive behaviors due to unsuitability. Concerning the high labor costs, low efficiency, and low model accuracy of the above approaches, this study developed a bottleneck attention-enhanced two-stream (BATS) Jinnan cattle action recognition method. It primarily comprises a Spatial Stream Subnetwork, a Temporal Stream Subnetwork, and a Bottleneck Attention Module. It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively, so as to extract richer and more robust features. Finally, the decision of the two branches can be fused to gain improved cattle action recognition performance. Compared with the traditional methods, the model proposed in this study has achieved state-of-the-art recognition performance, and the accuracy of motion recognition was 96.53%, which was 4.60% higher than other models. This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry.

Introduction

In smart animal husbandry, monitoring cattle behavior—such as eating, drinking, or lying down—is critical for assessing health, estrus cycles, and overall welfare. Historically, this required labor-intensive manual observation or invasive sensors that could cause animal stress. While computer vision offers a non-invasive alternative, existing models often struggle with environmental noise and low recognition accuracy. Many current approaches rely solely on single-stream networks that look at static images (spatial data) but ignore the crucial temporal dynamics of movement. This research introduces a robust, automated system tailored for the specific physical characteristics and behaviors of Jinnan cattle.

Methods: The BATS Architecture

The proposed Bottleneck Attention-enhanced Two-Stream (BATS) model utilizes a dual-pathway architecture to capture both appearance and motion.

The BATS model architecture, featuring spatial and temporal streams integrated with the Bottleneck Attention Module (BAM).

The architecture consists of:

  1. Spatial Stream Subnetwork: Processes individual RGB video frames to capture static information about objects and scenes.
  2. Temporal Stream Subnetwork: Processes Optical Flow to represent motion between consecutive frames, distinguishing actions like standing versus walking.
  3. Bottleneck Attention Module (BAM): Integrated into the ResNet backbone, BAM allows the network to focus on relevant features by capturing dependencies across both channels and spatial dimensions.

By placing BAM at each bottleneck of the network, the model extracts richer, more robust features that are less susceptible to the background noise typical of farm environments.

Dataset and Action Categories

The researchers established the Jinnan Cattle Action Dataset, collected from a farm in Yongji City, China. The dataset includes 1,821 video clips across six categories: drinking, eating, looking back, lying, walking, and standing.

An example of the "eating" action from the Jinnan Cattle Action Dataset.

The dataset captures cattle of various ages under varying illumination and angles. Data augmentation techniques like rotation, flipping, and color jittering were employed to prevent overfitting and enhance model generalizability.

Results and Discussion

The BATS model achieved a state-of-the-art accuracy of 96.53%, significantly outperforming standard architectures.

  • Comparative Performance: The model surpassed ResNet101 (95.10%), Vgg16 (93.57%), and MobileNetv2 (93.29%).
  • Ablation Study: Results confirmed that the combination of the two-stream approach and the BAM module was critical. The two-stream version of AlexNet, for instance, was 23.33% more accurate than its flow-only counterpart.
  • Efficiency: Despite the added complexity of the attention module, the model remains efficient enough for real-world deployment in smart farming systems.

Conclusion

This study demonstrates that integrating spatial-temporal streams with bottleneck attention significantly improves cattle action recognition. Future research will focus on increasing environmental diversity in datasets and exploring multimodal integration, such as combining video with audio or temperature data, to create more holistic monitoring systems for smart animal husbandry.

Summary updated

Abstract

Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle. However, there are still several challenges in the current Jinnan cattle action recognition. Traditional methods are based on manual characteristics and low recognition accuracy. This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding. The acquired data from the previous methods contain a lot of noise, which will cause individual cattle to have excessive behaviors due to unsuitability. Concerning the high labor costs, low efficiency, and low model accuracy of the above approaches, this study developed a bottleneck attention-enhanced two-stream (BATS) Jinnan cattle action recognition method. It primarily comprises a Spatial Stream Subnetwork, a Temporal Stream Subnetwork, and a Bottleneck Attention Module. It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively, so as to extract richer and more robust features. Finally, the decision of the two branches can be fused to gain improved cattle action recognition performance. Compared with the traditional methods, the model proposed in this study has achieved state-of-the-art recognition performance, and the accuracy of motion recognition was 96.53%, which was 4.60% higher than other models. This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry. Keywords: Jinnan cattle, action recognition, bottleneck attention, two-stream neural network DOI: 10.25165/j.ijabe.20241703.8202 Citation: Hao W L, Han M, Zhang K, Zhang L, Hao W B, Li F Z, et al. Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network. Int J Agric & Biol Eng, 2024; 17(3): 203-210.
BibTeX
@article{hao2024novel,
  author = {Wangli Hao and Meng Han and Kai Zhang and Li Zhang and Wangbao Hao and Fuzhong Li and Zhenyu Liu and 3. Yongji Puzhou Old City Beimenwai Cattle Farm, Yongji 044500, Shanxi, China},
  title = {{Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network}},
  journal = {International Journal of Agricultural and Biological Engineering},
  year = {2024},
  doi = {10.25165/j.ijabe.20241703.8202},
}
Figure