For decades, the "gold standard" for understanding animal behavior was a human expert with a stopwatch and a clipboard. Whether in a neuroscience lab or on a commercial farm, researchers relied on manual observation to score "ethograms"—detailed catalogs of actions like grooming, feeding, or social interaction.
However, this approach has reached a breaking point. Manual scoring is notoriously subjective, labor-intensive, and prone to human fatigue. As Hueihan Jhuang and Tomaso Poggio noted in Automated home-cage behavioural phenotyping of mice, analyzing a single 24-hour cycle of mouse behavior can require thousands of person-hours. Furthermore, the mere presence of a human observer can induce stress in animals, particularly in aquatic environments or sensitive laboratory settings, as highlighted by Caitlin L. Stockwell in her work on precision aquaculture.
A new research direction, often termed computational ethology, is replacing the human observer with a suite of deep learning models and multimodal sensors. This shift allows for continuous, objective, and non-invasive monitoring of animal welfare at an unprecedented scale.
From Pixels to Postures
The first challenge in automating behavior analysis is "seeing" the animal without physical tags. Early systems relied on simple background subtraction, but modern frameworks use markerless pose estimation to track specific body parts.
Oliver Sturman and Johannes Bohacek demonstrated in Deep learning based behavioral analysis enables high precision rodent tracking that open-source tools like DeepLabCut, when paired with simple classifiers, can match or even exceed the accuracy of human experts. By training models on "behavioral fingerprints," these systems eliminate the inter-rater variability that often plagues manual studies.
For more complex scenarios, such as multiple interacting animals, researchers like Cristina Segalin developed the Mouse Action Recognition System (MARS). MARS uses a hierarchical approach: first tracking the pose of two mice, then using those spatial features to classify social behaviors like sniffing or mounting. This moves the field from simple "blob tracking" to understanding the nuances of social dynamics.
Inferring Internal States
The most significant leap in this field is the move from tracking what an animal is doing to inferring how it feels. This is known as automated affective computing.
Traditional welfare assessments often use "grimace scales" to detect pain, but these are difficult for humans to apply consistently. Guang Li introduced PainSeeker, a deep learning framework that identifies pain in rats via facial expressions, even when the head is not directly facing the camera. Similarly, Sofia Broomé and Hedvig Kjellstrom have pushed this into the veterinary space, using 3D skeleton reconstruction to monitor "activity budgets" in horses to detect early signs of distress.
By analyzing subtle shifts in movement—such as the "streamlined" gaits identified in the AnimalFormer framework by Ahmed Rashid Qazi—researchers can now quantify welfare indicators that were previously invisible to the human eye.
Scaling to the Farm
While laboratory settings are controlled, the real-world challenge lies in Precision Livestock Farming (PLF). Here, the goal is to monitor thousands of animals in noisy, high-density environments.
To make this practical, researchers are focusing on two areas:
- Multimodal Integration: Meng Cui and Haohe Liu have shown that combining video with audio (vocalizations) or hydroacoustic data significantly improves the robustness of monitoring, such as assessing fish feeding intensity.
- Edge Deployment: Large models are too slow for real-time farm monitoring. Xiaobo Wang developed GSCW-YOLO, a lightweight model that recognizes goat behaviors at 175 frames per second on a tiny 5.9MB footprint. Similarly, Seyedeh Fa'ezeh Hosseini Noorbin demonstrated that transforming sensor data into spectrograms allows for high-accuracy cattle health monitoring on low-power ear tags.
The Future of the Digital Twin
The ultimate goal of this research is the creation of "digital twins" for animals. As envisioned by Aarushi Gupta and Matthias Wieland in their work on teat health in dairy cows, these systems provide a continuous data stream that allows for individualized care. Instead of waiting for a veterinarian to spot a limp or a lesion, AI systems can flag the earliest deviations from an animal's "normal" behavioral baseline, enabling intervention before a condition becomes critical.