Abstract
We introduce a novel approach to studying animal behaviour and the context in which it occurs, through the use of microphone backpacks carried on the backs of individual free-flying birds. These sensors are increasingly used by animal behaviour researchers to study individual vocalisations of freely behaving animals, even in the field. However such devices may record more than an animals vocal behaviour, and have the potential to be used for investigating specific activities (movement) and context (background) within which vocalisations occur. To facilitate this approach, we investigate the automatic annotation of such recordings through two different sound scene analysis paradigms: a scene-classification method using feature learning, and an event-detection method using probabilistic latent component analysis (PLCA). We analyse recordings made with Eurasian jackdaws (Corvus monedula) in both captive and field settings. Results are comparable with the state of the art in sound scene analysis; we find that the current recognition quality level enables scalable automatic annotation of audio logger data, given partial annotation, but also find that individual differences between animals and/or their backpacks limit the generalisation from one individual to another. we consider the interrelation of 'scenes' and 'events' in this particular task, and issues of temporal resolution.
Introduction
Bio-logging technologies, such as GPS and accelerometers, have significantly advanced our understanding of animal movement. However, these sensors often lack the ability to capture the environmental and social context of behavior. Audio recording via "on-bird" microphones offers a rich alternative, potentially revealing whether a bird is in a noisy colony, interacting with conspecifics, or being targeted by a predator. The primary obstacle to this approach is the sheer volume of data; manual annotation of hundreds of hours of audio is labor-intensive and often impossible without visual verification. This study explores Automatic Acoustic Recognition (AAR) to transform raw audio into structured behavioral time budgets.
Methods
The research evaluates two distinct computational paradigms for sound analysis, as illustrated in the system architecture (Figure 1).
- Classifier-based System: This approach treats 5-second audio segments as "scenes." It employs spherical k-means feature learning to transform Mel-spectrograms into a feature space, which is then classified using a Random Forest algorithm.
- Event Detection System: Utilizing Probabilistic Latent Component Analysis (PLCA), this system decomposes spectrograms into overlapping sound "exemplars" to identify specific micro-events, such as individual wing flaps.
Both systems were augmented with Hidden Markov Models (HMMs) to smooth predictions based on the temporal "grammar" of bird behavior. The models were tested on datasets from Eurasian jackdaws (Corvus monedula), including a video-validated captive dataset (8.4 hours) and a human-coded field dataset (18.5 hours). The annotation taxonomy included movements, vocalizations, and environmental contexts (Figure 2).
Results
The classifier-based "scene" approach consistently outperformed the PLCA "event" approach, leading by an average of 5 percentage points in AUC and 8 percentage points in F-score. This suggests that identifying general behavioral states over several seconds is more robust than pinpointing discrete events for biological time-budgeting.
Key findings include:
- Individual Variability: Models trained on one individual often failed to generalize to others due to unique "acoustic fingerprints" caused by backpack fit or individual gait.
- High-Performance Categories: The systems were most effective at identifying focal calls (the bearer's own voice), flying, and self-maintenance (preening).
- HMM Impact: Post-processing with HMMs improved the F-score by reducing "flickering" in predictions, though it occasionally suppressed brief, legitimate transitions, leading to a slight decrease in AUC.
Discussion
While the results demonstrate that scalable automatic annotation is feasible, the "individual problem" remains a significant hurdle. Current systems require partial manual annotation for each new bird to maintain high accuracy. Future research should explore multi-sensor fusion (e.g., combining audio with accelerometry) and dynamic time resolution to better capture behaviors occurring at different timescales. Improving feature invariance to backpack placement will also be critical for developing universal, "plug-and-play" automation for field researchers.
Summary updated