Abstract: Automated bioacoustic analysis is essential for biodiversity monitoring and conservation, requiring advanced deep learning models that can adapt to diverse bioacoustic tasks. This article presents a comprehensive review of large-scale pretrained bioacoustic foundation models and systematically investigates their transferability across multiple bioacoustic classification tasks. We overview bioacoustic representation learning including major pretraining data sources and benchmarks. On this basis, we review bioacoustic foundation models by thoroughly analysing design decisions such as model architecture, pretraining scheme, and training paradigm. Additionally, we evaluate selected foundation models on classification tasks from the BEANS and BirdSet benchmarks, comparing the generalisability of learned representations under both linear and attentive probing strategies. Our comprehensive experimental analysis reveals that BirdMAE, trained on large-scale bird song data with a self-supervised objective, achieves the best performance on the BirdSet benchmark. On BEANS, BEATs$_{NLM}$, the extracted encoder of the NatureLM-audio large audio model, is slightly better. Both transformer-based models require attentive probing to extract the full performance of their representations. ConvNext$_{BS}$ and Perch models trained with supervision on large-scale bird song data remain competitive for passive acoustic monitoring classification tasks of BirdSet in linear probing settings. Training a new linear classifier has clear advantages over evaluating these models without further training. While on BEANS, the baseline model BEATs trained with self-supervision on AudioSet outperforms bird-specific models when evaluated with attentive probing. These findings provide valuable guidance for practitioners selecting appropriate models to adapt them to new bioacoustic classification tasks via probing.
Introduction
Monitoring biodiversity is critical for conservation, but manual analysis of thousands of hours of field recordings is impossible. While Deep Learning (DL) has revolutionized species identification, standard models struggle with "the long tail"—rare or under-studied species with very little training data. Foundation models—models trained on massive, diverse datasets—offer a solution through transfer learning. However, the field has lacked a standardized benchmark to compare these models across different architectures (CNNs vs. Transformers), training paradigms (Supervised vs. Self-Supervised), and preprocessing steps. This paper provides a rigorous comparative review to identify which "backbone" generalizes best to new, unseen bioacoustic tasks.
Methodology and Probing Strategies
The authors categorize and evaluate models based on several design dimensions:
- Architectures: Traditional Convolutional Neural Networks (CNNs) like ConvNext and EfficientNet are compared against modern Transformer-based Vision Transformers (ViT).
- Training Paradigms: Supervised Learning (SL) on labeled species is contrasted with Self-Supervised Learning (SSL) using pretext tasks like Masked Autoencoding (MAE).
- Probing Strategies: A key technical contribution is the comparison between Linear Probing (training a single layer on top of a frozen model) and Attentive Probing (using a trainable attention mechanism to query the frozen model’s internal features).
- Standardization: All models were re-evaluated using consistent preprocessing (resampling, segment lengths, and spectrogram parameters) to ensure a fair comparison.
Data Landscape
The study leverages a massive array of data for pretraining and evaluation. Pretraining sources include general audio (AudioSet) and specialized bioacoustic repositories like Xeno-Canto and the Macaulay Library.
As shown in the data distribution, there is a significant taxonomic bias, with birds being the most represented category across all major datasets, while groups like reptiles and insects remain underrepresented.
Evaluation is conducted on two primary benchmarks: BEANS, covering multiple taxa (birds, bats, dogs, frogs, insects), and BirdSet, focused on avian species classification in both focal and noisy soundscape (PAM) data.
Key Findings and Performance Analysis
SSL as the New Gold Standard
Models trained with self-supervision, such as BirdMAE, generally outperform supervised models when transferring to new tasks. SSL allows the model to learn the underlying "physics" of sound without being constrained by specific taxonomic labels, which is crucial for generalizing to rare species.
The "Attention Tax"
Transformer-based models are powerful but require specific adaptation strategies. To extract their full potential, researchers must use attentive probing. Linear probing alone often fails to capture the complex temporal-spectral relationships learned by Transformers. While CNN-based models like Perch and ConvNext$_{BS}$ are more parameter-efficient for simple linear probing, they are eventually surpassed by Transformers under attentive probing.
Generalists vs. Specialists
General-purpose audio models (like BEATs, trained on YouTube clips) outperform bird-specific models on the general BEANS benchmark. However, for specialized bird tasks, domain-specific pretraining (like BirdMAE) still reigns supreme. Specifically, BirdMAE achieved the top performance on the BirdSet benchmark (86.54 AUROC), while BEATs$_{NLM}$ took the crown on BEANS (98.57 AUROC).
Conclusion and Future Outlook
This review demonstrates that while foundation models provide a powerful starting point for bioacoustics, the choice of model and probing strategy is critical. Even for models already trained on relevant species, training a new linear classifier on embeddings yields significantly better results than using "out-of-the-box" predictions.
Future research should explore hybrid pretraining (combining general and bioacoustic data), parameter-efficient fine-tuning (PEFT) methods like LoRA, and the development of foundation models trained directly on raw, noisy soundscape data rather than clean focal recordings.
Summary updated