SSL-Net: A Synergistic Spectral and Learning-based Network for Efficient Bird Sound Classification

Yiyuan Yang, Kaichen Zhou, Niki Trigoni, Andrew Markham

Abstract

Efficient and accurate bird sound classification is of important for ecology, habitat protection and scientific research, as it plays a central role in monitoring the distribution and abundance of species. However, prevailing methods typically demand extensively labeled audio datasets and have highly customized frameworks, imposing substantial computational and annotation loads. In this study, we present an efficient and general framework called SSL-Net, which combines spectral and learned features to identify different bird sounds. Encouraging empirical results gleaned from a standard field-collected bird audio dataset validate the efficacy of our method in extracting features efficiently and achieving heightened performance in bird sound classification, even when working with limited sample sizes. Furthermore, we present three feature fusion strategies, aiding engineers and researchers in their selection through quantitative analysis.

Introduction

Automated bird sound classification is a cornerstone of modern biodiversity monitoring. However, the field faces two significant challenges. First, the "data hunger" of deep learning models necessitates massive labeled datasets, which are costly and time-consuming to curate. Second, domain bias often limits the effectiveness of general-purpose pretrained audio models—originally trained on human speech or urban sounds—when applied to the intricate and diverse vocalizations of avian species.

To address these hurdles, we propose SSL-Net, a label-efficient framework designed to leverage the power of large-scale pretraining while maintaining the fine-grained acoustic sensitivity required for species-specific identification.

Methodology: The SSL-Net Architecture

The core of SSL-Net is a synergistic two-branch architecture designed to capture both high-level semantic information and low-level acoustic physics.

The SSL-Net architecture featuring the Spectral and Learned branches.

The Learned Branch

This branch utilizes an audio-pretrained transformer (such as BEATs) as its backbone. It extracts high-level semantic features from raw waveforms, which are then encoded through a ResNet18 module. This allows the model to benefit from broad audio knowledge learned from massive, non-specialized datasets.

The Spectral Branch

To ground the model in domain-specific acoustic characteristics, the Spectral Branch extracts three traditional "handcrafted" features: MelSpectrogram (MEL), Short-Time Fourier Transform (STFT), and Mel-Frequency Cepstral Coefficients (MFCC). These features serve as a "domain anchor," correcting the potential biases of the general-purpose learned branch by providing the specific acoustic "fingerprints" of bird calls.

Feature Fusion Strategies

A critical component of SSL-Net is how these two disparate feature sets are combined. The researchers investigated three strategies:

  1. Fixed Fusion: A baseline approach using simple feature concatenation.
  2. Shared Fusion: An attention-inspired mechanism using a Multilayer Perceptron (MLP) to dynamically reweight features based on their contextual importance.
  3. Sampling Fusion: A stochastic approach utilizing the Gumbel-Softmax distribution. This allows the model to "sample" features during training, fostering more robust and generalized representations.

Experimental Evaluation

The framework was evaluated using the Western Mediterranean Wetland Birds (WMWB) dataset, derived from the Xeno-canto database. The dataset includes 20 bird species and over 200 minutes of audio. To test "label efficiency," the authors specifically measured performance with limited data, reducing the training set to as few as 200 labeled samples per species.

Key Results and Discussion

SSL-Net demonstrated superior performance across multiple metrics. Using the Spectral + BEATs configuration, the model achieved an accuracy of 87.29%, a significant improvement over the ResNet50 baseline of approximately 77.7%.

Beyond raw accuracy, the fusion strategies proved highly robust. Even when labeled data was halved, the synergistic approach maintained significantly higher F1-scores compared to single-branch models. The discriminative power of the fused features is clearly visible in t-SNE visualizations, where the Sampling Fusion strategy produces much tighter and more distinct clusters for different species compared to the Learned branch alone.

t-SNE visualization comparing the feature clustering of the Learned branch versus the Sampling fusion approach.

Our analysis suggests that while pretrained models provide a strong foundation, they are inherently "biased" toward the domains they were trained on. The addition of traditional spectral features acts as a necessary correction, capturing nuances that general models might overlook. Among the fusion methods, Shared Fusion emerged as a practical "sweet spot," providing substantial accuracy gains with manageable computational complexity.

Conclusion and Future Work

SSL-Net offers a general and efficient framework for bird sound classification that excels even in low-data regimes. By combining the strengths of deep learning and traditional signal processing, it provides a powerful tool for ecological monitoring.

However, some limitations remain. The dual-branch architecture increases computational overhead compared to single-branch models. Furthermore, the system's performance is tied to the quality of the underlying pretrained backbone. Future research will focus on improving the model's robustness against extreme out-of-distribution noise, such as heavy wind or industrial sounds, and exploring more lightweight backbones to facilitate deployment on edge devices for real-time field monitoring.

Summary updated

Abstract

Efficient and accurate bird sound classification is of important for ecology, habitat protection and scientific research, as it plays a central role in monitoring the distribution and abundance of species. However, prevailing methods typically demand extensively labeled audio datasets and have highly customized frameworks, imposing substantial computational and annotation loads. In this study, we present an efficient and general framework called SSL-Net, which combines spectral and learned features to identify different bird sounds. Encouraging empirical results gleaned from a standard field-collected bird audio dataset validate the efficacy of our method in extracting features efficiently and achieving heightened performance in bird sound classification, even when working with limited sample sizes. Furthermore, we present three feature fusion strategies, aiding engineers and researchers in their selection through quantitative analysis.
BibTeX
@inproceedings{yang2024ssl,
  author = {Yiyuan Yang and Kaichen Zhou and Niki Trigoni and Andrew Markham},
  title = {{SSL-Net: A Synergistic Spectral and Learning-based Network for Efficient Bird Sound Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=k4BD6vn3ji},
}
Figure