Large‐scale analysis of frequency modulation in birdsong data bases

Dan Stowell, Mark D. Plumbley

TL;DR

This paper introduces and evaluates four signal processing methods for analyzing fine-grained frequency modulation (FM) in birdsong, demonstrating that even simple methods can robustly extract species-identifying information. The findings suggest that FM features, particularly those capturing extremes, offer complementary information to traditional frequency measures and can be efficiently extracted for large-scale bioacoustic studies.

What Problem Does This Paper Address?

Birdsong is rich with frequency modulation (FM), believed to play crucial roles in adaptation to acoustic environments and social signaling. However, standard audio analysis techniques like the Short-Time Fourier Transform (STFT) often obscure the rapid, fine-temporal details of this FM. This makes it challenging to leverage large birdsong databases for detailed bioacoustic research. The paper aims to bridge this gap by exploring and comparing high-resolution signal processing methods capable of capturing these subtle FM nuances and assessing their suitability for large-scale, automated analysis.

Methods

The authors investigated four signal processing methods for extracting FM information:

  • ss (Simple Spectrographic): A method related to manual spectrographic analysis but automated. It identifies the peak energy frequency in each frame and calculates the magnitude of the frequency jump between successive frames, summarizing this with statistics like the median.
  • rm (Ring Modulation/Heterodyne Chirplet): A chirplet analysis method that leverages peak-energy detection to capture local modulations.
  • mp (Matching Pursuit): An implementation of the Matching Pursuit technique, designed for sparse signal decomposition.
  • dd (Distribution Derivative Method): A method that uses peak-energy sinusoid detection to extract FM information.

These methods were applied to extract "frames" or "atoms" containing frequency and FM rate information. Summary statistics (median, 5th, 75th, and 95th percentiles) were then computed for each recording to create feature sets. The effectiveness of these features was evaluated using information gain for feature selection and Support Vector Machines (SVM) for species classification. Robustness to noise and MP3 compression was also tested.

Datasets

The study utilized three datasets:

  • PhyllASA: A curated dataset of 45 recordings from 5 species within the Phylloscopus genus, in WAV format.
  • PassaASA: A broader dataset of 762 recordings from 84 species across the order Passeriformes, in WAV format.
  • PhyllXC: A large dataset of 1390 recordings from 56 Phylloscopus species, sourced from the Xeno-Canto archive. This dataset is notable for being "crowdsourced" and primarily in MP3 format, presenting challenges related to varying quality and compression artifacts.

Key Insights and Ideas

  • Simplicity can be powerful: The simplest method tested ("ss") performed as well as, or better than, more complex methods in terms of classification accuracy and robustness. This suggests that advanced signal processing isn't always necessary for extracting meaningful bioacoustic information.
  • FM extremes matter: Features representing the extremes of FM (75th and 95th percentiles) were found to be particularly important for species identification, offering information complementary to basic frequency measures.
  • Bandwidth is less informative: Contrary to expectations, bandwidth features showed very low predictive power for species identity in this study.
  • Robustness is key for large datasets: The "ss", "rm", and "dd" methods demonstrated strong robustness to added noise and MP3 compression, making them suitable for analyzing real-world, potentially degraded audio archives.
  • Automated analysis is feasible: The methods are designed for full automation, eliminating the need for manual syllable segmentation, which is crucial for processing large volumes of data.

Key Results

  • Classification Performance: The "FM+Frequency features" set achieved the highest classification performance (average AUC around 70%), with the "Top-2" features (median frequency and 75th percentile FM) also performing strongly.
  • Method Performance: The "ss" and "dd" methods tied for the best classification results, outperforming "rm" and "mp".
  • Speed: The "ss" method was significantly faster, running at least 20 times faster than other methods, making it highly practical for large-scale analysis.
  • Robustness: "ss", "rm", and "dd" features showed high correlation (>0.95) with original features after noise and MP3 degradation. "rm" was particularly robust. The "mp" method's features were found to be unreliable under degradation.

Limitations and Future Work

  • Subspace Method Inappropriateness: The subspace method of Badeau et al. (2006) was found to be inappropriate for rapid FM modulations in birdsong due to its assumption of smooth FM variation.
  • Limited Feature Set: The study focused on frequency and FM characteristics, excluding macro-level features like syllable lengths or rates, which are more complex to extract automatically.
  • Future Directions: The authors suggest combining their automated FM extraction with richer temporal modeling, advanced segmentation techniques, and physiological models of vocal production. They also highlight the potential for combining FM analysis with physiological models of the songbird vocal production mechanism.

Summary updated

Abstract

Summary Birdsong often contains large amounts of rapid frequency modulation (FM). It is believed that the use or otherwise of FM is adaptive to the acoustic environment and also that there are specific social uses of FM such as trills in aggressive territorial encounters. Yet temporal fine detail of FM is often absent or obscured in standard audio signal analysis methods such as Fourier analysis or linear prediction. Hence, it is important to consider high‐resolution signal processing techniques for analysis of FM in bird vocalizations. If such methods can be applied at big data scales, this offers a further advantage as large data sets become available. We introduce methods from the signal processing literature which can go beyond spectrogram representations to analyse the fine modulations present in a signal at very short time‐scales. Focusing primarily on the genus Phylloscopus , we investigate which of a set of four analysis methods most strongly captures the species signal encoded in birdsong. We evaluate this through a feature selection technique and an automatic classification experiment. In order to find tools useful in practical analysis of large data bases, we also study the computational time taken by the methods, and their robustness to additive noise and MP3 compression. We find three methods which can robustly represent species‐correlated FM attributes and can be applied to large data sets, and that the simplest method tested also appears to perform the best. We find that features representing the extremes of FM encode species identity supplementary to that captured in frequency features, whereas bandwidth features do not encode additional information. FM analysis can extract information useful for bioacoustic studies, in addition to measures more commonly used to characterize vocalizations. Further, it can be applied efficiently across very large data sets and archives.
BibTeX
@inproceedings{stowell2014largescale,
  author = {Dan Stowell and Mark D. Plumbley},
  title = {{Large‐scale analysis of frequency modulation in birdsong data bases}},
  booktitle = {Methods},
  year = {2014},
  doi = {10.1111/2041-210x.12223},
}
Figure