Robust Multicomponent Tracking of Ultrasonic Vocalizations

Reyhaneh Abbasi, Nicki Holighaus, Péter Balázs, Vincent Lostanlen, Dustin J. Penn, Sarah M. Zala2025 · DOI

TL;DR

This paper introduces a novel method for robustly tracking ultrasonic vocalizations (USVs) in mice, even in noisy conditions. By combining advanced time-frequency analysis techniques like multitaper reassignment with empirical Wiener shrinkage and a dual Short-Time Fourier Transform (STFT) and Constant-Q Transform (CQT) approach, the method accurately identifies both the fundamental frequency and its harmonic partials, crucial for understanding mouse communication.

What Problem Does This Paper Address?

Ultrasonic vocalizations (USVs) are a vital communication channel for mice, carrying information about their identity and emotional state. Analyzing these USVs is key to understanding their social behavior and reproductive success. However, USV recordings often suffer from low signal-to-noise ratios (SNR), making it challenging to accurately extract their spectro-temporal features. Existing methods primarily focus on tracking the fundamental frequency and struggle with noisy data or the detection of harmonic partials, which are also important for communication. This paper aims to bridge this gap by providing a more robust and comprehensive method for USV ridge tracking, capable of identifying both the fundamental frequency and its harmonics even in challenging acoustic environments.

Methods

The proposed method builds upon time-frequency reassignment (TFR), a technique that sharpens time-frequency representations by re-allocating energy based on local instantaneous frequency and group delay. To enhance robustness to noise, the authors incorporate two key improvements:

  • Empirical Wiener Shrinkage: This pre-processing step helps to denoise the audio signal in the STFT domain, reducing the impact of background noise.
  • Multitaper Reassignment: Instead of using a single window for TFR, this technique employs multiple orthogonal windows (tapers). Averaging the reassigned spectrograms from these tapers emphasizes deterministic signal components while suppressing random noise.

Furthermore, the method utilizes two different time-frequency representations:

  • Short-Time Fourier Transform (STFT): Used for initial denoising and fundamental frequency ridge estimation.
  • Constant-Q Transform (CQT): This transform is particularly well-suited for analyzing harmonic tones due to its logarithmic frequency scale and frequency-dependent resolution. The CQT is used in conjunction with autocorrelation analysis to detect harmonic partials. The equivariance property of the CQT under frequency transposition allows for the detection of harmonic components independently of the fundamental frequency.

The overall workflow involves denoising, computing multitaper reassigned STFTs for fundamental frequency ridge estimation, and then computing multitaper reassigned CQTs followed by autocorrelation for harmonic partial detection.

Datasets

The study utilizes USVs recorded from adult wild-derived house mice (Mus musculus musculus) using the RECORDER USGH-software at a sampling rate of 300 kHz. The USVs analyzed range from 30 kHz to 120 kHz and have durations between 3 ms and 200 ms. These recordings were previously classified by biologists into nine distinct categories (UP, D, C2, C3, C, U, UI, F, H) based on acoustic features. For evaluation, a subset of 197 USVs were manually annotated by the authors, with initial estimates refined using a custom Python GUI.

Key Insights and Ideas

  • Harmonics Matter: The paper highlights that previous methods often ignored harmonic partials, which are present in mouse USVs and can carry important communicative information. Their method explicitly targets these harmonics.
  • Noise is the Enemy (and the Challenge): The core innovation lies in making time-frequency analysis robust to noise. The combination of Wiener shrinkage and multitapering is crucial for achieving this.
  • STFT vs. CQT Synergy: Using both STFT (for fundamental frequency) and CQT (for harmonics) leverages the strengths of each transform. The CQT's logarithmic scale is particularly advantageous for analyzing harmonic relationships.
  • Autocorrelation for Harmonics: Applying autocorrelation to CQT coefficients is an elegant way to detect repeating patterns indicative of harmonic partials.
  • Data-Driven Approach: The method aims to be largely data-driven, reducing the need for extensive manual parameter tuning, which is a significant advantage for practical applications.

Key Results

The proposed method demonstrates significant improvements in USV ridge estimation:

  • Overall Precision: Achieved the highest overall precision of 89.6% for fundamental frequency estimation, outperforming DeepSqueak and other baseline methods.
  • Frequency Deviation: Showed a low mean frequency deviation of 1.5 ± 5.1 kHz for the fundamental frequency.
  • Class-wise Precision: Exhibited substantial improvements across various USV classes, particularly in lower SNR conditions.
  • Harmonic Detection: Achieved a precision of 89% and a recall of 59% for detecting the first harmonic partial, with a mean deviation of 0.8 ± 0.15 kHz. This is a notable achievement, as harmonic detection was not addressed by prior methods.
  • Robustness to SNR: The method consistently outperformed others in low-SNR conditions, as visualized in Figure 2.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Harmonic Recall: While precision for harmonic detection is high, the recall is moderate. Future work will focus on advanced noise reduction techniques to improve harmonic detection.
  • Synthetic Data Verification: The authors plan to verify harmonic detection using synthetic data with known ground truth to address potential inaccuracies in manual annotations.
  • Integration into Classification: The ultimate goal is to integrate this improved ridge estimation into USV classification algorithms for enhanced performance.
  • Computational Efficiency: Transitioning all components to a CQT-based framework is proposed to improve computational efficiency.
  • Automated Parameter Learning: The current method relies on predefined thresholds for harmonic detection. Future work will aim to automatically learn these parameters from data, leading to a fully automated system.

Summary updated

Abstract

Ultrasonic vocalizations (USV) convey information about individual identity and arousal status in mice. We propose to track USV as ridges in the time–frequency domain via a variant of time– frequency reassignment (TFR). The key idea is to perform TFR with empirical Wiener shrinkage and multitapering to improve robustness to noise. Furthermore, we perform TFR over both the short-term Fourier transform and the constant-Q transform so as to detect both the fundamental frequency and its harmonic partial (if any). Experimental results show that our approach effectively estimates multicomponent ridges with high precision and low frequency deviation.
BibTeX
@misc{abbasi2025robust,
  author = {Reyhaneh Abbasi and Nicki Holighaus and Péter Balázs and Vincent Lostanlen and Dustin J. Penn and Sarah M. Zala},
  title = {{Robust Multicomponent Tracking of Ultrasonic Vocalizations}},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10890025},
}
Figure