Cosmopolite Sound Monitoring (CoSMo): A Study of Urban Sound Event Detection Systems Generalizing to Multiple Cities

Florian Angulo, Slim Essid, Geoffroy Peeters, Christophe Mietlicki

Introduction

Urban noise pollution is a significant public health concern. Sound Event Detection (SED) systems aim to automatically identify noise sources like sirens or construction. However, models trained on one city's data (e.g., New York) often fail when deployed in another (e.g., Singapore) due to domain shift. The CoSMo framework investigates how these systems generalize across diverse urban environments, focusing on the impact of acoustic proximity and polyphony.

Methodology

The study evaluates three architectures:

  1. CRNN: A standard baseline using convolutional layers and a Gated Recurrent Unit (GRU).
  2. PaSST: A vision transformer-based model pretrained on AudioSet with supervision.
  3. openL3: A CNN-based model pretrained on AudioSet using unsupervised audio-visual correspondence.

Researchers also compared audio frontends, specifically standard log-mel spectrograms versus Per-Channel Energy Normalization (PCEN), to test robustness against background noise.

Datasets and Proximity Annotation

The study utilizes two primary datasets: SONYC-UST-v2 (NYC) and SINGA:PURA (Singapore). A key contribution is the manual re-annotation of SONYC for "proximity" (near vs. far sounds), allowing for a detailed analysis of how distance affects detection accuracy.

Baseline Performance and Acoustic Factors

F1-score performance on the SONYC dataset across different acoustic conditions and models. Cosmopolite Sound Monitoring (CoSMo): A Study of Urban Sound Event Detection Systems Generalizing to Multiple Cities, page 5

F1-score performance on the SONYC dataset across different acoustic conditions and models.

As shown in Figure 4, models consistently struggle with distant ("far") sounds and high polyphony environments. In the NYC test set, "near" sounds were detected with significantly higher F1-scores compared to "far" sounds. While the CRNN and openL3 models perform competitively, the supervised PaSST model shows lower robustness in these challenging conditions.

Generalization to New Urban Environments

The "generalization gap" is most evident when testing NYC-trained models on Singapore data. Relative F1-score when evaluating on SINGA:PURA with different training configurations compared to target-domain training. Cosmopolite Sound Monitoring (CoSMo): A Study of Urban Sound Event Detection Systems Generalizing to Multiple Cities, page 5

Relative F1-score when evaluating on SINGA:PURA with different training configurations compared to target-domain training.

Figure 5 illustrates that models trained solely on SONYC (NYC) suffer a significant performance drop when evaluated on SINGA:PURA (Singapore). Interestingly, unsupervised openL3 embeddings proved more robust than supervised PaSST embeddings during this cross-city transfer. Semi-supervised learning using unlabeled data from the target city (SONYC + SINGA:PURA unlabelled) provided only marginal improvements, highlighting the difficulty of unsupervised domain adaptation in complex urban soundscapes.

Conclusion

Current SED systems face significant hurdles in cross-city deployment. Distance and polyphony remain the primary drivers of performance degradation. Future work should explore unsupervised sound source separation and more advanced domain adaptation techniques to bridge the gap between diverse urban soundscapes and build truly "cosmopolite" monitoring systems.

Summary updated

Abstract

Measuring noise in cities and automatically identifying the corresponding sound sources are a crucial challenge for policymakers. Indeed, such information helps addressing noise pollution and improving the well-being of urban dwellers. In recent years, researchers have provided annotated datasets recorded in two major cities to foster the development of urban sound event detection (SED) systems. This paper presents an in-depth study of the behaviour of state-of-the-art SED systems well suited to our problem, combining three far-field real recordings datasets which can be used jointly during training. In our evaluation, we highlight the performance gaps existing between simple and hard recording examples based on the salience of sound events and the polyphony of the recordings. We provide new proximity annotations for this analysis. We evaluate the ability of urban SED systems to generalize across cities with varying degrees of training supervision. We show that such generalization is hindered mostly by the difficulties current urban SED systems have to detect sound events with low salience along with sound events in highly polyphonic soundscapes.
BibTeX
@inproceedings{angulo2023cosmopolite,
  author = {Florian Angulo and Slim Essid and Geoffroy Peeters and Christophe Mietlicki},
  title = {{Cosmopolite Sound Monitoring (CoSMo): A Study of Urban Sound Event Detection Systems Generalizing to Multiple Cities}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095833},
}
Figure