Introduction
Sound Event Localization and Detection (SELD) represents a significant challenge in machine listening, requiring systems to simultaneously identify sound categories and their precise 3D spatial coordinates. A persistent bottleneck in this field is "homogeneous overlap"—the occurrence of multiple simultaneous events belonging to the same class, such as two people speaking at once. Traditional models like SELDNet often fail to distinguish these as separate entities, while modern solutions relying on Permutation Invariant Training (PIT) introduce significant computational complexity.
To address these issues, researchers from Samsung R&D Institute Poland introduced CoLoC (Conditioned Localizer and Classifier). This two-stage framework decomposes the SELD task into sequential localization and classification steps, effectively managing overlapping sounds without the overhead of PIT.
The CoLoC Framework
CoLoC employs a two-stage pipeline based on Convolutional Recurrent Neural Networks (CRNNs):
- Self-Conditioned SSG Localizer: Utilizing Sequential Set Generation (SSG), this module recursively identifies Directions of Arrival (DOAs). It predicts one location and uses it as a "hint" to find the subsequent location, continuing until a "stop" token (the origin) is reached. This recursive nature allows the model to dynamically handle an arbitrary number of sound sources.
- Location-Conditioned Classifier: This stage takes the audio features alongside a specific DOA provided by the localizer to predict the class probability for that specific spatial point.
By making the localizer class-agnostic, the system focuses on the physical properties of the sound field before assigning semantic labels, which enhances its ability to handle complex overlaps.
Data Representation: Stacked-Tracks
A key innovation in training CoLoC is the "Stacked-Tracks" meta-data format. In this scheme, active sound events are organized into consistent rows, "stacking" from the bottom up. When an event concludes, higher-level events "fall" to fill the vacancy. This ensures a stable and predictable input structure for the recursive localizer.
Training Strategies and Robustness
To optimize performance and mitigate the risks of a two-stage architecture, the authors implemented several specific strategies:
- $L_{1.5}$ Loss for Localization: The authors adopted an $L_{1.5}$ norm for DOA estimation. They found that while $L_2$ loss tends to average multiple sources and $L_1$ loss can be too unstable, $L_{1.5}$ provides an ideal balance for stable training.
- Angle Perturbation: To prevent error propagation—where small inaccuracies in localization lead to classification failures—the classifier is trained with "Angle Perturbation." By randomly jittering input DOAs by 5 degrees, the classifier becomes more resilient to imperfect localizer outputs.
Experimental Results
The model was evaluated using the STARSS22 dataset of real-world recordings, supplemented by a synthetic dataset of 1,200 mixtures to ensure data density. Results indicate that CoLoC significantly outperforms the DCASE2022 baseline.
As shown in the results table, the max_ov2 configuration improved the F1-score from 21% to 33% and reduced the Localization Error ($LE_{CD}$) from 29.3° to 24.6°. While the system showed a slightly higher overall Error Rate ($ER_{20^\circ}$) due to the propagation of "phantom" source predictions, its gains in precision and localization accuracy are substantial.
Conclusion
CoLoC demonstrates that decomposing SELD into conditioned stages is a viable and efficient alternative to permutation-invariant approaches. By focusing on class-agnostic localization followed by conditioned classification, the model successfully navigates the complexities of homogeneous sound overlaps. Future research may explore hybridizing Sequential Set Generation with PIT to further reduce error propagation and eliminate false positive "phantom" sources.
Summary updated