Introduction
Sound Event Localization and Detection (SELD) is a critical task in machine hearing that requires a system to simultaneously identify the type of a sound (e.g., a dog barking) and its spatial origin (Direction-of-Arrival, or DoA). While significant progress has been made, traditional class-based SELD systems encounter a fundamental limitation: they struggle to distinguish between multiple overlapping instances of the same sound class, such as two people speaking simultaneously from different locations. This is often referred to as the "double talker" problem.
To address this, researchers from the University of Surrey and ByteDance have developed a novel Event-Independent Network. This architecture shifts the paradigm from class-based prediction to a track-based approach, allowing the system to handle complex polyphonic scenarios where sound classes overlap.
Methodology
The proposed system moves away from predicting specific classes directly and instead adopts an end-to-end, track-based prediction framework. The architecture is defined by three primary components:
- Feature Extraction: The network processes First-Order Ambisonics (FOA) signals. It utilizes a 1-D convolutional layer to extract log-mel spectrograms for identity and intensity vectors for spatial localization directly within the model pipeline.
- Track-Wise Prediction and tPIT: Instead of assigning outputs to fixed classes, the network predicts activity on generic "tracks." To solve the track permutation problem—where a sound might jump between tracks across different frames—the authors implement frame-level Permutation Invariant Training (tPIT). This strategy calculates loss for all possible track assignments and optimizes for the most consistent one, forcing the network to maintain stable track identities.
- Event Activity Detection (EAD) Branch: A dedicated EAD branch integrates features from both the Sound Event Detection (SED) and DoA branches. This branch acts as a gatekeeper, improving the precision of event onset and offset detection by leveraging both spectral identity and spatial consistency.
Experimental Results
The model was rigorously evaluated using the TAU-NIGENS Spatial Sound Events 2020 dataset, which includes 700 samples across 14 classes with realistic reverberation and moving sources. The "Event-Ind" model was compared against several DCASE 2020 baselines and track-wise ablation studies.
As shown in the performance metrics, the Event-Independent approach significantly outperformed the baselines. It achieved the lowest Error Rate ($ER_{20^\circ}$) and highest F-score ($F_{20^\circ}$), while also showing superior localization accuracy ($LE_{CD}$) and recall ($LR_{CD}$). The results confirm that the combination of tPIT and the EAD branch is essential for accurately resolving polyphonic sound events.
Conclusion
The Event-Independent Network provides a robust solution to the challenges of polyphonic SELD. By decoupling detection from specific classes and utilizing track-based prediction with tPIT, the system can effectively localize and identify overlapping sounds of the same type. Future work will look toward extending the system to handle more than two simultaneous tracks and optimizing the architecture for real-time, low-latency applications on edge devices.
Summary updated