Detection and positioning of overlapped sounds in a room environment

Rupayan Chakraborty, Climent Nadeu, Taras ButkoInterspeech 2012

TL;DR

This paper tackles the challenge of identifying and locating multiple overlapping sounds in a room, a common issue in smart environments. They compare two approaches: a model-based method and a signal-separation-based method, finding the former slightly better for detection. They also introduce a novel position assignment system to link detected sounds to their locations, outperforming signal-level fusion with a likelihood-based approach.

What Problem Does This Paper Address?

In smart environments like meeting rooms, understanding the acoustic scene is crucial for applications ranging from human-computer interaction to security. A major hurdle is dealing with overlapped acoustic events (AEs) – sounds that occur simultaneously. This makes it difficult to accurately detect what is happening and where it's happening. Existing systems often struggle when multiple sounds overlap, leading to detection errors. The paper aims to improve both the detection of these overlapping sounds and the subsequent assignment of detected sounds to their correct spatial locations.

Methods

The authors explore two main strategies for Acoustic Event Detection (AED) in the presence of overlapping sounds:

  • Model-Based AED: This approach requires training separate models for every possible combination of overlapping events. While it avoids explicit signal separation, it can lead to a combinatorial explosion of models if many event types and simultaneous sources are involved. For their specific scenario (one speech source overlapping with one other AE), this approach is manageable.
  • Signal Separation-Based AED: This strategy first separates the mixed audio signals into individual source signals, then applies an AED system to these cleaner signals. They investigate two separation techniques:
  • Blind Source Separation (BSS) using a deflation method: This is a more complex, iterative technique that aims to extract source signals without prior knowledge of the sources or mixing process. It's computationally intensive but expected to yield better separation.
  • Null Steering Beamforming (NSB): This is a simpler array processing technique that adapts the microphone array's directional pattern to steer towards a desired source and place "nulls" (directions of minimal sensitivity) towards interfering sources. It requires prior knowledge of source directions, often provided by an Acoustic Source Localization (ASL) system.

For Position Assignment (PA), they propose a system that takes the detected AE identity and the estimated source positions from an ASL system and resolves the ambiguity of which sound came from which position. This system uses two NSBs in parallel: one designed to nullify the speech source and the other to nullify the non-speech AE. The outputs of these beamformers are then fed into a binary classifier that uses a likelihood ratio to make the assignment. They compare two methods for combining information from different microphone pairs: signal-level fusion and classifier-level (likelihood) fusion.

Datasets

The authors used a custom database recorded in their smart-room environment at Universitat Politècnica de Catalunya (UPC). This database consists of:

  • Isolated acoustic events: Used for training, development, and testing. They recorded up to 8 sessions of audio data with isolated AEs, using all 24 microphones from the 6 T-shaped arrays.
  • Overlapped acoustic events: Generated by adding speech signals to isolated AE signals. The speech signal was mixed with the AE signal at a random starting point, and the mean power of the speech was matched to that of the AE.

The data was partitioned into training (7 sessions) and testing (1 session) sets, following previous work [1][3].

Key Insights and Ideas

  • Model-based AED is surprisingly effective for simpler scenarios: Despite the potential for combinatorial explosion, the model-based approach achieved the highest detection accuracy in their specific meeting-room scenario (one AE overlapping with speech). This suggests that for well-defined, limited overlapping scenarios, it can be a strong contender.
  • NSB offers a good trade-off for separation: The Null Steering Beamformer, despite being simpler and less computationally demanding than BSS, performed only slightly worse than the BSS system in terms of detection accuracy. This highlights its practical utility for real-time applications.
  • Likelihood fusion is superior for position assignment: Combining information from different microphone pairs at the classifier (likelihood) level proved to be more effective than combining them at the signal level for resolving the position assignment ambiguity. This suggests that decisions based on higher-level evidence are more robust.
  • The same NSB can serve dual purposes: The NSB technique, initially used for signal separation in AED, is effectively repurposed as a core component of the position assignment system, demonstrating its versatility.

Key Results

  • AED Performance: The model-based AED system achieved the highest average detection accuracy (89%). The BSS-based system followed with 82%, and the NSB-based system with 81%.
  • Position Assignment Performance: The PA system using 3 microphones and classifier-level likelihood combination achieved the highest Position Assignment Rate (PAR) of 94%. This was notably better than signal-level combination (91%) and the 2-microphone configurations (83-90%).
  • Diff_LL Metric: The frequency-domain NSB system, while having a lower PAR (83%), showed a significantly higher Diff_LL (2.64) compared to the time-domain 2-mic system (1.51), indicating better signal separation when assignments were correct. The 3-mic, likelihood combination system also achieved a high Diff_LL of 3.84.

Limitations and Future Work

  • Model-based AED scalability: The paper acknowledges that the model-based approach becomes "unfeasible when the number of classes and the number of simultaneous sources are large." This is a significant limitation for more complex acoustic environments.
  • ASL accuracy dependency: The position assignment system relies on accurate source positions provided by the ASL system. Errors in ASL would directly impact PA performance.
  • NSB frequency tuning: For the time-domain NSB, an exhaustive search for optimal frequency tuning was performed. While effective, this process can be time-consuming. The frequency-domain approach avoids this tuning but yielded slightly lower PAR.
  • Future work: The paper implicitly suggests exploring more scalable AED methods for complex scenarios and further optimizing the NSB for better performance across different frequency ranges. The success of the likelihood fusion in PA also points towards further investigation of decision-level combination strategies.

Summary updated

BibTeX
@inproceedings{chakraborty2012detection,
  author = {Rupayan Chakraborty and Climent Nadeu and Taras Butko},
  title = {{Detection and positioning of overlapped sounds in a room environment}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure