Towards a Generalization of Relative Transfer Functions to More Than One Source

Antoine Deleforge, Sharon Gannot, Walter Kellermann

Abstract

We propose a natural way to generalize relative transfer functions (RTFs) to more than one source. We first prove that such a generalization is not possible using a single multichannel spectro-temporal observation, regardless of the number of microphones. We then introduce a new transform for multichannel multi-frame spectrograms, i.e., containing several channels and time frames in each time-frequency bin. This transform allows a natural generalization which satisfies the three key properties of RTFs, namely, they can be directly estimated from observed signals, they capture spatial properties of the sources and they do not depend on emitted signals. Through simulated experiments, we show how this new method can localize multiple simultaneously active sound sources using short spectro-temporal windows, without relying on source separation.

Introduction

In acoustic signal processing, Acoustic Transfer Functions (ATFs) describe the transformation of sound from a source to a microphone, encapsulating spatial information such as position and reverberation. However, estimating ATFs typically requires knowledge of the original source signal, which is rarely available in practical, uncontrolled environments. To circumvent this, researchers utilize Relative Transfer Functions (RTFs). By calculating the ratio of the ATF of one microphone to another, the unknown source signal is canceled, leaving a feature dependent solely on the source's location. While effective for a single source, this approach fails when multiple sources are active simultaneously, as their signals mix in ways that prevent simple cancellation. Current solutions often necessitate computationally expensive source separation prior to localization.

Theoretical Framework

The "No-Go" Theorem

The authors provide a rigorous mathematical proof (Theorem 1) demonstrating that no non-constant function can extract multi-source spatial information from a single multichannel observation without being influenced by the source signals. This establishes that a single snapshot of audio is insufficient for generalizing RTFs to multiple sources.

Plücker Spectrogram Transform

To overcome this limitation, the paper introduces a multi-frame approach using a window of $K$ time frames, where $K$ corresponds to the number of sources. The core innovation is the Plücker spectrogram transform, which utilizes the Plücker embedding from algebraic geometry. This technique represents $K$-dimensional subspaces (the span of the sources) as vectors in a higher-dimensional space. By applying this transform, a mixture of $K$ sources can be mathematically treated as a single "compound" source. This allows for the definition of a Generalized RTF (GRTF) that cancels the complex interaction of the $K$ source signals, isolating the joint spatial characteristics of the sources.

Experimental Evaluation

The proposed method was evaluated using simulated experiments involving Head-Related Transfer Functions (HRTFs) for the four-microphone array of a NAO humanoid robot. The experiments utilized mixtures of 1, 2, and 3 white-noise sources at Signal-to-Noise Ratios (SNRs) of 50 dB (low noise) and 10 dB (high noise).

Key Results

The GRTF method demonstrated high precision in low-noise environments. At 50 dB SNR, the mean azimuth error was remarkably low, ranging from 0.04° for a single source to 1.45° for three simultaneous sources. In the three-source case, 91% of sources were perfectly localized using just 64ms of audio. Furthermore, the Matlab implementation processed three sources in approximately 436ms per second of signal, indicating potential for real-time applications.

Mean azimuth error for 1, 2, and 3 sources at different SNR levels.

However, performance is sensitive to noise. As shown in the table above, at 10 dB SNR, the mean azimuth error increased significantly, reaching 27.4° for three sources.

Conclusion and Future Work

While the GRTF method offers a powerful new way to localize multiple sources simultaneously without prior separation, several challenges remain. The current formulation is sensitive to additive noise, and the experiments assumed a known number of sources ($K$). Future research will focus on estimating the source count, addressing the "curse of dimensionality" in supervised learning contexts, and evaluating performance with sparse signals such as human speech.

Summary updated

Abstract

We propose a natural way to generalize relative transfer functions (RTFs) to more than one source. We first prove that such a generalization is not possible using a single multichannel spectro-temporal observation, regardless of the number of microphones. We then introduce a new transform for multichannel multi-frame spectrograms, i.e., containing several channels and time frames in each time-frequency bin. This transform allows a natural generalization which satisfies the three key properties of RTFs, namely, they can be directly estimated from observed signals, they capture spatial properties of the sources and they do not depend on emitted signals. Through simulated experiments, we show how this new method can localize multiple simultaneously active sound sources using short spectro-temporal windows, without relying on source separation.
BibTeX
@inproceedings{deleforge2015towards,
  author = {Antoine Deleforge and Sharon Gannot and Walter Kellermann},
  title = {{Towards a Generalization of Relative Transfer Functions to More Than One Source}},
  booktitle = {EUSIPCO},
  year = {2015},
  url = {https://openreview.net/forum?id=wObomkzo0m-o},
}
Figure