A Joint Diagonalization Based Efficient Approach to Underdetermined Blind Audio Source Separation Using the Multichannel Wiener Filter

Nobutaka Ito, Rintaro Ikeshita, Hiroshi Sawada, Tomohiro Nakatani

Introduction

Blind Source Separation (BSS) is a fundamental challenge in audio signal processing, often described as the "cocktail party problem." The goal is to isolate individual sound sources, such as specific voices or instruments, from a set of mixed recordings captured by microphones. While Independent Component Analysis (ICA) provides an efficient solution for "determined" cases—where the number of microphones equals the number of sources—real-world scenarios are frequently "underdetermined," involving more sources than available sensors.

The Computational Bottleneck in Underdetermined BSS

In underdetermined BSS, high-quality separation is typically achieved through methods like Full-rank Covariance Analysis (FCA). FCA models the spatial characteristics of sources using full-rank spatial covariance matrices (SCMs). However, the primary bottleneck of FCA is its reliance on the Multichannel Wiener Filter (MWF). Calculating the MWF requires inverting a matrix for every time-frequency point in a spectrogram. For a standard 8-second audio clip, this translates to over 120,000 matrix inversions per iteration, rendering the algorithm computationally prohibitive for large-scale or real-time applications.

FastFCA: Leveraging Joint Diagonalization

To address this inefficiency, the authors propose FastFCA. The core innovation is the introduction of a Joint Diagonalization (JD) constraint. By restricting the SCMs of all sources to a class that can be jointly diagonalized by a single matrix, the complex task of matrix inversion is simplified into scalar operations.

Signal flow in FastFCA showing the transformation to a decorrelation domain where Wiener filtering becomes a scalar operation.

As shown in the signal flow above, the process involves transforming the observed mixtures into a decorrelation domain. In this domain, the multichannel filtering reduces to one-channel Wiener filters (WFs), which are significantly faster to compute. The authors developed two optimization schemes for this model: Iterative Projection with Expectation-Maximization (IP+EM) and Iterative Projection with Majorization-Minimization (IP+MM).

The BSS Cube Framework

The paper introduces a theoretical framework called the "BSS Cube" to categorize and relate various BSS methods. This framework illustrates how different algorithms emerge by toggling three specific constraints:

  1. Joint Diagonalizability: Whether the SCMs share a common diagonalizing matrix.
  2. Rank-1 SCMs: Whether the spatial model is simplified to a rank-1 approximation.
  3. NMF-based Source Modeling: Whether Non-negative Matrix Factorization is used to model the spectral structure.
The BSS Cube illustrating the relationship between various BSS methods based on constraints like joint diagonalizability and rank.

FastFCA occupies a unique position in this cube, bridging the gap between computationally efficient determined methods (like ICA) and flexible but slow underdetermined methods (like FCA). It maintains the capability for underdetermined separation while achieving computational efficiency through the JD constraint.

Experimental Evaluation

The researchers evaluated FastFCA using 8-second English speech signals recorded with a 3-microphone array in rooms with varying reverberation times ($RT_{60}$ from 130ms to 440ms).

Key Results:

  • Efficiency: FastFCA achieved a massive speedup, reducing the Real-Time Factor (RTF) by orders of magnitude compared to standard FCA.
  • Separation Quality: The Signal-to-Distortion Ratio (SDR) of FastFCA remained comparable to the much slower FCA. Interestingly, the JD constraint acted as a regularizer, sometimes leading to lower estimation errors for spatial covariance by preventing overfitting to noise.
  • Algorithm Comparison: The IP+MM variant generally outperformed the IP+EM version in low-reverberation environments.

Conclusion and Future Work

FastFCA represents a significant step forward in making underdetermined BSS practical. By transforming a matrix-heavy bottleneck into scalar divisions via joint diagonalization, it offers a high-performance, low-latency alternative to traditional FCA.

Future research directions include addressing the sensitivity of the algorithm to initialization, exploring non-Gaussian source distributions to further improve separation quality, and investigating methods to accelerate the overall convergence of the iterative updates.

Summary updated

Abstract

Blind source separation (BSS) of audio signals aims to separate original source signals from their mixtures recorded by microphones. The applications include automatic speech recognition in a noisy/multi-speaker environment, hearing aids, and music analysis. Independent component analysis (ICA) can perform BSS efficiently, but it is basically inapplicable to the underdetermined case-the number of sources > the number of microphones. In contrast, a BSS approach using the multichannel Wiener filter (MWF) is applicable even to the underdetermined case, but conventional methods based on this approach-including full-rank spatial covariance analysis (FCA)-are highly inefficient. This is because these methods require massive numbers of matrix inversions to design the MWF. To obtain the best of both worlds, we take a joint diagonalization approach: We restrict spatial covariance matrices of all sources to the class of jointly diagonalizable matrices. This enables the above matrix inversions to be replaced by mere scalar inversions of the diagonal elements of diagonal matrices. Based on this, we present FastFCA and FastMNMF-efficient methods for underdetermined BSS. In an experiment, FastFCA was several orders of magnitude faster than FCA without sacrificing separation performance. We also present a unified framework for underdetermined and determined BSS, which highlights theoretical connections between various methods including ours. The efficiency of our BSS methods makes them suitable for large data (e.g., data augmentation for machine learning) or limited computational resources encountered in, e.g., hearing aids, distributed microphone arrays, and online BSS.
BibTeX
@article{ito2021joint,
  author = {Nobutaka Ito and Rintaro Ikeshita and Hiroshi Sawada and Tomohiro Nakatani},
  title = {{A Joint Diagonalization Based Efficient Approach to Underdetermined Blind Audio Source Separation Using the Multichannel Wiener Filter}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2021},
  doi = {10.1109/taslp.2021.3079815},
}
Figure