Blind Separation of Filtered Sources Using State-Space Approach

Liqing Zhang, Andrzej CichockiNeurIPS 1998 · OpenReview

Introduction

Most standard Blind Source Separation (BSS) and Independent Component Analysis (ICA) algorithms rely on a simplifying assumption: "instantaneous mixtures." They assume signals hit sensors at the exact same time without any echo or environmental distortion. However, in real-world scenarios—such as telecommunications, brain activity monitoring, or acoustic "cocktail party" environments—signals are typically filtered or convolved. They arrive at different times and are distorted by the environment.

While Finite Impulse Response (FIR) filters can model some of these effects, their capabilities are limited. This paper introduces a robust framework for BSS in dynamic environments by leveraging state-space models. State-space models are significantly more powerful because they can represent a wider range of systems (including AR, MA, and ARMA models) and provide an internal description of the system's dynamics.

Methodology: A Two-Pronged Approach

Training state-space models for BSS is notoriously difficult because both the original source signals and the internal state of the mixing system are unknown. To overcome this, the authors propose a demixing model based on a stable linear state-space system and uniquely split the learning process into two distinct procedures: separation and state estimation.

Block diagram of the proposed state-space demixing model.

1. Separation (Updating Matrices)

To update the output matrices ($C$ and $D$), the algorithm minimizes the mutual information between output signals. The authors derive a novel learning algorithm using a search direction inspired by Amari’s natural gradient descent. This optimization ensures that the system's outputs become as statistically independent as possible.

2. State Estimation and "Hidden Innovation"

To estimate the internal state ($x$) of the system, the authors employ a Kalman filter. Traditionally, a Kalman filter requires an "innovation" or residual signal—the difference between the actual output and the predicted output. Because the "true" output (the original source signal) is unknown in blind separation, the authors introduce a breakthrough concept called "Hidden Innovation."

Hidden innovation treats the change in the output resulting from the matrix updates as a proxy for the error. By defining innovation based on the adjustment direction of the demixing system rather than a known ground truth, the Kalman filter can function without a reference signal. This bridges a critical gap between unsupervised learning (BSS) and supervised control theory.

Experimental Setup and Key Results

The authors demonstrated the effectiveness of their approach through synthetic numerical simulations. They modeled a 3-input, 3-output (MIMO) system with 10th-order filters, using independent, identically distributed (i.i.d.) source signals and additive Gaussian noise. This synthetic setup was designed to be significantly more challenging than standard instantaneous mixture problems.

After 3000 iterations, the system successfully recovered the source signals. The most compelling evidence of this success is seen in the "global transfer function"—the mathematical combination of the mixing and demixing systems.

Impulse responses of the global transfer function after convergence.

As shown in the figure above, the global transfer function converged toward a permutation matrix multiplied by a diagonal scaling matrix. In visual terms, the plots of the global transfer function coefficients show sharp, isolated peaks at specific lags. This indicates that the system successfully learned to "undo" the complex filtering and shuffling, leaving only the original signals (albeit potentially in a different order or volume).

Limitations and Future Work

While highly effective, the proposed state-space approach has a few limitations:

  • Theoretical Proofs: Rigorous mathematical proofs for the convergence and stability of the "hidden innovation" approach remain to be fully elaborated.
  • Matrix Initialization: The algorithm currently assumes that the state matrices $A$ and $B$ are either known or designed appropriately (e.g., using a canonical controller form). A fully "blind" system would ideally learn these internal dynamics from scratch.
  • Computational Complexity: Running a Kalman filter and natural gradient updates simultaneously is more resource-intensive than standard ICA. This complexity might limit its immediate application in very high-dimensional, real-time environments without further optimization.

Despite these limitations, decoupling matrix learning from state estimation allows the algorithm to remain computationally efficient relative to its complexity, avoiding the stability issues often found in "all-in-one" adaptive filters. The state-space approach presents a highly generalized and promising solution for complex deconvolution tasks.

Summary updated

Abstract

In this paper we present a novel approach to multichannel blind separation/generalized deconvolution, assuming that both mixing and demixing models are described by stable linear state-space sys(cid:173) tems. We decompose the blind separation problem into two pro(cid:173) cess: separation and state estimation. Based on the minimization of Kullback-Leibler Divergence, we develop a novel learning algo(cid:173) rithm to train the matrices in the output equation. To estimate the state of the demixing model, we introduce a new concept, called hidden innovation, to numerically implement the Kalman filter. Computer simulations are given to show the validity and high ef(cid:173) fectiveness of the state-space approach. 1
BibTeX
@inproceedings{zhang1998blind,
  author = {Liqing Zhang and Andrzej Cichocki},
  title = {{Blind Separation of Filtered Sources Using State-Space Approach}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1998},
  url = {https://openreview.net/forum?id=rkVWidW_Zr},
}
Figure