Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise

Niklas Pfister, Sebastian Weichwald, Peter Bühlmann, Bernhard Schölkopf

TL;DR

coroICA (confounding-robust independent component analysis) extends classical ICA to multi-group and multi-subject data corrupted by hidden, stationary confounding noise. By diagonalizing differences of covariance (or auto-covariance) matrices computed within groups, it cancels out group-specific background noise and extracts a shared, stable unmixing matrix that generalizes to unseen subjects and environments.

What Problem Does This Paper Address?

Standard Independent Component Analysis (ICA) assumes observed multivariate signals are clean linear mixtures of independent sources ($X = AS$). While noisy extensions exist, they typically enforce severe restrictions: noise must be independent across time, independent identically distributed (iid), or Gaussian.

In real-world settings—such as multi-subject electroencephalography (EEG) recordings, climate time series, or audio mixtures—data naturally arrives in groups (e.g., individual subjects or recording sessions). Within each group, signals are frequently corrupted by background interference (sensor drift, room acoustics, or unobserved confounding variables) that is autocorrelated over time and mutually correlated across channels. Standard ICA algorithms conflate this stationary group-level noise with true source dynamics, resulting in biased unmixing matrices that fail to generalize across groups.

Methods

The authors introduce the model: $$X_i = A S_i + H_i$$ where $A$ is an invertible mixing matrix shared across groups, $S_i$ is a vector of mutually independent (non-stationary) source signals, and $H_i$ is a vector of confounding noise variables.

  • Group-Wise Stationary Confounding: Within any group $g$, the noise process $(H_i)_{i \in g}$ is weakly stationary, meaning its covariance and auto-covariance remain constant over time within that group.
  • Differencing Trick: Because $\text{Cov}(H_k) = \text{Cov}(H_l)$ for time points $k, l$ in the same group, subtracting covariance (or auto-covariance) matrices computed on different time blocks within the same group cancels out the confounding term entirely: $$V \left(\text{Cov}(X_k) - \text{Cov}(X_l)\right) V^\top = \text{Cov}(S_k) - \text{Cov}(S_l)$$ where $V = A^{-1}$. Since the true sources are independent, the right-hand side is purely diagonal.

  • Estimation via Joint Diagonalization: Each group is partitioned into sub-blocks. coroICA computes empirical covariance differences (either between neighbor blocks, all pairs, or block vs. complement) and jointly diagonalizes them using the non-orthogonal uwedge algorithm. The authors provide three variants:

  • coroICA (var): Uses zero-lag covariance differences (variance signals).
  • coroICA (TD): Uses auto-covariance differences across time lags $\tau \in T$ (time-dependence signals).
  • coroICA (var & TD): Combines both.
  • Theoretical Identifiability: The authors prove that as long as source variances or auto-covariances vary independently across at least three sub-blocks/groups, $A$ is uniquely identifiable up to permutation and scale, even with autocorrelated noise.
  • Causal Link: They demonstrate that the model maps directly to linear Structural Causal Models (SCMs) with feedback loops and hidden confounders ($X = BX + S + H$), where non-stationarities act as soft interventions.

Datasets

  • Synthetic Benchmarks:
  • Data Set 1: Block-wise shifting variance signals ($d = 22, n = 10^5, m = 10$ groups) under varying confounding strengths.
  • Data Set 2: GARCH-type non-stationary sources paired with either auto-regressive (AR) or iid confounding noise.
  • Antarctic Ice Core Data: 800,000-year records of $\text{CO}_2$ (EPICA Dome C, Bereiter et al., 2015) and temperature $\delta T$ (Jouzel et al., 2007), sampled every 500 years.
  • CovertAttention EEG (Data Set 3): 8 subjects, 60 channels at 200 Hz, performing covert visual attention shifts across 6 cued directions (Treder et al., 2011).
  • BCI Competition IV-2a EEG (Data Set 4): 9 subjects, 22 channels at 250 Hz, performing 4 motor imagery tasks across 2 separate recording days (Tangermann et al., 2012).

Key Insights and Ideas

  • Canceling Noise without Explicit Modeling: Rather than attempting to estimate high-dimensional noise parameters, taking intra-group differences makes the stationary noise vanish algebraically.
  • Robustness Without Efficiency Loss: While designed for confounded data, coroICA remains competitive with standard second-order methods (like choiICA or SOBI) even when no confounding is present.
  • Generalization as Invariance: In multi-subject EEG, subject-specific artifacts act as group-wise stationary noise. coroICA filters out these idiosyncrasies, isolating shared neural generators.
  • Principled Causal Inference: Connecting ICA to structural vector autoregression (SVAR) allows estimating causal effects (e.g., Equilibrium Climate Sensitivity) under hidden stationary confounding that would otherwise bias ordinary LiNGAM or VAR approaches.

Key Results

  • Synthetic Experiments: As confounding noise increased from 0.125 to 3.0, the Minimum Distance (MD) error of competing methods (fastICA, choiICA) degraded severely (MD $\approx 0.5$ to $0.8$), while coroICA stayed below $0.15$. Under autocorrelated noise, only coroICA (var & TD) reliably recovered the true mixing across all signal types.
  • Climate Science: Applying coroICA to the $\text{CO}_2$–temperature ice core data yielded Equilibrium Climate Sensitivity estimates largely contained within the IPCC's "likely" range (1.5°C to 4.5°C), whereas competing ICAs fluctuated outside plausible physical bounds depending on lag selection.
  • EEG Cross-Subject Stability & Decoding: On held-out subjects, coroICA’s Mean Covariance Instability Score (MCIS) steadily declined as more training subjects were added. When features from unmixed sources were fed to downstream classifiers, coroICA achieved superior out-of-sample classification accuracy on both EEG benchmarks as the training pool grew (e.g., reaching ~40% accuracy on 4-class motor imagery across 8 training subjects).

Limitations and Future Work

  • Non-Stationarity Requirement: Sources must undergo sufficient changes in variance or autocorrelation; the method cannot separate purely stationary independent sources.
  • Stationary Confounding Assumption: The noise must remain weakly stationary within each designated group. Abrupt within-block noise shifts require manual or automated segmentation.
  • Hyperparameter Sensitivity: Partition size and the maximum lag $\tau$ must be chosen sensibly based on signal duration and autocorrelation decay.
  • Computational Complexity: Pairwise matrix differences scale quadratically with partition count ($|\mathcal{P}_g|^2$), though the authors offer linear alternatives (neighbor-only and complement differencing) to mitigate runtime.

Summary updated

Abstract

We introduce coroICA, confounding-robust independent component analysis, a novel ICA algorithm which decomposes linearly mixed multivariate observations into independent components that are corrupted (and rendered dependent) by hidden group-wise stationary confounding. It extends the ordinary ICA model in a theoretically sound and explicit way to incorporate group-wise (or environment-wise) confounding. We show that our proposed general noise model allows to perform ICA in settings where other noisy ICA procedures fail. Additionally, it can be used for applications with grouped data by adjusting for different stationary noise within each group. Our proposed noise model has a natural relation to causality and we explain how it can be applied in the context of causal inference. In addition to our theoretical framework, we provide an efficient estimation procedure and prove identifiability of the unmixing matrix under mild assumptions. Finally, we illustrate the performance and robustness of our method on simulated data, provide audible and visual examples, and demonstrate the applicability to real-world scenarios by experiments on publicly available Antarctic ice core data as well as two EEG data sets. We provide a scikit-learn compatible pip-installable Python package coroICA as well as R and Matlab implementations accompanied by a documentation at https://sweichwald.de/coroICA/
BibTeX
@article{pfister2019robustifying,
  author = {Niklas Pfister and Sebastian Weichwald and Peter Bühlmann and Bernhard Schölkopf},
  title = {{Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise}},
  journal = {J. Mach. Learn. Res.},
  year = {2019},
  url = {https://openreview.net/forum?id=fGECpbH1qI},
}

Loading related content...

Figure