Separation Is a Riesz Energy: How Many Environments Does ICA Need?

Yuda Bi, Yubo Guo, Jiaxiao Cai, Xiaonai Li, Zhuolin Li, Vince D. Calhoun

TL;DR

This paper reveals that the difficulty of Independent Component Analysis (ICA) for Gaussian sources is governed by the Riesz 2-energy of their variance "signatures" across different environments. The authors prove a "dimension law" for sample complexity: while you need $O(d^2)$ samples for 2 environments, just 4 well-spread environments drop the requirement to $O(d)$, making high-dimensional source separation significantly more efficient.

What Problem Does This Paper Address?

Standard Independent Component Analysis (ICA) relies on sources being non-Gaussian to identify the underlying mixing matrix. If your sources are Gaussian, i.i.d. samples only allow you to recover the components up to an arbitrary rotation—a fundamental ambiguity.

However, real-world data is often non-stationary. If the variances of these Gaussian sources change across different "environments" (e.g., different time segments, datasets, or experimental conditions), the sources become identifiable. This is known as the diversity route to ICA.

While researchers knew this was possible qualitatively, the quantitative theory was missing:

  • How many environments ($K$) do we actually need?
  • How many samples ($n$) are required as the number of components ($d$) grows?
  • Which environments are "better" for separation?

This paper provides the first finite-sample, high-dimensional theory to answer these questions.

Methods

The authors model each source $j$ by its signature $p_j$, a vector representing its variance across $K$ environments. They show that the Fisher information $S_{ij}$ between two sources is proportional to the squared distance between their signatures.

Key Innovations:

  1. Riesz Energy Formulation: They define the cost of separating source $j$ as its Riesz 2-energy: $E_j = \sum_{i \neq j} S_{ij}^{-1}$. This treats source separation as a physical system where signatures "repel" each other; the closer they are in variance space, the higher the "energy" (and difficulty) of separation.
  2. The Quadratic Pencil: Instead of traditional joint diagonalization, which can be sensitive to how environments are weighted, they propose a spectral estimator called a quadratic pencil: $\sum_k w_k (\hat{\Sigma}_k - p_j(k)I)^2$. This matrix vanishes only along the direction of source $j$, allowing for precise "probing" of individual components.
  3. Minimax Theory: They establish lower bounds showing that no estimator can beat the Riesz rate $E_j/n$ and demonstrate that their quadratic pencil attains this rate.

Datasets

  • Synthetic Simulations: Extensive tests with Gaussian sources and various signature geometries (random, lattice-based, and Halton points) for dimensions up to $d=1000$.
  • Real-world fMRI: Task-based fMRI data from 813 subjects in the Human Connectome Project (HCP). They used $K=8$ environments (seven tasks plus a resting state) to separate $d=200$ temporally independent components.

Key Insights and Ideas

  • Dimension, Not Number: The most striking finding is that the dimension the signatures span matters far more than the number of environments. Adding more environments that lie on the same line doesn't help much; you need environments that spread the signatures into new dimensions.
  • The Dimension Law: The sample complexity follows a clear phase transition based on the signature space dimension ($K-1$):
  • $K=2$ (1D): Error scales as $d^2/n$.
  • $K=3$ (2D): Error scales as $d \log d / n$.
  • $K \ge 4$ (3D+): Error scales as $d/n$.
  • Geometry of Difficulty: Separation is a local problem. The difficulty is dominated by a source's nearest neighbors in the signature space. This is why the Riesz energy (which blows up at short distances) is the perfect descriptor.
Figure 1. Separation is a Riesz energy. (a) Each source is a point pi in the (K −1)-dimensional space of signatures; separating source j costs the inverse-square energy of its interactions (blue), which a quadratic pencil centred at pj measures, while a linear pencil P

Key Results

  • Sample Complexity: For $K \ge 4$ well-spread environments, the number of samples needed for a fixed error level is linear in the number of components ($n \propto d$). This is a major improvement over the $n \propto d^2$ requirement often assumed for "blind" methods.
  • Linear vs. Quadratic Pencils: The paper proves that taking a single linear combination of covariances (a "linear pencil") is fundamentally limited, paying a $d^2/n$ penalty regardless of how many environments you have. Probing signatures individually (the "quadratic" approach) avoids this bottleneck.
  • fMRI Validation: In the HCP dataset, the Riesz energy successfully ranked which brain components were hardest to separate. Components with nearly identical variance profiles across tasks (high Riesz energy) showed the lowest reliability in split-half tests.
Figure 1. Separation is a Riesz energy. (a) Each source is a point pi in the (K −1)-dimensional space of signatures; separating source j costs the inverse-square energy of its interactions (blue), which a quadratic pencil centred at pj measures, while a linear pencil P

Limitations and Future Work

  • Gaussian Assumption: The core theorems assume Gaussian sources. While experiments show the results hold for other distributions (like Laplace), the formal minimax theory for non-Gaussian "diversity" ICA is still an open area.
  • Known Signatures: The most efficient estimators require knowing the variance signatures $p_j$. While the authors provide a grid-search algorithm to find them, it can be computationally expensive in very high dimensions without a good initialization.
  • Independence: The theory assumes environments are independent. Extending this to correlated environments (like time-lags in SOBI) is a suggested next step.
  • The "Linear Regime" Conjecture: The authors conjecture that standard Jacobi-based joint diagonalization attains the Riesz rate even without knowing signatures, but a formal proof for the $K \ge 4$ case remains an open problem.

Summary updated

Abstract

Can independent components be recovered when all of them are Gaussian? Yes, if their variances change across environments; but how many environments, and how many samples as the number of components $d$ grows? We answer both for the joint diagonalization of $K$ covariances. Each component has a signature, its variances across the environments. Separating component $j$ costs the Riesz 2-energy $E_j$ of this signature, a sum, over the other components, of inverse pairwise Fisher informations. For Gaussian sources no estimator in $O(d)$ beats $cE_j/(n+E_j)$ at any $n$ and $d$, and above an explicit sample size a spectral estimator attains $E_j/n$ up to a constant, for every mixing and without the signatures (given a lower bound on the pairwise information). Since signatures live in $K-1$ dimensions, the rate obeys a dimension law: $d^2/n$, $d\log d/n$ and $d/n$ for $K=2$, $3$ and $K\ge4$ well-spread environments. What counts is the dimension the signatures span, not their number, and $\log d$ well-designed environments make $d\log d$ samples suffice. Projecting the covariances onto one linear combination pays $d^2/n$ at first order instead. In simulations up to $d=1000$ the predictions hold and joint diagonalization with $K\ge4$ attains the rate once $n/d$ is in the hundreds; on task fMRI the energy says which components are hard to separate.
BibTeX
@misc{bi2026separation,
  author = {Yuda Bi and Yubo Guo and Jiaxiao Cai and Xiaonai Li and Zhuolin Li and Vince D. Calhoun},
  title = {{Separation Is a Riesz Energy: How Many Environments Does ICA Need?}},
  year = {2026},
  eprint = {2611.06228},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06228},
}
Figure