Many sensors distort what they record. Amplifiers saturate, detectors compress, and clipping bends the signal before it is stored. When several independent signals are first mixed linearly and each sensor then applies its own unknown monotonic distortion, the result is a post-nonlinear mixture. The standard way to undo it has two steps: straighten each sensor channel back to a linear mixture, then run ordinary independent component analysis (ICA), which recovers independent sources from linear mixtures by exploiting their non-Gaussian statistics. The straightening step usually relies on a Central Limit Theorem argument: a sum of independent signals should look roughly Gaussian, so each channel is warped until it looks Gaussian, or until its entropy is maximal. That argument breaks down in the settings where ICA is most useful. With only two to five sources, and with a spiky source such as speech mixed with a flat one such as power-line hum or a digital symbol stream, the true linear mixture is far from Gaussian. Forcing it toward Gaussian adds a distortion of its own.
We propose to replace the fixed Gaussian target with a target the model predicts for itself. Each sensor's straightening map is the monotone map that sends the observed distribution of that channel onto the distribution that the current source estimates and mixing weights imply for it. That implied distribution is computed from flexible per-source densities that can be heavy-tailed, flat, skewed, or discrete. The procedure then alternates between separating, re-estimating the source densities and the implied channel distributions, and re-warping. In two sentences: Gaussianization assumes each channel is secretly Gaussian, which is wrong for small mixtures of dissimilar sources. We instead let the separated sources say what each channel should look like, and warp the channel to match.
Core Claim
For a monotone distortion $f_i$ on sensor $i$, the straightening map is fully determined by one thing: the distribution of the undistorted mixture $u_i = \sum_j a_{ij} s_j$. If $\hat E_i$ is the empirical cumulative distribution of the observed channel $x_i$ and $F_i$ is the cumulative distribution of $u_i$, then $g_i = F_i^{-1} \circ \hat E_i$ inverts $f_i$ exactly, with no parametric model of $f_i$. Gaussianization sets $F_i$ to the standard normal. Our method sets it self-consistently:
$$ g_i^{(t+1)} = \big(F_i^{(t)}\big)^{-1} \circ \hat E_i, \qquad F_i^{(t)} = \text{CDF of } \textstyle\sum_j \hat a^{(t)}_{ij}\, \hat s^{(t)}_j , $$
Here $\hat a^{(t)}_{ij}$ are the mixing weights estimated at iteration $t$ by a linear ICA run on the currently straightened data. The terms $\hat s^{(t)}_j$ are independent draws from the per-source densities fitted at that iteration, so $F_i^{(t)}$ is the distribution of a convolution of scaled source densities. We compute it by Monte Carlo or by numerical convolution.
The true mixing matrix and true source marginals form a fixed point of this map. The claim to test is threefold. The iteration reaches that fixed point from a Gaussianization or entropy-maximization starting point. Doing so improves final source recovery over both. The gain is largest exactly where the Central Limit Theorem argument is weakest: few sources, opposite kurtosis signs, skewed or discrete marginals. We also predict the reverse. As the number of mixed sources grows, the implied $F_i$ approaches Gaussian and the advantage should vanish. If it does not shrink, the gain comes from something other than the stated mechanism.
Relation to Prior Work
Two papers supply the straightening step we modify and motivate the gap. The linearize-then-separate method of Blind Separation of Post-nonlinear Mixtures using Linearizing Transformations and Temporal Decorrelation Gaussianizes each channel by rank and then separates with temporal decorrelation. That method therefore needs sources with distinct autocorrelations and fails on white sources. Inverting Monotonic Nonlinearities by Entropy Maximization (MaxEnt) maximizes output entropy under unit variance. It reports roughly +3 dB over Gaussianization and +6 dB over uniformization, but only on uniform and 3-level sources, and it leaves the linear separation step to an off-the-shelf ICA. Both methods fix the target distribution in advance. Ours re-estimates the target from the model, and it never needs temporal structure.
Four papers on heterogeneous sources supply the separation and density components but assume linear mixing. Extended ICA Removes Artifacts from Electroencephalographic Recordings switches its nonlinearity by kurtosis sign. An alternative switching criterion for independent component analysis (ICA) makes that switch cheaper. ICA based on Split Generalized Gaussian models skew and tail weight jointly. Robust Independent Component Analysis Using Quadratic Negentropy uses kernel densities. We reuse the Split Generalized Gaussian family, plus a discrete-mixture option, as the per-source density model inside the loop. That model is what lets the implied $F_i$ be non-Gaussian in the right way.
Correction: Independent Component Analysis for Brain fMRI Does Indeed Select for Maximal Independence constrains the design. It shows that Infomax and FastICA break when components approach zero excess kurtosis. This implies that any straightening step which pushes channels toward Gaussian erodes the contrast the second stage depends on.
Other nonlinear methods differ in kind. Provable Subspace Identification Under Post-Nonlinear Mixtures straightens channels without any distributional assumption, using the null space of a tall mixing matrix. It needs more sensors than sources, recovers only the subspace, and trains one network per channel. Our method works with square mixtures, where it has no competitor of that geometric kind. Deep Deterministic Nonlinear ICA via Total Correlation Minimization with Matrix-Based Entropy Functional (DDICA) learns an unrestricted deep unmixing map. That generality leaves its outputs unanchored to the physical sources. It also has no mechanism for modeling individual sources' statistics.
Research Program
We will build the alternating procedure with three interchangeable parts. The initialization is Gaussianization or MaxEnt. The linear separator is Extended Infomax, or natural-gradient ICA with a Split Generalized Gaussian likelihood. The density and convolution module fits per-source densities and computes each channel's implied distribution. Each straightening map is a monotone interpolated quantile map, so an iteration costs about one sort per channel plus one linear ICA. We expect this to be much cheaper than training a neural inverse.
On the analysis side, we will study the fixed points of the map for two sources. We want to establish when the Gaussianization fixed point is spurious, and whether it can be attracting when both sources are close to Gaussian. We will also characterize how the error in the fitted densities propagates into the straightening map. Sample-size requirements should scale with the accuracy of quantile tails, not with fourth-order cumulants, and we will test that empirically.
Baselines
The naive reference is linear ICA run directly on the distorted data, using the same Extended Infomax separator. DDICA's authors report that on strongly nonlinear mixtures FastICA and Infomax give output "indistinguishable from noise" (DDICA). This baseline sets the floor.
The first strongest competitor is MaxEnt followed by the same flexible linear ICA we use. It is the best-documented straightening method on this exact benchmark, and once it is paired with a mixed-kurtosis separator it is the obvious "just chain the two" answer. The file reports MaxEnt only as roughly +3 dB SNR over Gaussianization and +6 dB over uniformization in two-source simulations (MaxEnt). It gives no absolute post-separation SNR on Lacuna for MaxEnt combined with Extended ICA, so we will reproduce it ourselves. We will run MaxEnt with both its polynomial and neural inverses, tuned over degree and width with the same trial budget our method gets for its density family.
The second strongest competitor is joint mutual-information minimization in the style of Taleb and Jutten. MaxEnt's authors report its runtime (2.86 s against 0.28 s for MaxEnt), but no accuracy figure for it was found on Lacuna. We will implement it with the same monotone parameterization.
For contrast, we will also run DDICA with matched parameter count and training steps. On tall configurations we will run Lyu and Fu's method followed by the same linear ICA, using as reference their reported face/scrambled classification error of about 0.39, against about 0.43 for PCA and 0.46 for raw data (Lyu and Fu).
The key ablation keeps everything else fixed and freezes the target $F_i$ at Gaussian. This is iterated Gaussianization with the same separator, and it isolates the self-consistent target. A second ablation swaps the Split Generalized Gaussian density for a symmetric one, which isolates the skew modeling.
Beating the chained MaxEnt baseline means a statistically reliable gain in median per-source SNR and Amari error over 50 random mixing matrices, in the two-to-four-source, mixed-kurtosis regime, at equal wall-clock time or less.
Why not just…?
Why not just chain MaxEnt with Extended ICA? This is the real threat. If MaxEnt's entropy criterion already straightens small heterogeneous mixtures well, our loop adds cost for nothing. MaxEnt's own evidence covers uniform and 3-level sources only, which are both sub-Gaussian. Its justification is the same Central Limit Theorem heuristic we argue fails for speech-plus-hum mixtures. The source-count and kurtosis-heterogeneity sweeps settle this directly. If the chained baseline matches us at two to three sources with opposite-sign kurtosis, we will report that the self-consistent target is unnecessary.
Why not just Gaussianize? Solé-Casals et al. already measure a roughly 3 dB deficit against MaxEnt. The Calhoun et al. result explains why Gaussianized channels starve the separator of contrast. Temporal decorrelation, the fallback in the Gaussianization pipeline, cannot help with white sources, which our benchmarks include.
Why not just use a general deep nonlinear ICA like DDICA? It can find independent coordinates that are not the physical sources, and it models no per-source statistics. In a post-nonlinear setting, the structural restriction to monotone per-channel maps should be an advantage rather than a limitation. Our matched-budget comparison tests exactly this, measured by source correlation rather than by independence alone.
Why not just use the null-space method? It is provable and assumption-light, but only when sensors outnumber sources. On tall configurations it may beat us, and we will report that honestly. Our claim concerns square mixtures, which it cannot handle.
Proposed Experiments
The core benchmark reproduces MaxEnt's setup: 2×2 and 4×4 mixtures under $\tanh(\alpha x)+0.1x$ and $x^3+\beta x$ distortions. We extend it with sources drawn from Gao et al.'s five distributions (exponential, Gamma, Chi-square, Beta, Uniform) and from a subset of the 18-distribution suite used by QICA, including skewed and multimodal members. We then sweep the number of sources from 2 to 8 and vary the fraction of sub-Gaussian sources.
We report three metrics. Straightening SNR measures agreement with the true $As$ up to per-channel scale. Final per-source SNR and Amari error measure separation. Wall-clock time measures cost.
The first decisive check is the predicted crossover. Our advantage over iterated Gaussianization should be largest at two sources with opposite kurtosis and should shrink toward zero by six to eight sources. A second check starts from the Gaussianization solution on nearly Gaussian mixtures, to see whether the loop escapes it or stays stuck.
For real data, we apply synthetic sensor saturation to multichannel EEG in the configuration Jung et al. used, with 19 to 22 channels. There we test whether the sub-Gaussian line-noise component and the super-Gaussian blink component are still each isolated into a single component. Separately, we run the tall face-ERP task against Lyu and Fu's 0.39 error, where we expect to be competitive but not necessarily better.
References
- Blind Separation of Post-nonlinear Mixtures using Linearizing Transformations and Temporal Decorrelation
- Inverting Monotonic Nonlinearities by Entropy Maximization
- Extended ICA Removes Artifacts from Electroencephalographic Recordings
- An alternative switching criterion for independent component analysis (ICA)
- ICA based on Split Generalized Gaussian
- Robust Independent Component Analysis Using Quadratic Negentropy
- Correction: Independent Component Analysis for Brain fMRI Does Indeed Select for Maximal Independence
- Provable Subspace Identification Under Post-Nonlinear Mixtures
- Deep Deterministic Nonlinear ICA via Total Correlation Minimization with Matrix-Based Entropy Functional