A Normative and Biologically Plausible Algorithm for Independent Component Analysis

Yanis Bahroun, Dmitri B. Chklovskii, Anirvan M. Sengupta

Introduction

Humans and animals effortlessly solve the "cocktail party problem"—isolating a single voice in a noisy room or separating mixed visual and olfactory stimuli into distinct sources. In signal processing, this challenge is known as Blind Source Separation (BSS) and is typically solved using Independent Component Analysis (ICA).

However, standard ICA algorithms are not biologically plausible. To serve as a realistic model of a biological brain circuit, an algorithm must meet two strict criteria. First, it must operate online, processing data streams one sample at a time on the fly without storing large batches of data in memory. Second, it must rely on local learning rules, meaning synaptic weight updates depend only on information physically available at the synapse (e.g., the activity of the pre- and post-synaptic neurons), without requiring global knowledge of the network's state or non-biological "weight transport." Most existing ICA algorithms fail these tests, requiring offline batch processing, explicit pre-whitening of data, or non-local learning rules.

A Normative Approach to Neural ICA

To bridge the gap between machine learning BSS techniques and theoretical neuroscience, the authors take a normative approach: starting with a high-level mathematical objective and deriving the neural network dynamics directly from it.

They base their approach on the Fourth-Order Blind Identification (FOBI) algorithm, which separates sources based on their kurtosis (how "spiky" or "flat" a statistical distribution is compared to a Gaussian). Because standard FOBI requires biologically impossible weight-sharing, the authors design a novel similarity-preserving objective function. By converting this into a min-max optimization problem and solving it online via stochastic gradient descent/ascent, they yield a set of update rules that perfectly map onto a single-layer neural network.

Schematic of the FOBI procedure and the derived biologically plausible neural network architecture.

Biological Mechanisms: Two-Compartment Neurons and Global Modulation

The resulting network architecture introduces several biologically inspired innovations that solve traditional roadblocks in neural ICA.

Two-Compartment Neurons: The network utilizes two-compartment neurons to solve the weight transport problem. The dendritic compartments handle the initial projection (implicitly whitening the data), while the somatic (cell body) compartments reconstruct the sources via fast lateral inhibitory connections. By splitting the neuron into a dendrite and a soma, the network can perform the two distinct mathematical steps of FOBI—whitening and rotation—without needing to copy weights to a second layer.

Global Modulation of Hebbian Plasticity: The derived synaptic learning rule is local but features a unique twist: it is modulated by a third factor, specifically the total somatic activity of the output neurons. Depending on this global activity level, the synapse switches between Hebbian learning (Long-Term Potentiation) and anti-Hebbian learning (Long-Term Depression). Mathematically, this global modulation is necessary. Biologically, it perfectly aligns with how brains use diffusible neuromodulators (like dopamine or nitric oxide), extracellular calcium, or local field potentials to globally broadcast activity levels to local synapses.

Furthermore, unlike some older biologically inspired models that only work if all sources deviate from a Gaussian distribution in the same direction, this network's lateral inhibitory connections allow it to separate mixtures of both sub-Gaussian (flat-topped) and super-Gaussian (spiky) distributions simultaneously.

Experimental Validation

The authors evaluated their algorithm on standard BSS benchmarks, including synthetic data (mixtures of periodic signals and Laplace random noise), real-world audio (speech recordings from the TSP database mixed with uniform noise), and real-world grayscale natural scene images.

The algorithm successfully and cleanly recovered the original independent sources across all three domains, matching the performance of classic offline ICA algorithms. Crucially, unlike many neural ICA models, this algorithm processes raw mixtures directly, handling the whitening step implicitly within the dendritic computations. In numerical comparisons, the proposed model either outperformed or matched seminal neural ICA algorithms (like Herault-Jutten, EASI, and Bell-Sejnowski) while maintaining strict biological plausibility.

Limitations and Future Directions

While highly effective, the algorithm has limitations inherent to its mathematical foundation. Because it relies on 4th-order moments (kurtosis) to separate signals, it fundamentally cannot separate sources that have the exact same kurtosis. Additionally, kurtosis-based methods are notoriously sensitive to statistical outliers. The authors suggest that altering the scaling rule to be a sublinear function of total activity could mitigate this sensitivity.

Future work will explore extending this framework to nonlinear ICA, multi-layered architectures, and incorporating temporal correlations, such as Slow Feature Analysis. Combining this unsupervised BSS method with biologically plausible supervised learning algorithms could pave the way for more comprehensive and realistic models of cognitive processing.

Summary updated

Abstract

The brain effortlessly solves blind source separation (BSS) problems, but the algorithm it uses remains elusive. In signal processing, linear BSS problems are often solved by Independent Component Analysis (ICA). To serve as a model of a biological circuit, the ICA neural network (NN) must satisfy at least the following requirements: 1. The algorithm must operate in the online setting where data samples are streamed one at a time, and the NN computes the sources on the fly without storing any significant fraction of the data in memory. 2. The synaptic weight update is local, i.e., it depends only on the biophysical variables present in the vicinity of a synapse. Here, we propose a novel objective function for ICA from which we derive a biologically plausible NN, including both the neural architecture and the synaptic learning rules. Interestingly, our algorithm relies on modulating synaptic plasticity by the total activity of the output neurons. In the brain, this could be accomplished by neuromodulators, extracellular calcium, local field potential, or nitric oxide.
BibTeX
@inproceedings{bahroun2021normative,
  author = {Yanis Bahroun and Dmitri B. Chklovskii and Anirvan M. Sengupta},
  title = {{A Normative and Biologically Plausible Algorithm for Independent Component Analysis}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=GuyFdXUSiN},
}
Figure