Abstract
The brain effortlessly solves blind source separation (BSS) problems, but the algorithm it uses remains elusive. In signal processing, linear BSS problems are often solved by Independent Component Analysis (ICA). To serve as a model of a biological circuit, the ICA neural network (NN) must satisfy at least the following requirements:
- The algorithm must operate in the online setting where data samples are streamed one at a time, and the NN computes the sources on the fly without storing any significant fraction of the data in memory.
- The synaptic weight update is local, i.e., it depends only on the biophysical variables present in the vicinity of a synapse. Here, we propose a novel objective function for ICA from which we derive a biologically plausible NN, including both the neural architecture and the synaptic learning rules. Interestingly, our algorithm relies on modulating synaptic plasticity by the total activity of the output neurons. In the brain, this could be accomplished by neuromodulators, extracellular calcium, local field potential, or nitric oxide.
Introduction
The human brain is a master of Blind Source Separation (BSS). Consider the "cocktail party problem": your ears receive a chaotic mixture of voices, music, and ambient noise, yet you can effortlessly isolate a single speaker. In signal processing, this task is typically handled by Independent Component Analysis (ICA).
However, most traditional ICA algorithms, such as Infomax, are "biologically implausible." They often require offline processing (access to the entire dataset) and non-local information (synapses needing data from distant parts of the network). To bridge this gap, researchers have developed a new ICA algorithm that adheres to biological constraints: it operates online and uses local learning rules.
A Normative Framework for ICA
The authors take a normative approach, starting with a mathematical objective function and deriving the neural architecture and learning rules directly from it. They adapt the Fourth-Order Blind Identification (FOBI) procedure, which separates sources based on their kurtosis (the "spikiness" or "flatness" of their distribution).
By reformulating FOBI into a similarity-preserving objective, the authors derived a single-layer neural network that utilizes two-compartment neurons. In this model, dendritic compartments handle data "whitening" (decorrelating inputs), while somatic compartments rotate the data to isolate independent sources.
A critical innovation is the three-factor learning rule. Synaptic updates depend on pre-synaptic activity, post-synaptic activity, and a global modulating factor—the total activity of all output neurons. This global signal coordinates the network without requiring complex, non-local wiring, mimicking biological mechanisms like neuromodulators (e.g., dopamine) or nitric oxide.
Experimental Results
The algorithm was validated across three distinct data types:
- Synthetic Data: Mixtures of periodic signals and Laplace noise, testing the ability to handle both "spiky" (leptokurtic) and "flat" (platykurtic) distributions.
- Real-world Speech: Audio from the TSP speech database, featuring mixed male and female speakers.
- Natural Images: Grayscale images of natural scenes.
The model achieved high-fidelity recovery with near-perfect overlap on synthetic and speech data. It performed competitively with "gold standard" non-biological algorithms like EASI and Infomax. Notably, it outperformed these models in scenarios with non-white (colored) data because it performs whitening internally rather than requiring a pre-processing step.
Discussion and Limitations
The two-compartment structure derived here aligns remarkably well with the anatomy of pyramidal cells in the neocortex, suggesting that biological neurons might be specialized for different stages of the ICA process.
However, some limitations remain. The algorithm cannot separate sources with identical kurtosis values and can be sensitive to outliers. Future work aims to extend this normative framework to multi-layered, non-linear networks to better reflect the complexity of the brain's processing power.
Summary updated