Abstract
Traditional neural network models typically map input signals to output activity levels, where information is primarily characterized by the mean firing rate. In this paradigm, fluctuations are often dismissed as noise. We propose a conceptual shift: the "covariance perceptron," which utilizes the second-order statistics of fluctuating activity—covariances—as the basis for stimulus-related information. By tuning both afferent and recurrent weights, we demonstrate that networks can learn to map input covariances to specific output patterns. This framework is particularly effective for classifying time series based on their spatio-temporal dynamics, transforming temporal structures into spatial covariances.
Introduction
A long-standing challenge in neuroscience is reconciling the high variability of cortical activity with the brain's ability to perform robust, rapid computations. Classical "rate-based" models suggest that neurons represent information via mean activity levels, implying that variability must be averaged out over time or across large populations. However, this averaging process is inherently slow, creating a paradox: if the brain relies on slow averaging, how does it make split-second decisions?
The covariance perceptron addresses this by treating variability not as noise, but as the signal itself. By focusing on the second-order statistics (covariances) of neural activity, we can capture the propagating nature of time series and their underlying dynamical properties. This approach moves beyond the first-order statistics (means) that dominate classical perceptron theory.
The Covariance Perceptron Framework
The covariance perceptron is modeled using discrete-time recurrent neuronal networks governed by a Multivariate Autoregressive (MAR) process. Unlike standard models that focus on the mean signal, this framework explicitly considers the correlations between neurons and across time points.
Key technical contributions include:
- A New Delta Rule: We derived a gradient-descent learning rule that adjusts both afferent (feedforward) and recurrent weights to minimize the distance between the network's actual output covariance and a desired target covariance.
- Recurrent Weight Tuning: Recurrent connections are trained to transform temporal structures (lagged covariances) into spatial patterns. This allows the network to extract information from the timing of signals and map it onto the magnitude of output fluctuations.
- Local Learning Rules: To enhance biological plausibility, we developed a local approximation of the weight update rule. This version only requires information available at the synapse level, avoiding the need to solve complex global equations at every step.
Performance and Key Insights
The model was tested across several challenging tasks, demonstrating its ability to exploit variability as a rich information source.
Cracking "Hidden" Dynamics
One of the most striking results was the classification of signals with "hidden dynamics." These are time series where the mean activity and zero-lag correlations are identical across categories, making them indistinguishable to a standard perceptron. However, their temporal structure (lagged covariance) differs. The covariance perceptron successfully classified these signals, proving it can "see" the hidden temporal structure of a system.
Moving MNIST
On the Moving MNIST task, where digits move across a receptor field, the network reached ~71% accuracy in simultaneously identifying the digit and its direction of motion. This is particularly impressive given that the task involves complex spatio-temporal patterns that violate the basic linear assumptions of the model.
Discussion and Biological Consistency
The covariance perceptron is remarkably efficient. Because it exploits the network's internal dynamics, it can achieve high accuracy with far fewer weights than standard machine learning models applied to the same raw features. Furthermore, it outperformed standard Recurrent Neural Networks (RNNs) trained via Back-Propagation Through Time (BPTT) when matched for parameter count.
Crucially, this framework aligns with biological observations of synaptic plasticity. Spike-Timing-Dependent Plasticity (STDP)—the way biological synapses actually change—is inherently sensitive to the correlations and timing of spikes. The covariance perceptron provides a formal theoretical bridge between these low-level plasticity rules and high-level network function.
Limitations and Future Work
The primary limitation is that the current theory is built on linear dynamics. While it performs well on non-linear data like MNIST, a fully non-linear theory of covariance-based learning remains an open challenge. Additionally, the model assumes stationarity in the signal statistics. Future work will explore how these networks handle non-stationary signals and implement these rules in fully spiking, biologically realistic simulations.
Summary updated