The covariance perceptron: A new paradigm for classification and processing of time series in recurrent neuronal networks

Matthieu Gilson, David Dahmen, Rubén Moreno‐Bote, Andrea Insabato, Moritz Helias

Abstract

Traditional neural network models typically map input signals to output activity levels, where information is primarily characterized by the mean firing rate. In this paradigm, fluctuations are often dismissed as noise. We propose a conceptual shift: the "covariance perceptron," which utilizes the second-order statistics of fluctuating activity—covariances—as the basis for stimulus-related information. By tuning both afferent and recurrent weights, we demonstrate that networks can learn to map input covariances to specific output patterns. This framework is particularly effective for classifying time series based on their spatio-temporal dynamics, transforming temporal structures into spatial covariances.

Introduction

A long-standing challenge in neuroscience is reconciling the high variability of cortical activity with the brain's ability to perform robust, rapid computations. Classical "rate-based" models suggest that neurons represent information via mean activity levels, implying that variability must be averaged out over time or across large populations. However, this averaging process is inherently slow, creating a paradox: if the brain relies on slow averaging, how does it make split-second decisions?

The covariance perceptron addresses this by treating variability not as noise, but as the signal itself. By focusing on the second-order statistics (covariances) of neural activity, we can capture the propagating nature of time series and their underlying dynamical properties. This approach moves beyond the first-order statistics (means) that dominate classical perceptron theory.

The Covariance Perceptron Framework

The covariance perceptron is modeled using discrete-time recurrent neuronal networks governed by a Multivariate Autoregressive (MAR) process. Unlike standard models that focus on the mean signal, this framework explicitly considers the correlations between neurons and across time points.

Types of correlations in neural time series, including signal, within-trial (zero-lag and lagged), and noise correlations.

Key technical contributions include:

  • A New Delta Rule: We derived a gradient-descent learning rule that adjusts both afferent (feedforward) and recurrent weights to minimize the distance between the network's actual output covariance and a desired target covariance.
  • Recurrent Weight Tuning: Recurrent connections are trained to transform temporal structures (lagged covariances) into spatial patterns. This allows the network to extract information from the timing of signals and map it onto the magnitude of output fluctuations.
  • Local Learning Rules: To enhance biological plausibility, we developed a local approximation of the weight update rule. This version only requires information available at the synapse level, avoiding the need to solve complex global equations at every step.

Performance and Key Insights

The model was tested across several challenging tasks, demonstrating its ability to exploit variability as a rich information source.

Geometric intuition for the covariance perceptron. While classes may overlap in mean activity space (right), they can be clearly separated in covariance space (middle) based on their time-series dynamics (left).

Cracking "Hidden" Dynamics

One of the most striking results was the classification of signals with "hidden dynamics." These are time series where the mean activity and zero-lag correlations are identical across categories, making them indistinguishable to a standard perceptron. However, their temporal structure (lagged covariance) differs. The covariance perceptron successfully classified these signals, proving it can "see" the hidden temporal structure of a system.

Moving MNIST

On the Moving MNIST task, where digits move across a receptor field, the network reached ~71% accuracy in simultaneously identifying the digit and its direction of motion. This is particularly impressive given that the task involves complex spatio-temporal patterns that violate the basic linear assumptions of the model.

Discussion and Biological Consistency

The covariance perceptron is remarkably efficient. Because it exploits the network's internal dynamics, it can achieve high accuracy with far fewer weights than standard machine learning models applied to the same raw features. Furthermore, it outperformed standard Recurrent Neural Networks (RNNs) trained via Back-Propagation Through Time (BPTT) when matched for parameter count.

Crucially, this framework aligns with biological observations of synaptic plasticity. Spike-Timing-Dependent Plasticity (STDP)—the way biological synapses actually change—is inherently sensitive to the correlations and timing of spikes. The covariance perceptron provides a formal theoretical bridge between these low-level plasticity rules and high-level network function.

Limitations and Future Work

The primary limitation is that the current theory is built on linear dynamics. While it performs well on non-linear data like MNIST, a fully non-linear theory of covariance-based learning remains an open challenge. Additionally, the model assumes stationarity in the signal statistics. Future work will explore how these networks handle non-stationary signals and implement these rules in fully spiking, biologically realistic simulations.

Summary updated

Abstract

Abstract Learning in neuronal networks has developed in many directions, in particular to reproduce cognitive tasks like image recognition and speech processing. Implementations have been inspired by stereotypical neuronal responses like tuning curves in the visual system, where, for example, ON/OFF cells fire or not depending on the contrast in their receptive fields. Classical models of neuronal networks therefore map a set of input signals to a set of activity levels in the output of the network. Each category of inputs is thereby predominantly characterized by its mean. In the case of time series, fluctuations around this mean constitute noise in this view. For this paradigm, the high variability exhibited by the cortical activity may thus imply limitations or constraints, which have been discussed for many years. For example, the need for averaging neuronal activity over long periods or large groups of cells to assess a robust mean and to diminish the effect of noise correlations. To reconcile robust computations with variable neuronal activity, we here propose a conceptual change of perspective by employing variability of activity as the basis for stimulus-related information to be learned by neurons, rather than merely being the noise that corrupts the mean signal. In this new paradigm both afferent and recurrent weights in a network are tuned to shape the input-output mapping for covariances, the second-order statistics of the fluctuating activity. When including time lags, covariance patterns define a natural metric for time series that capture their propagating nature. We develop the theory for classification of time series based on their spatio-temporal covariances, which reflect dynamical properties. We demonstrate that recurrent connectivity is able to transform information contained in the temporal structure of the signal into spatial covariances. Finally, we use the MNIST database to show how the covariance perceptron can capture specific second-order statistical patterns generated by moving digits. Author summary The dynamics in cortex is characterized by highly fluctuating activity: Even under the very same experimental conditions the activity typically does not reproduce on the level of individual spikes. Given this variability, how then does the brain realize its quasi-deterministic function? One obvious solution is to compute averages over many cells, assuming that the mean activity, or rate, is actually the decisive signal. Variability across trials of an experiment is thus considered noise. We here explore the opposite view: Can fluctuations be used to actually represent information? And if yes, is there a benefit over a representation using the mean rate? We find that a fluctuation-based scheme is not only powerful in distinguishing signals into several classes, but also that networks can efficiently be trained in the new paradigm. Moreover, we argue why such a scheme of representation is more consistent with known forms of synaptic plasticity than rate-based network dynamics.
BibTeX
@article{gilson2020covariance,
  author = {Matthieu Gilson and David Dahmen and Rubén Moreno‐Bote and Andrea Insabato and Moritz Helias},
  title = {{The covariance perceptron: A new paradigm for classification and processing of time series in recurrent neuronal networks}},
  journal = {PLOS Computational Biology},
  year = {2020},
  doi = {10.1371/journal.pcbi.1008127},
}
Figure