Abstract: We propose a simple network of Hawkes processes as a cognitive model capable of learning to classify objects. Our learning algorithm, named HAN for Hawkes Aggregation of Neurons, is based on a local synaptic learning rule based on spiking probabilities at each output node. We were able to use local regret bounds to prove mathematically that the network is able to learn on average and even asymptotically under more restrictive assumptions.
Introduction
In neuroscience, it is widely accepted that the brain learns through local rules—synaptic changes determined by the activity of the specific neurons they connect. However, modern machine learning predominantly relies on global rules like backpropagation, which are considered biologically implausible due to their requirement for backward information flow. This paper bridges the theoretical gap by introducing HAN (Hawkes Aggregation of Neurons), a spiking neural network model. By framing synaptic weight updates as an "expert aggregation" problem, the authors provide a mathematical proof that such bio-inspired local rules can lead to provable global learning and optimal classification performance.
The HAN Architecture and Algorithm
The authors propose a two-layer network architecture. The Input Layer consists of neurons representing object features (e.g., "blue," "square") that fire spikes following a Bernoulli distribution. The Output Layer contains neurons representing categories, modeled as discrete-time Hawkes processes. The probability of an output neuron spiking depends on the weighted sum of activity from its presynaptic input neurons.
The core innovation of the HAN algorithm is treating each output neuron as a "forecaster" and its presynaptic connections as "experts." Instead of standard gradient descent, the network utilizes online learning algorithms like Exponentially Weighted Average (EWA) or Polynomially Weighted Average (PWA) to update weights based on a local "gain" signal. If an object belongs to a specific class, the corresponding output neuron receives a reward signal, while others are penalized. Using Kalikow decomposition for linear Hawkes processes, the authors can interpret synaptic weights directly as a probability distribution over which input neuron "caused" the output spike.
Theoretical Guarantees
By applying online learning theory to the Hawkes network, the authors derive oracle inequalities. These provide a mathematical guarantee that the HAN network performs nearly as well as an "ideal" network with optimal weights. Specifically, they prove an upper bound on the "regret" of the network, showing that the error compared to an optimal classifier shrinks at a rate of $O(M^{-1/2})$ as the number of training objects $M$ increases. This represents a significant step in proving that local, "blind" updates can converge to globally optimal solutions.
Experimental Results
The model was evaluated using synthetic datasets designed to mimic cognitive psychology experiments, such as the "Component-Cue" model. In these tasks, objects are defined by multiple characteristics, and the network must learn to classify them based on feature combinations.
As shown in the performance results, HAN reached perfect classification accuracy significantly faster than the classic Component-Cue model. The study also highlighted the importance of inhibition; incorporating non-linear activation functions and inhibitory connections significantly accelerated learning compared to simpler linear models. Furthermore, an ablation study demonstrated the model's robustness: even when 5 out of 6 features were removed, the HAN model with EWA maintained approximately 85% accuracy, whereas simpler versions dropped to near-chance levels.
Conclusion and Future Work
HAN provides a rigorous mathematical foundation for local learning in spiking networks, achieving performance comparable to a standard Perceptron despite the added complexity of modeling individual spikes. However, the current gain formula requires supervised feedback. Future research will focus on moving toward purely unsupervised local rules, such as pure Spike-Timing-Dependent Plasticity (STDP), and extending these mathematical proofs to deeper architectures with hidden layers.
Summary updated