TL;DR
This paper introduces a spiking neural network model that uses an internally generated "surprise" signal to rapidly adapt to changing rules in sequential data. This surprise signal, derived from a mismatch between prediction and observation, modulates synaptic plasticity, enabling faster learning and preventing catastrophic forgetting, thus addressing the stability-plasticity dilemma.
What Problem Does This Paper Address?
The brain's remarkable ability to learn and adapt to new rules, even when they change unexpectedly, is a fundamental aspect of intelligence. This is known as the stability-plasticity dilemma: how can a system remain stable and retain old knowledge while also being plastic enough to learn new information quickly? Previous models have struggled to reconcile these two requirements, often leading to either slow adaptation or overwriting of existing memories. This paper tackles this challenge by proposing a biologically plausible mechanism for how "surprise" – a reaction to unexpected events – can drive rapid learning and rule switching in neural networks.
Methods
The authors propose a self-supervised spiking neural network model called SpikeSuM (Spike-based Surprise-Modulated). The core idea is to extract a "surprise" signal from the network's internal activity. Specifically, this signal arises from a momentary imbalance between excitation and inhibition in a prediction error layer. This surprise signal then acts as a "third factor" in a three-factor learning rule, modulating synaptic plasticity.
Key aspects of the method include:
- Spiking Neural Network: The model uses spiking neurons, which communicate through discrete events (spikes), making it more biologically realistic than traditional rate-based networks.
- Prediction Error Layer: This layer compares incoming sensory information with predictions generated from past experiences. A mismatch leads to increased activity.
- Surprise Signal Generation: The surprise signal is derived from the overall activity of the prediction error layer, reflecting the degree of prediction error.
- Three-Factor Learning Rule: Synaptic plasticity (weight changes) is modulated by three factors: presynaptic activity (spikes), postsynaptic activity (membrane potential), and the surprise signal. This rule increases plasticity during surprising events.
- Modular Architecture (SpikeSuM-C): For continual learning where rules can reappear, a modular network (SpikeSuM-C) is introduced. Each module acts as a memory for a specific rule, and a context selector module (CSM) dynamically selects the appropriate module for learning or recall.
Datasets
The paper uses simulated datasets designed to mimic "volatile sequence tasks." These tasks involve sequences of stimuli generated by underlying rules (e.g., transitions between rooms in an apartment). The rules can change abruptly ("change points"), and the tasks are designed to test:
- Volatile sequence task without re-occurrence of rules: Each rule switch introduces a completely new transition matrix.
- Volatile sequence task with re-occurrence of rules: Rules can reappear, requiring the network to recall previously learned patterns.
No real-world datasets are used for training or evaluation.
Key Insights and Ideas
- Surprise as a Learning Accelerator: Surprise isn't just a reaction; it's a crucial signal that actively boosts learning speed. When something unexpected happens, the network becomes more plastic, allowing it to quickly update its internal model.
- Internal Surprise Generation: The model demonstrates that surprise doesn't need to be an external cue. It can be intrinsically generated by the network's own prediction errors, making it a self-regulating learning mechanism.
- Stability-Plasticity Solution: The surprise-modulated plasticity offers a way to balance stability and plasticity. While normal learning might be slow or overwrite old memories, surprise events trigger rapid, targeted updates, and the modular architecture prevents interference with previously learned rules.
- Predictive Coding as a Byproduct: The model's approach to extracting surprise naturally leads to a form of predictive coding, where the network learns to anticipate future events based on past experiences.
Key Results
- Rapid Re-adaptation: The SpikeSuM model demonstrates significantly faster relearning after rule switches compared to networks without surprise modulation. For instance, it can adapt to a new rule within the first presentation step ("one-shot learning").
- Memory Protection: In the modular SpikeSuM-C network, when a previously encountered rule reappears, the network quickly reactivates the correct module, avoiding relearning from scratch and protecting existing memories.
- Accurate Rule Detection: The SpikeSuM-C model successfully detects rule switches with high accuracy, especially in tasks with fewer possible transitions. For K=2 transitions, detection success rates are over 90%.
- Comparison to Benchmarks: The SpikeSuM model performs comparably to, and in some cases better than, established surprise-based machine learning algorithms like BOCPA and VarSMiLe on the simulated tasks.
Limitations and Future Work
- Simulated Data: The model is evaluated on simulated tasks. Future work could involve testing its performance on real-world sequential data.
- Neuromodulator Mapping: While the model uses a "third factor" for plasticity modulation, the exact mapping to specific neuromodulators (like dopamine or norepinephrine) is conceptual and requires further investigation. The paper notes that a one-to-one mapping is unlikely.
- Planning: The authors suggest that the learned predictions could be used for planning by "imagining" future states, which is left as future work.
- Integration with Larger Networks: The authors envision their surprise-detection modules being integrated into larger, more complex networks, such as those used for visual processing or cognitive tasks.
Summary updated