Diffusion of Neuromodulators for Temporal Credit Assignment

João Barretto-Bittar, Anna Levina, Emmanouil Giannakakis, Roxana ZeraatiarXiv 2026 · DOI · arXiv

TL;DR

Researchers have developed a biologically inspired learning mechanism for spiking neural networks where error signals "diffuse" through the network like chemicals in a brain. This approach allows neural networks with very sparse feedback connections to learn complex temporal tasks, narrowing the performance gap between biologically plausible learning rules and standard backpropagation.

What Problem Does This Paper Address?

In artificial intelligence, we typically train neural networks using backpropagation, a method that requires precise, "surgically" delivered error signals to every single synapse. However, the biological brain is much messier. Real neural circuits are sparsely connected, and feedback isn't always direct.

This creates a temporal credit assignment problem: how does a specific neuron deep in the brain know it made a mistake if it doesn't receive a direct "error message" from the output? While biologically plausible alternatives like Eligibility Propagation (e-prop) exist, their performance collapses when feedback connectivity is sparse—which is the reality for most biological systems. The authors sought to find a mechanism that allows learning to occur even when most neurons are "in the dark" regarding direct feedback.

Methods

The authors propose a mechanism inspired by volume transmission—the way neuromodulators like dopamine or serotonin diffuse through the extracellular space to affect populations of neurons, rather than just crossing a single synapse.

  1. Spatially Embedded RSNNs: They built Recurrent Spiking Neural Networks (RSNNs) where neurons (LIF and Adaptive LIF types) are placed on a 2D grid. Connectivity is local; neurons are more likely to connect to their neighbors.
  2. Diffusive Credit Signaling: Instead of sending error signals to every neuron, feedback is sent to only a small subset (10%). This signal then "diffuses" to neighboring neurons over several time steps using a Cellular Automaton-like rule (specifically a Moore neighborhood).
  3. Learning Rule: They combined this diffusion with e-prop. The weight update for a synapse depends on a local "eligibility trace" (a memory of recent activity) and the local concentration of the diffusing credit signal.
  4. Decay and Spread: The model includes a decay rate ($k$) to simulate how neuromodulators are reabsorbed or degraded over time in a real brain.

Datasets

The model was evaluated on three classic temporal benchmark tasks:

  • Pattern Generation: The network must learn to reproduce a complex 1D signal (a sum of five sinusoids) in response to Poisson noise.
  • Delayed Match-to-Sample (DMS): A working memory task where the network sees two sequential cues and must decide if they are the same or different after a delay.
  • Cue Accumulation: An evidence integration task where the network observes a sequence of seven cues (left or right) and must determine which side had the majority.

Key Insights and Ideas

  • Proximity as a Proxy for Feedback: By embedding neurons in space, the network can use spatial proximity to distribute information. If a neuron's neighbor gets an error signal, the neuron can "eavesdrop" on that signal via diffusion.
  • Sparse Feedback is Sufficient: The paper demonstrates that you don't need a dedicated feedback line for every neuron. A "low-resolution" chemical cloud of information is enough to guide synaptic plasticity.
  • Biologically Plausible "Smoothing": Diffusion acts as a spatiotemporal filter. It smooths out the error signal over space and time, which appears to help stabilize learning in recurrent circuits where timing is critical.
  • Closing the Gap: This mechanism significantly reduces the performance gap between e-prop (which is local and online) and BPTT (Backpropagation Through Time), which is the non-biological gold standard that requires "unrolling" the network through time.

Key Results

  • Significant Performance Boost: In all three tasks, RSNNs using diffusion outperformed standard e-prop when feedback was sparse.
  • Pattern Generation: Diffusion allowed the network to achieve a much lower Normalized Mean Squared Error (nMSE) compared to standard e-prop, nearly matching the performance of BPTT.
  • Robustness to Decay: The benefit of diffusion was consistent across various decay rates ($k \in \{0.25, 0.5, 0.75, 0.9\}$), suggesting the mechanism is robust and doesn't require perfect parameter tuning.
  • Connectivity Efficiency: The model achieved high performance with only ~10% connectivity, mimicking the metabolic and structural efficiency of biological brains.

Limitations and Future Work

  • Simplified Geometry: The study uses a uniform 2D grid. Real brain tissue has complex 3D geometries and varying extracellular "tortuosity" (obstacles) that would affect diffusion patterns.
  • Single Modulator: The paper models a single diffusing signal. In reality, the brain uses multiple neuromodulators (Dopamine, Acetylcholine, etc.) that interact in complex ways to signal different types of information (reward, surprise, attention).
  • Fixed Connectivity: The spatial positions and connections were fixed during training. Future work could explore how structural plasticity (growing new connections) might interact with diffusing signals.
  • Scaling: While successful on these benchmarks, it remains to be seen how well diffusive credit assignment scales to much larger, deeper architectures used in modern AI.

Summary updated

Abstract

Biological learning achieves temporal credit assignment despite sparse and imprecise feedback, often relying on neuromodulatory signals acting over space and time. Here, we introduce a learning mechanism in which error information diffuses locally through the network, similar to volume transmission of neuromodulators. This distributed modulation allows neurons to learn even in the absence of direct feedback, using the local concentration of the diffusing credit signal. Applied to recurrent spiking neural networks with sparse feedback connectivity, diffusive credit signaling improves learning across three benchmark tasks. Using eligibility propagation as a baseline learning mechanism, we show how diffusion-based modulation can provide a plausible mechanism for credit assignment in sparsely connected neural circuits.
BibTeX
@misc{barrettobittar2026diffusion,
  author = {João Barretto-Bittar and Anna Levina and Emmanouil Giannakakis and Roxana Zeraati},
  title = {{Diffusion of Neuromodulators for Temporal Credit Assignment}},
  year = {2026},
  eprint = {2603.08949},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.08949},
}

Loading related content...

Figure