MADSNet: A Multi-Scale Axial-Attentive Dense Spiking Network for Spike Camera Image Reconstruction

Sirui Li, Rui Zhao, Siqi Cai, Haizhou Li, Tiejun Huang, Yajing ZhengarXiv 2026 · arXiv

TL;DR

MADSNet is a novel Spiking Neural Network (SNN) architecture that achieves state-of-the-art image reconstruction from high-speed spike cameras. By combining multi-scale feature extraction with efficient axial attention, it outperforms leading Artificial Neural Networks (ANNs) in image quality while running 15x faster and consuming 40x less energy.

What Problem Does This Paper Address?

Conventional cameras capture frames at fixed intervals, often leading to motion blur in high-speed scenes. Spike cameras offer a neuromorphic alternative, mimicking the human retina by firing asynchronous binary spikes whenever a pixel accumulates enough light. This allows for ultra-high temporal resolution (tens of thousands of "frames" per second).

However, converting these sparse, binary spike streams back into high-quality intensity images is an "ill-posed" challenge. Historically, researchers faced a trade-off:

  1. Artificial Neural Networks (ANNs): High reconstruction quality but massive computational costs (high latency and energy consumption) due to dense multiply-accumulate operations.
  2. Spiking Neural Networks (SNNs): Highly efficient "accumulate-only" computation but typically shallow architectures that produce lower-fidelity images.

MADSNet aims to break this trade-off, proving that with the right architecture, SNNs can match or exceed ANN performance without sacrificing their inherent efficiency.

Methods

The authors propose a hierarchical spiking encoder-decoder called MADSNet. Its design focuses on increasing representation capacity while maintaining the multiplication-free nature of SNNs. Key innovations include:

  • Multi-Scale Spike Stem (MS3Stem): Instead of a single convolution, this module uses three parallel branches (1x1, 3x3, and 5x5) to capture both fine temporal correlations and broad spatial patterns simultaneously.
  • Axial Spike Attention (ASA): To capture long-range dependencies without the quadratic cost of standard self-attention, ASA factorizes 2D spatial modeling into two 1D axes (horizontal and vertical).
  • Dense Spike Bottleneck: Inspired by DenseNet but adapted for spikes, this uses additive cross-block skips to enhance feature reuse and stabilize information flow in deep spiking architectures.
  • Spike-to-Continuous Fusion Block: A critical bridge that uses channel and spatial gating to smoothly transition from discrete binary spike activations to continuous intensity values.
  • IBRA-LIF Neurons: The network utilizes the Integer Binary-Range Alignment Leaky Integrate-and-Fire neuron model, which prevents the "activation collapse" (where signals die out in deep layers) common in standard SNNs.
The overall architecture of MADSNet, featuring the hierarchical encoder-decoder and specialized spiking modules.

Datasets

The model was evaluated on several benchmarks:

  • SREDS: A large-scale simulated dataset for gray-scale spike reconstruction.
  • RBSS: A simulated dataset used to test the model's transferability to color spike reconstruction (Bayer-pattern).
  • Real-Captured Data: Spike streams from actual hardware (BSS) were used for qualitative visual comparisons.

Key Insights and Ideas

  • Architecture > Neuron Model: The authors argue that the "quality gap" in SNNs isn't a fundamental limitation of spiking neurons themselves, but rather a result of insufficient architectural design. By bringing modern deep learning concepts (multi-scale, attention, dense connectivity) to the SNN domain, they achieved a massive jump in performance.
  • The Importance of the "Bridge": Converting binary spikes to 0-255 intensity is non-trivial. The Spike-to-Continuous Fusion Block was found to be the second most impactful component in ablation studies, highlighting that how you exit the spiking domain is just as important as how you process within it.
  • Solving Activation Collapse: Deep SNNs often fail because activations drop to zero in later layers. The combination of the Dense Bottleneck and IBRA-LIF neurons was essential for making a deep spiking architecture viable.

Key Results

MADSNet sets a new benchmark for spike camera reconstruction:

  • Superior Quality: On the SREDS benchmark, it achieved 37.232 dB PSNR, outperforming the previous SNN baseline (SSIR) by 4.3 dB and even surpassing the leading ANN-based model (Spk2ImgNet) by 1.94 dB.
  • Extreme Efficiency: Compared to the ANN-based Spk2ImgNet, MADSNet is:
  • ~15x faster (20.4 ms vs. 305.4 ms latency).
  • ~3.4x lighter in GPU memory usage.
  • ~40x more energy-efficient (118.4 mJ vs. 4690 mJ).
  • Color Transfer: Despite being designed for gray-scale, MADSNet transferred exceptionally well to color reconstruction, outperforming dedicated ANN models.

Limitations and Future Work

While MADSNet is significantly more efficient than ANNs, it is still heavier and slower than very shallow SNNs like SSIR. The authors note that to fully realize the energy-saving potential of "accumulate-only" operations, the model needs to be deployed on specialized neuromorphic hardware.

Future research directions include "architectural slimming" to reduce the overhead of the deeper layers and exploring even more efficient ways to handle the spatial-chromatic coupling in color spike cameras.

Summary updated

Abstract

Spike cameras encode dynamic scenes into high-temporal-resolution binary spike streams for efficient high-speed vision. Although spiking neural networks (SNNs) can process such signals with sparse and accumulate-only computation, existing SNN-based reconstruction methods are typically shallow and still lag behind ANN-based models in reconstruction fidelity. This paper proposes MADSNet, a Multi-scale Axial-attentive Dense Spiking Network for spike camera image reconstruction. MADSNet adopts a hierarchical spiking encoder--decoder, where a multi-scale spike stem extracts early features, axial spike attention captures long-range dependencies, a dense spiking bottleneck enhances feature reuse, and a spike-to-continuous decoder reconstructs continuous intensity images from spike-domain representations. Experiments on gray-scale and color spike reconstruction show that MADSNet outperforms SSIR by 4.3 dB PSNR and Spk2ImgNet by 1.94 dB on SREDS. Compared with Spk2ImgNet, it achieves about 15$\times$ lower latency and uses only about 1/40 of the estimated energy, demonstrating that carefully designed SNN architectures can improve reconstruction quality while preserving computational efficiency.
BibTeX
@misc{li2026madsnet,
  author = {Sirui Li and Rui Zhao and Siqi Cai and Haizhou Li and Tiejun Huang and Yajing Zheng},
  title = {{MADSNet: A Multi-Scale Axial-Attentive Dense Spiking Network for Spike Camera Image Reconstruction}},
  year = {2026},
  eprint = {2611.12980},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12980},
}
Figure