SpikingIR: A Novel Converted Spiking Neural Network for Efficient Image Restoration

Yang Ouyang, Zihan Cheng, Xiaotong Luo, Guoqi Li, Yanyun Qu

TL;DR

SpikingIR is a novel framework that brings the energy efficiency of Spiking Neural Networks (SNNs) to high-precision image restoration tasks like dehazing and super-resolution. By introducing specialized modules to map discrete spikes to continuous values and "recycle" leftover neural energy, it achieves performance comparable to standard Artificial Neural Networks (ANNs) while slashing energy consumption by up to 50%.

What Problem Does This Paper Address?

Image restoration (IR)—the process of removing haze, noise, or blur from an image—is a "low-level" vision task that requires extreme precision. While deep learning has mastered this field, the resulting models are power-hungry, making them difficult to deploy on battery-constrained edge devices like smartphones or drones.

Spiking Neural Networks (SNNs) offer a brain-inspired, energy-efficient alternative because they communicate via binary "spikes" rather than continuous numbers. However, this binary nature is a double-edged sword: it creates massive quantization errors and information loss. In classification tasks (is this a cat?), a few lost spikes don't matter much. In restoration (what is the exact color of this pixel?), even tiny errors lead to severe visual degradation. Most existing SNNs try to fix this by running for hundreds of "timesteps," but this destroys the energy efficiency they were meant to provide.

Methods

The authors propose a two-stage ANN-to-SNN conversion framework called SpikingIR. It starts with a pre-trained ANN and converts it into an SNN using two novel components:

  1. Convolutional Pixel Mapping (CPM): Standard SNNs simply count spikes at the output layer to determine a pixel value. CPM instead uses a convolutional layer to map discrete spike patterns into a continuous space. It also includes a Residual Membrane Potential Information Shortcut (RMPIS), which captures the "leftover charge" in the final neurons that didn't quite reach the threshold to fire a spike, ensuring no data is wasted at the finish line.
  2. Membrane Potential Reuse Neuron (MPRN): In a typical SNN, any "membrane potential" (stored charge) left in a neuron after the final timestep is discarded. MPRN adds a "bonus" post-processing step with a relaxed firing threshold, allowing the neuron to release this residual information as one last spike.
  3. Energy-Aware Fine-Tuning: After conversion, the model is fine-tuned using surrogate gradients. The loss function includes an "energy constraint" that penalizes excessive spiking, forcing the model to find the most efficient path to a high-quality restoration.

Datasets

The researchers evaluated SpikingIR across three major IR tasks:

  • Image Dehazing: Trained on indoor synthetic data; tested on TestSet A and SOTS-outdoor.
  • Image Denoising: Evaluated on Set12 and BSD68 using various Gaussian noise levels ($\sigma = 15, 25, 50$).
  • Super-Resolution: Tested on Set5 and Urban100 for $2\times$ and $4\times$ upscaling.

Key Insights and Ideas

  • Information Entropy as a Metric: The authors use information theory to prove that CPM significantly increases the "information capacity" of the network's output compared to vanilla spike counting, explaining why their images look so much sharper.
  • Waste Not, Want Not: A significant amount of data in SNNs is "trapped" in sub-threshold membrane potentials. By treating this residual charge as a signal rather than noise (via MPRN), the model achieves higher accuracy without needing more timesteps.
  • Low Latency is Possible: While many SNN conversions require 50–100 timesteps to reach decent accuracy, SpikingIR achieves near-lossless performance in as few as 5 to 10 timesteps.

Key Results

  • Superior Dehazing: On the SOTS-outdoor dataset, SpikingIR actually outperformed its original ANN baseline (21.47 dB vs 20.12 dB PSNR) while using only 10 timesteps.
  • Efficiency Gains: Across all tasks, SpikingIR maintained a "conversion rate" (performance relative to the ANN) of over 99%.
  • Energy Savings: The model reduced energy consumption by approximately 50% compared to the ANN baseline in dehazing and denoising tasks, and up to 80% in $2\times$ super-resolution.
  • Visual Fidelity: Qualitative results show that SpikingIR avoids the "blocky" artifacts and color shifts typical of standard SNN-converted models.

Limitations and Future Work

  • Architectural Constraints: The authors used older baseline models (AOD-Net, DnCNN, VDSR) because current SNN training tools struggle to support modern components like Transformers, LayerNorm, or complex attention mechanisms.
  • Deep Structure Accumulation: In very deep networks (like VDSR), conversion errors still accumulate. While SpikingIR mitigates this, the SNN still slightly lags behind the ANN in pure PSNR for the most complex super-resolution tasks.
  • Hardware Realization: The energy savings are currently calculated based on theoretical spike counts; the authors note that the true potential will be realized when these models are run on dedicated neuromorphic hardware (like Intel's Loihi or SynSense's chips).

Summary updated

Abstract

Image restoration has made great progress with the rise of deep learning, but its energy consumption limits its real-world applications. Spiking Neural Networks (SNNs) are seen as energy-efficient alternatives to Artificial Neural Networks (ANNs). Applying SNNs to image restoration (IR) remains challenging, primarily due to the limited information capacity of spike-based signals. This limitation leads to quantization errors and information loss, while IR tasks are highly sensitive to output precision and error. Thus, the restoration performance suffers significantly. To address this challenge, we propose SpikingIR, an ANN-to-SNN conversion framework for IR that reduces information loss and quantization error. SpikingIR mainly consists of two components: Convolutional Pixel Mapping (CPM) and Membrane Potential Reuse Neuron (MPRN), which are designed to alleviate quantization errors and information loss in the output and intermediate layers, respectively. Specifically, CPM maps discrete outputs into a continuous space, better aligning with pixel-level details. From the perspective of information entropy, we show that outputs of CPM contain more information than the original outputs. MPRN introduces a post-processing step with relaxed firing conditions to extract residual membrane potential, reducing information waste. Furthermore, we fine-tune the converted model to jointly optimize both accuracy and energy efficiency. Experimental results demonstrate that SpikingIR achieves performance comparable to ANN counterparts across various IR benchmarks while reducing energy consumption by up to 50%.
BibTeX
@inproceedings{ouyang2026spikingir,
  author = {Yang Ouyang and Zihan Cheng and Xiaotong Luo and Guoqi Li and Yanyun Qu},
  title = {{SpikingIR: A Novel Converted Spiking Neural Network for Efficient Image Restoration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i10.37769},
}

Loading related content...

Figure