Improving Spiking Sparse Recovery via Non-Convex Penalties

Xiang Zhang, Lei Yu, Gang ZhengarXiv 2020 · DOI · arXiv

Abstract

Compared with digital methods, sparse recovery based on spiking neural networks has great advantages like high computational efficiency and low power-consumption. However, current spiking algorithms cannot guarantee more accurate estimates since they are usually designed to solve the classical optimization with convex penalties, especially the $\ell_{1}$-norm. In fact, convex penalties are observed to underestimate the true solution in practice, while non-convex ones can avoid the underestimation. Inspired by this, we propose an adaptive version of spiking sparse recovery algorithm to solve the non-convex regularized optimization, and provide an analysis on its global asymptotic convergence. Through experiments, the accuracy is greatly improved under different adaptive ways.

Introduction

Sparse Recovery (SR) is a fundamental task in signal processing, aimed at reconstructing high-dimensional signals from a limited number of measurements. While traditionally solved using convex $\ell_1$-norm regularization, this approach often suffers from an underestimation bias, where high-amplitude components are excessively penalized. Spiking Neural Networks (SNNs) offer a promising, energy-efficient alternative for solving SR problems. However, existing SNN-based algorithms typically inherit the limitations of convex penalties. This paper introduces Adaptive Spiking Sparse Recovery (A-SSR), a framework that incorporates non-convex penalties into the SNN dynamical system to enhance reconstruction accuracy.

Methodology

The A-SSR framework is built upon the Integrate-and-Fire (IAF) neuron model. The key innovation lies in the introduction of an adaptive leakage mechanism. Unlike standard SNNs with constant leakage, A-SSR dynamically adjusts the leakage current based on the derivative of a non-convex penalty function (e.g., exponential, logarithmic, or arctan) and the neuron's firing rate.

Evolution of spiking sparse recovery from convex to non-convex adaptive frameworks.

As illustrated in the figure above, the transition from standard SSR to A-SSR involves moving from a constrained $\ell_1$ optimization to a non-convex $C(\mathbf{a})$ regularized problem. Mathematically, as a neuron's firing rate increases—signaling a high-amplitude component—the adaptive mechanism reduces the effective leakage, allowing the neuron to fire more freely and providing a more accurate estimate of the true signal value. The authors rigorously prove the global asymptotic convergence of this non-convex system (Theorem 1), ensuring stability despite the inherent complexity of non-convex optimization.

Experimental Evaluation

The proposed A-SSR algorithm was validated through extensive numerical simulations using the Nengo platform. The experiments utilized synthetic sparse signals and a $100 \times 200$ dictionary matrix, with performance evaluated under various noise conditions (SNR of 20dB) and sparsity levels.

Performance comparison between standard SSR and A-SSR variants across multiple metrics.

Key Results

  • Enhanced Accuracy: A-SSR significantly outperforms standard SSR. In benchmark tests, standard SSR achieved a Normalized Mean Square Error (NMSE) of -19.95 dB, whereas A-SSR reached -28.97 dB.
  • Faster Convergence: The adaptive mechanism facilitates quicker stabilization. As shown in the convergence plots, A-SSR variants reach lower NMSE levels faster than the baseline SSR.
  • Robustness: A-SSR maintains a higher probability of recovery and lower NMSE across varying levels of signal sparsity, measurement ratios, and noise levels. Among the tested non-convex penalties, the exponential penalty (A-SSR-Exp) generally demonstrated the best performance.

Discussion and Future Work

The A-SSR framework demonstrates that non-convexity can be effectively managed within the dynamical framework of SNNs to achieve superior performance. By "encouraging the winners"—reducing penalties for active neurons—the system avoids the systematic underestimation typical of convex methods.

However, several avenues for future research remain. The performance of A-SSR is sensitive to the choice of penalty function and hyperparameter tuning (e.g., $\lambda$ and $\gamma$). Furthermore, while the theoretical and simulation results are promising, future work should focus on hardware implementation (e.g., on FPAA chips) and scaling the algorithm to handle larger, real-world datasets such as high-resolution medical imaging or genomic data.

Conclusion

A-SSR represents a significant step forward in spiking sparse recovery, combining the energy efficiency of SNNs with the superior accuracy of non-convex optimization. The proposed adaptive leakage mechanism provides a biologically inspired yet mathematically rigorous way to overcome the limitations of traditional convex sparse recovery methods.

Summary updated

Abstract

Compared with digital methods, sparse recovery based on spiking neural networks has great advantages like high computational efficiency and low power-consumption. However, current spiking algorithms cannot guarantee more accurate estimates since they are usually designed to solve the classical optimization with convex penalties, especially the $\ell_{1}$-norm. In fact, convex penalties are observed to underestimate the true solution in practice, while non-convex ones can avoid the underestimation. Inspired by this, we propose an adaptive version of spiking sparse recovery algorithm to solve the non-convex regularized optimization, and provide an analysis on its global asymptotic convergence. Through experiments, the accuracy is greatly improved under different adaptive ways.
BibTeX
@misc{zhang2020improving,
  author = {Xiang Zhang and Lei Yu and Gang Zheng},
  title = {{Improving Spiking Sparse Recovery via Non-Convex Penalties}},
  year = {2020},
  eprint = {2009.09163},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.09163},
}
Figure