Special Session: Neuromorphic hardware design and reliability from traditional CMOS to emerging technologies

Fabio Pavanello, Elena Ioana Vatajelu, Alberto Bosio, Thomas Van Vaerenbergh, Peter Bienstman, Benoit Charbonnier, Alessio Carpegna, Stefano Di Carlo, Alessandro SavinoarXiv 2023 · DOI · arXiv

This paper, a special session report from the IEEE, provides a comprehensive roadmap for the future of neuromorphic hardware. It bridges the gap between today’s digital CMOS accelerators and tomorrow’s integrated photonic systems, all while keeping a sharp eye on the critical issue of hardware reliability.

TL;DR

The paper surveys the evolution of neuromorphic hardware from traditional digital CMOS to emerging silicon photonics, highlighting how these new architectures overcome the "memory wall." Crucially, it demonstrates that Spiking Neural Networks (SNNs) are not just more energy-efficient but also inherently more resilient to hardware faults than traditional Artificial Neural Networks (ANNs).

What Problem Does This Paper Address?

Traditional computing follows the Von Neumann architecture, where the processor and memory are separate. In the age of Deep Learning, this creates a "communication bottleneck"—moving massive amounts of data between memory and the CPU/GPU consumes the lion's share of energy and time.

Neuromorphic computing aims to solve this by co-locating memory and processing, mimicking the biological brain. However, as we move toward edge computing (like IoT devices), we face two major hurdles:

  1. Efficiency: How do we perform massive matrix multiplications with minimal power?
  2. Reliability: Hardware isn't perfect. Manufacturing defects, aging, and environmental factors cause faults. How do these faults impact the "intelligence" of the system?

Methods

The authors review three primary hardware paradigms:

  • Digital Accelerators: Specialized ASICs (like IBM’s TrueNorth and Intel’s Loihi) and FPGAs that use event-driven, single-bit signaling to exploit the sparsity of spikes in SNNs.
  • Augmented Silicon Photonics: A cutting-edge approach using light instead of electricity. It leverages Mach-Zehnder Interferometers (MZIs) to perform matrix-vector multiplications at the speed of light with near-zero latency.
  • Phase Change Materials (PCMs): Integrating non-volatile materials into photonic circuits to store synaptic weights without requiring constant power.

To address reliability, the authors discuss fault injection frameworks (like Ares) that simulate how neuron and synapse failures affect network accuracy.

Datasets

The paper primarily references the MNIST handwritten digit dataset to benchmark the fault resilience of different network types (MLP vs. SNN) and training algorithms (BPTT, STDP, Shadow Training). It also mentions vowel recognition tasks for evaluating photonic feed-forward networks.

Key Insights and Ideas

  • The Power of Sparsity: SNNs are inherently suited for hardware because they communicate via single-bit spikes. This drastically reduces memory bandwidth requirements compared to the 32-bit floating-point numbers used in standard GPUs.
  • Wavelength Multiplexing: Photonics allows for massive parallelism by encoding different data streams onto different frequencies (colors) of light, all traveling through the same waveguide simultaneously.
  • Non-Volatile Photonics: By using PCMs, researchers can "set and forget" synaptic weights. Unlike traditional silicon photonics that require constant energy to maintain a phase shift, PCM-based weights consume zero static power.
  • Resilience by Design: A standout finding is that SNNs are more robust than Multi-Layer Perceptrons (MLPs). Specifically, SNNs trained with Back-Propagation Through Time (BPTT) or online learning show the highest resilience to synaptic faults.
Comparison of traditional Von Neumann architecture vs. neuromorphic co-location

Key Results

The paper provides striking quantitative comparisons across technologies:

  • Energy Efficiency: While a Google TPU (digital) consumes ~430 fJ per MAC (Multiply-Accumulate operation), sub-$\lambda$ nanophotonics can potentially reach a staggering 0.03 fJ/MAC.
  • Latency: Photonic approaches achieve latencies under 0.05 ns, orders of magnitude faster than the ~2000 ns typical of digital TPUs.
  • Fault Tolerance: In traditional MLPs, having just 10% faulty neurons can lead to a 62% misclassification rate. In contrast, SNNs maintain much higher accuracy under similar fault conditions, especially when trained to account for hardware imperfections.
Fig. 3: (a) Photonic implementation of a network for 9x9 matrix multiplication based on Mach-Zenhder Interferometers (MZIs) [60]. Inset describes what each crossing consists of, i.e., a phase shifter (φ), then a 50/50 splitter, another phase shifter (2θ), and then a 50/50 combiner. (b) Top-view of phase shifter implementations in waveguides based on (i) thermo- optic effect, (ii) electro-optic effect, (iii) carrier plasma effect, and (iv) Phase Change Material (PCM)-induced shift. Blue lines where no crossing is present are optical waveguides.

Limitations and Future Work

Despite the promise of photonics, the authors note that the reliability of integrated photonic circuits is still an open research question. The physical size of optical components also limits the scale of matrices that can be implemented on a single chip.

Future work is directed toward:

  1. Pruning and Block Decomposition: Techniques to fit larger neural networks onto limited photonic hardware.
  2. Hybrid Integration: Combining III-V materials (for lasers) with silicon platforms to create more efficient, scalable "spiking lasers."
  3. Standardization: Reaching a consensus on signal-to-spike coding and training algorithms to make SNN hardware more mainstream.

Summary updated

Abstract

The field of neuromorphic computing has been rapidly evolving in recent years, with an increasing focus on hardware design and reliability. This special session paper provides an overview of the recent developments in neuromorphic computing, focusing on hardware design and reliability. We first review the traditional CMOS-based approaches to neuromorphic hardware design and identify the challenges related to scalability, latency, and power consumption. We then investigate alternative approaches based on emerging technologies, specifically integrated photonics approaches within the NEUROPULS project. Finally, we examine the impact of device variability and aging on the reliability of neuromorphic hardware and present techniques for mitigating these effects. This review is intended to serve as a valuable resource for researchers and practitioners in neuromorphic computing.
BibTeX
@misc{pavanello2023special,
  author = {Fabio Pavanello and Elena Ioana Vatajelu and Alberto Bosio and Thomas Van Vaerenbergh and Peter Bienstman and Benoit Charbonnier and Alessio Carpegna and Stefano Di Carlo and Alessandro Savino},
  title = {{Special Session: Neuromorphic hardware design and reliability from traditional CMOS to emerging technologies}},
  year = {2023},
  eprint = {2305.01818},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.01818},
}
Figure