A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)

Saber Moradi, Qiao Ning, Fabio Stefanini, Giacomo Indiveri

TL;DR

This paper introduces DYNAPS, a novel multi-core neuromorphic architecture designed for dynamic, event-based neural networks. It tackles the challenge of memory and communication overhead in large-scale neuromorphic systems by employing a hierarchical, mixed-mode routing scheme and heterogeneous on-chip memory structures, enabling efficient and flexible computation.

What Problem Does This Paper Address?

Neuromorphic computing aims to mimic the brain's structure and function for efficient computation, particularly for tasks involving sensory processing and real-time data. A key challenge in building large-scale neuromorphic systems is managing the massive amount of communication and memory required by spiking neural networks. These networks use asynchronous "events" (spikes) to transmit information, which can lead to high bandwidth and latency issues, especially when neurons have many connections (high fan-out). Existing approaches often struggle with either flexibility (limiting network topologies) or scalability (requiring excessive memory and power), leading to a "von Neumann bottleneck" where data transfer between memory and processing units becomes a bottleneck. This paper addresses this by proposing a new architecture that optimizes memory usage and communication for highly interconnected, event-based neural networks.

Methods

The core innovation of DYNAPS lies in its memory-optimized routing scheme and heterogeneous memory structures.

  • Two-Stage Routing: Instead of direct point-to-point routing, DYNAPS uses a two-stage approach. First, neurons send "tag" data to intermediate nodes. Second, these intermediate nodes broadcast the tags to clusters of destination neurons. This reduces the total memory needed to store routing information.
  • Hierarchical and Mesh Routing: The architecture combines hierarchical routing (efficient for local communication within clusters) with a 2D mesh routing strategy (for inter-chip communication). This is implemented using three levels of asynchronous routers (R1, R2, R3).
  • Heterogeneous Memory: The system distributes memory across different components: SRAM for source addresses and CAM (Content-Addressable Memory) for destination addresses. This "in-memory computing" approach minimizes data movement.
  • Asynchronous VLSI Design: The entire architecture is built using Quasi Delay-Insensitive (QDI) asynchronous circuits, which are robust to variations in gate delays and can offer power efficiency.
  • Mixed-Signal Design: Neurons and synapses are implemented using analog subthreshold circuits for biophysically realistic behavior and low power, while routing and memory management are handled by digital asynchronous circuits.

Datasets

The paper mentions using a Poker card suit recognition dataset generated by a Dynamic Vision Sensor (DVS). This dataset consists of time-stamped events from a DVS observing poker cards being flipped, processed to center symbols and extract edge features. This dataset was used to demonstrate the real-time classification capabilities of the DYNAPS prototype.

Key Insights and Ideas

  • Memory Efficiency through Clustering and Tagging: The core insight is that by grouping neurons into clusters and using a tag-based routing system, the memory required for routing can be drastically reduced compared to traditional methods, especially for networks with clustered connectivity.
  • Heterogeneous Memory as a Key Enabler: Distributing different types of memory (SRAM for source, CAM for destination) directly within the processing nodes and routers is crucial for achieving "in-memory computing" and avoiding the von Neumann bottleneck.
  • Asynchronous Design for Power and Scalability: The use of QDI asynchronous circuits is fundamental to achieving low-power operation and robust communication, which are critical for scaling to large neuromorphic systems.
  • Flexibility for Diverse Architectures: The proposed routing scheme is designed to be highly configurable, allowing it to support a wide range of event-based neural network architectures, not just specific types like CNNs.

Key Results

  • Significant Memory Reduction: The proposed routing scheme reduces memory requirements from $F \log_2(N)$ bits/neuron (conventional) to approximately $2\sqrt{\alpha F}\log_2(\alpha C)\log_2(\alpha N)$ bits/neuron, with a factor of $\alpha=1$ leading to $2\sqrt{F}\log_2(C)\log_2(N)$. For example, a network with 1 million neurons and 10,000 fan-out would require less than 1.2 kbits/neuron with DYNAPS, compared to 160 kbits/neuron with conventional routing.
  • Prototype Chip Performance: A prototype chip with 1k neurons and a mixed-signal analog/digital design was fabricated. It demonstrated low power consumption (comparable to state-of-the-art digital solutions) and low latency.
  • Real-time CNN Classification: The prototype successfully implemented a Convolutional Neural Network (CNN) for real-time poker card suit recognition using DVS data, achieving classification results within 30 ms of stimulus onset.
  • Scalability Advantage: The paper highlights that DYNAPS offers linear scaling with CNN model size, unlike some other architectures that exhibit quadratic scaling due to dedicated routing cores.

Limitations and Future Work

  • I/O Speed Bottleneck: The current prototype's overall system performance is limited by the speed of its I/O circuits, not the internal asynchronous routers. This is expected to be less of an issue in larger, multi-chip designs where communication is primarily on-chip.
  • Timing Assumptions in CAM: While the asynchronous CAM array follows a handshaking protocol, its interface makes timing assumptions to minimize area. Worst-case scenario timing assumptions are made to ensure correct operation.
  • Future Work: The authors suggest that future revisions could improve throughput by increasing pipeline stages in the R3 router. They also point to the potential of using advanced fabrication processes (like 28nm FD-SOI) to further enhance performance and scalability, leading to a new generation of ultra-low-power, low-latency, and compact neuromorphic processors for real-time event-based sensory signal processing. The architecture is also well-suited for exploring emerging memory technologies like Resistive Random Access Memory (RRAM).

Summary updated

Abstract

Neuromorphic computing systems comprise networks of neurons that use asynchronous events for both computation and communication. This type of representation offers several advantages in terms of bandwidth and power consumption in neuromorphic electronic systems. However, managing the traffic of asynchronous events in large scale systems is a daunting task, both in terms of circuit complexity and memory requirements. Here we present a novel routing methodology that employs both hierarchical and mesh routing strategies and combines heterogeneous memory structures for minimizing both memory requirements and latency, while maximizing programming flexibility to support a wide range of event-based neural network architectures, through parameter configuration. We validated the proposed scheme in a prototype multi-core neuromorphic processor chip that employs hybrid analog/digital circuits for emulating synapse and neuron dynamics together with asynchronous digital circuits for managing the address-event traffic. We present a theoretical analysis of the proposed connectivity scheme, describe the methods and circuits used to implement such scheme, and characterize the prototype chip. Finally, we demonstrate the use of the neuromorphic processor with a convolutional neural network for the real-time classification of visual symbols being flashed to a dynamic vision sensor (DVS) at high speed.
BibTeX
@article{moradi2018scalable,
  author = {Saber Moradi and Qiao Ning and Fabio Stefanini and Giacomo Indiveri},
  title = {{A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)}},
  journal = {IEEE Trans. Biomed. Circuits Syst.},
  year = {2018},
  url = {https://openreview.net/forum?id=IOgyp45Tjo},
}
Figure