PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference

Aayush Ankit, Izzat El Hajj, Sai Rahul Chalamalasetti, Geoffrey Ndu, Martin Foltin, R. Stanley Williams, Paolo Faraboschi, Wen-mei W. Hwu, John Paul Strachan, Kaushik Roy, Dejan S. Milojicic

Introduction

As digital CMOS scaling approaches the "energy wall," traditional architectures struggle to keep pace with the computational demands of modern machine learning. Memristor crossbars have emerged as a promising alternative, capable of performing Matrix-Vector Multiplication (MVM)—the primary bottleneck in AI workloads—directly in memory using analog signals. This approach significantly reduces the energy cost associated with data movement.

However, early memristor-based accelerators were often fixed-function ASICs, lacking the flexibility to handle diverse neural network topologies like LSTMs or complex control flows. PUMA (Programmable Ultra-efficient Memristor-based Accelerator) addresses this by providing a full-stack solution that bridges the gap between ultra-efficient analog computing and general-purpose programmability.

Core Architecture

PUMA utilizes a hierarchical hardware organization consisting of Cores, Tiles, and Nodes. At the heart of each core is the Matrix-Vector Multiplication Unit (MVMU), which leverages memristor crossbars to execute high-speed operations. To maintain a small hardware footprint, the architecture employs a Vector Functional Unit (VFU) that uses Temporal SIMD, executing wide vector operations over multiple cycles for non-linear functions like ReLU or Sigmoid.

The PUMA Core Architecture, illustrating the integration of the MVMU, VFU, and control logic.

Another key innovation is the ROM-Embedded RAM, a modification to standard SRAM that allows it to store look-up tables for transcendental functions (such as $tanh$) without increasing area or latency. This integration ensures that the digital components do not bottleneck the analog core's efficiency.

Analog MVM and Bit Slicing

The MVMU performs 16-bit MVM operations through a technique known as bit-slicing. Since individual memristor cells typically store lower precision (e.g., 2 bits), PUMA combines multiple cells to represent higher-precision weights.

Analog MVM mechanism and the bit-slicing approach used to achieve high-precision computation.

As shown in the figure above, the analog MVM process converts input voltages ($V_i$) through Digital-to-Analog Converters (DACs), which then interact with the memristor conductances ($g_{ij}$). The resulting currents ($I_j$) are integrated and converted back to digital signals via Analog-to-Digital Converters (ADCs). A "Shift-&-Add" unit then reconstructs the final high-precision result from the bit-sliced components.

Software Stack and Programmability

To make this analog hardware accessible, PUMA includes a specialized Instruction Set Architecture (ISA) and a custom compiler. The ISA allows software to explicitly manage data movement and spatial execution. The PUMA compiler handles the complex task of partitioning tensors across thousands of cores, scheduling instructions to minimize register pressure, and coalescing MVM operations to maximize parallelism.

One of the most significant findings of the PUMA research is that the "programmability tax"—the overhead of adding an ISA and general-purpose units—is only about 20-30% compared to fixed-function memristor ASICs. This is a remarkably low price for the flexibility to run any machine learning model.

Performance and Results

PUMA was evaluated against a diverse suite of benchmarks, including MLPs, Deep LSTMs for translation, and CNNs like Vgg19. When compared to state-of-the-art digital hardware, the results are striking:

  • Efficiency: PUMA achieves 837 GOPS/s/W, representing up to a 2,446× energy reduction over Pascal GPUs for Deep LSTM workloads.
  • Latency: It provides up to 66× lower latency than GPUs for inference tasks.
  • Area Efficiency: PUMA demonstrates 8.3× higher peak area-efficiency than the Google TPU, as it eliminates the need for heavy weight-reuse strategies by performing computation exactly where the weights are stored.

Conclusion and Future Work

PUMA proves that analog memristor computing can be made programmable without sacrificing its inherent efficiency. While currently focused on inference due to the high write latency of memristors, PUMA sets a new standard for domain-specific accelerators. Future work may address the ADC/DAC bottleneck, which remains a primary consumer of area and power in mixed-signal designs, and explore higher-density multi-bit memristor cells to further push the boundaries of AI efficiency.

Summary updated

BibTeX
@inproceedings{ankit2019puma,
  author = {Aayush Ankit and Izzat El Hajj and Sai Rahul Chalamalasetti and Geoffrey Ndu and Martin Foltin and R. Stanley Williams and Paolo Faraboschi and Wen-mei W. Hwu and John Paul Strachan and Kaushik Roy and Dejan S. Milojicic},
  title = {{PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference}},
  booktitle = {ASPLOS},
  year = {2019},
  doi = {10.1145/3297858.3304049},
}
Figure