The Lynchpin of In-Memory Computing: A Benchmarking Framework for Vector-Matrix Multiplication in RRAMs

Md Tawsif Rahman Chowdhury, Huynh Quang Nguyen Vo, Paritosh Ramanan, Murat Yildirim, Gozde TutuncuogluarXiv 2024 · DOI · arXiv

Introduction

Modern computing is increasingly constrained by the Von Neumann bottleneck, where the constant movement of data between separate processing and memory units consumes up to 62.7% of energy in some server workloads. Resistive Random Access Memory (RRAM) offers a promising alternative through "In-Memory Computing" (IMC). By arranging RRAM devices in crossbar arrays, Vector-Matrix Multiplication (VMM)—the core operation for AI and linear algebra—can be performed directly where data is stored, reducing computational complexity from $O(n^2)$ to $O(1)$.

However, RRAM hardware is physically imperfect. Non-idealities such as non-linear conductance and cycle-to-cycle variation introduce mathematical errors that can compromise computational accuracy. This paper introduces MELISO (In-Memory Linear Solver), a framework designed to quantify and model these errors.

The MELISO Framework

MELISO is an end-to-end benchmarking framework that bridges high-level software specifications with low-level hardware physics. It allows researchers to simulate how device-level noise propagates through complex computations.

The MELISO framework architecture, integrating Python-based user interfaces with high-performance C++ simulation engines to model RRAM crossbar arrays.

The framework integrates three primary layers:

  • MelisoPy (Python): A user interface for defining device types, crossbar dimensions, and error tolerances.
  • Cython Wrapper: A high-performance bridge passing data to the simulation engine.
  • MLP+ NeuroSim (C++): An enhanced version of the industry-standard NeuroSim tool used to simulate hardware physics and hardware mapping.

Key Insights: The Impact of Non-Idealities

Using a Monte Carlo evaluation method with 1,000 random $32 \times 32$ matrices, the researchers identified several critical factors affecting VMM reliability across different material chemistries, including Ag:a-Si, $TaO_x/HfO_x$, $AlO_x/HfO_2$, and EpiRAM.

Non-Linearity as a Primary Culprit

One of the most significant findings is that the relationship between device non-linearity and error variance is exponential. Even small increases in non-linearity lead to massive, unpredictable errors in the final VMM output, making it the primary driver of computational inaccuracy.

Impact of non-linearity magnitude on VMM error, showing the exponential growth of error variance.

Statistical Realities and Material Hierarchy

The study revealed that RRAM errors do not follow a standard Gaussian (bell-curve) distribution. Instead, they exhibit high skewness and heavy tails, requiring complex statistical models like Johnson $S_u$ or SHASH for accurate representation.

Material choice also plays a decisive role. EpiRAM emerged as the top performer due to its superior memory window and low variation. In contrast, the error variance for $AlO_x/HfO_2$ was nearly 30 times higher than that of EpiRAM, highlighting the critical importance of material selection in hardware design.

Results and Future Directions

The benchmarking demonstrated that increasing precision—from 1-bit to 11-bit weight bits—dramatically reduces error magnitude. Furthermore, maintaining a large Memory Window (the ratio between maximum and minimum conductance) is essential for distinguishing signal levels and mitigating noise.

Future work will focus on scaling the framework to larger crossbar arrays where line resistance becomes a factor and extending the analysis to more complex optimization solvers and neuromorphic virtualization techniques to mask hardware quirks.

Summary updated

Abstract

The Von Neumann bottleneck, a fundamental challenge in conventional computer architecture, arises from the inability to execute fetch and data operations simultaneously due to a shared bus linking processing and memory units. This bottleneck significantly limits system performance, increases energy consumption, and exacerbates computational complexity. Emerging technologies such as Resistive Random Access Memories (RRAMs), leveraging crossbar arrays, offer promising alternatives for addressing the demands of data-intensive computational tasks through in-memory computing of analog vector-matrix multiplication (VMM) operations. However, the propagation of errors due to device and circuit-level imperfections remains a significant challenge. In this study, we introduce MELISO (In-Memory Linear Solver), a comprehensive end-to-end VMM benchmarking framework tailored for RRAM-based systems. MELISO evaluates the error propagation in VMM operations, analyzing the impact of RRAM device metrics on error magnitude and distribution. This paper introduces the MELISO framework and demonstrates its utility in characterizing and mitigating VMM error propagation using state-of-the-art RRAM device metrics.
BibTeX
@misc{chowdhury2024lynchpin,
  author = {Md Tawsif Rahman Chowdhury and Huynh Quang Nguyen Vo and Paritosh Ramanan and Murat Yildirim and Gozde Tutuncuoglu},
  title = {{The Lynchpin of In-Memory Computing: A Benchmarking Framework for Vector-Matrix Multiplication in RRAMs}},
  year = {2024},
  eprint = {2409.06140},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.06140},
}
Figure