Computing at the Data Source with In-Memory Architectures

464 concepts294 papers2 proposals

In traditional computing, the processor and memory are distinct entities connected by a narrow bus. This "von Neumann architecture" creates a fundamental efficiency limit known as the memory wall: the energy and time required to shuttle data between storage and logic often exceed the cost of the computation itself. As Laborieux et al. and Liu et al. observe, this bottleneck is particularly severe for data-intensive tasks like neural network inference, where billions of weights must be fetched repeatedly.

To break this wall, researchers are developing In-Memory Computing (IMC) and Processing-in-Memory (PIM). Rather than bringing data to the processor, these architectures bring the processor to the data, or transform the memory itself into a computational engine.

In-Situ Computation: Memory as Logic

The most radical form of this idea is Compute-in-Memory (CIM), where the physical properties of memory cells are used to perform arithmetic. In these designs, the memory array does not just store a value; it participates in the calculation.

For example, in Resistive RAM (ReRAM) or Phase-Change Memory (PCM) crossbar arrays, weights are stored as the conductance of individual cells. By applying voltages representing input data to the rows, the resulting current at the bottom of the columns represents the sum of products (Vector-Matrix Multiplication)—a core operation of neural networks. Laborieux et al. demonstrate this using RRAM for ternary neural networks, while Mazzarello et al. highlight how PCM cells can integrate storage and processing in a single material.

This "in-situ" approach is not limited to exotic materials. Kosta et al. show that standard 8T-SRAM arrays can perform computations through charge accumulation and sharing, effectively turning a cache into a parallel processor. By executing operations where the weights reside, these systems eliminate the energy-intensive data movement entirely.

Near-Data Processing: Closing the Physical Gap

While CIM changes how memory works, Near-Data Processing (NDP) or Processing-Near-Memory (PNM) changes where the logic sits. By placing specialized accelerators or simple cores directly adjacent to memory, the system gains massive internal bandwidth.

Several strategies exist for this physical co-location:

  • 3D Stacking: Azarkhish describes "Smart Memory Cubes" that integrate streaming coprocessors into the logic base of 3D-stacked memory, providing high throughput for convolutional neural networks.
  • Peripheral Integration: In high-density storage like 3D NAND Flash, Hwang et al. propose placing processing logic directly beneath the memory array (Peripheral-under-Array), allowing for low-latency LLM token generation with negligible area overhead.
  • Memory Expansion Modules: For large-scale tasks like vector search in Retrieval-Augmented Generation (RAG), Quinn et al. and Kim et al. suggest offloading computations to CXL-enabled memory modules, bypassing the capacity limits of traditional GPUs.

Solving the Irregularity Problem

In-memory computing is uniquely suited for "irregular" workloads—tasks where data access patterns are unpredictable and branch-heavy, such as graph processing or genomics. Traditional CPUs struggle with these because they cannot effectively pre-fetch data into cache.

Gui et al. and Zheng et al. show that PIM accelerators can handle complex graph relationships by performing in-situ bitwise logic, drastically reducing the "memory wall" effect in triangle counting and hypergraph applications. Similarly, Nassereldine et al. demonstrate that performing sequence alignment directly inside memory chips can achieve a 28x speedup over high-end server CPUs by eliminating the overhead of moving genomic data.

The Path to Efficiency

The shift toward in-memory computing represents a transition from general-purpose, centralized processing to distributed, specialized architectures. As Fan et al. note, the field is evolving from simple bitwise operations toward complex floating-point arithmetic.

By fusing memory and logic—whether through the analog properties of a memristor or the tight integration of 3D stacking—these systems mimic the distributed efficiency of the biological brain. As Bose et al. and James et al. argue, this co-location is the key to achieving the orders-of-magnitude improvements in energy efficiency required for the next generation of AI and edge computing.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 293 papers
  2. Version 1 · 2026-10-04 · 255 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_in_memory_computing_processing_in_memory_fac16158,
  title = {in-memory computing (processing-in-memory)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-memory-computing-processing-in-memory/rdr_fac161581ddd42f6a54cc39f7e5efc49},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_in_memory_computing_processing_in_memory_fac16158_v2,
  title = {in-memory computing (processing-in-memory)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-memory-computing-processing-in-memory/rdr_fac161581ddd42f6a54cc39f7e5efc49/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake