Collapsing the Memory Wall with In-Memory Computing

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2255 papers326 conceptsfrozen 2026-10-04

In traditional computing, we treat memory like a warehouse and the processor like a factory. To do any work, data must be hauled from the warehouse, processed in the factory, and shipped back. This "von Neumann" architecture has served us for decades, but it has hit a physical limit known as the memory wall.

As Said Hamdioui, Pierre‐Emmanuel Gaillardon, and Dietmar Fey explain in Guest Editorial Memristive-Device-Based Computing, the energy cost of moving data between these two points now dwarfs the cost of the actual computation. For data-heavy tasks like AI inference or large-scale scientific simulations, the "shipping" costs are bankrupting our energy budgets.

The research direction emerging from these 162 papers is a fundamental shift: In-Memory Computing (IMC). Instead of moving data to the processor, we move the processing into the memory.

The Intuition: Co-localization

The core idea is co-localization. In a biological brain, there is no separate "CPU" and "RAM." Synapses store information and process signals simultaneously. Giacomo Indiveri and Shih-Chii Liu argue in Memory and Information Processing in Neuromorphic Systems that by making memory elements act as active operators rather than passive storage, we can eliminate the data movement bottleneck entirely.

This isn't just a theoretical dream; it is being implemented across the entire memory hierarchy, from high-speed SRAM to dense non-volatile storage.

Near-Memory vs. In-Memory

The field generally splits into two strategies for bringing logic and data together:

  1. Near-Memory Processing (NMP): This involves placing specialized logic units right at the edge of the memory banks. For example, Liu Ke and Carole-Jean Wu developed RecNMP, which offloads sparse embedding lookups—a major bottleneck in recommendation systems—directly to DRAM modules. This reduced latency by nearly 10x by avoiding the system bus.
  2. In-Memory Computing (IMC): This is more radical. It uses the physical properties of the memory cells themselves to perform math. In STT-CiM, Shubham Jain and Kaushik Roy show how the resistive properties of Magnetic RAM (MRAM) can be used to perform vector operations directly within the array.

How it Works: The Physics of Math

To understand why this is so efficient, consider a matrix-vector multiplication—the "inner loop" of almost all AI. In a standard processor, you must fetch every weight and every input, multiply them, and sum them up.

In an IMC architecture using Resistive RAM (RRAM) or SRAM, you can treat the memory array as a physical grid. By applying voltages to the rows (inputs) and measuring the current at the columns, the circuit performs the multiplication and addition simultaneously via Ohm’s Law and Kirchhoff’s Current Law. As Qunsong Zeng and colleagues demonstrate in Realizing Ultra-Fast and Energy-Efficient Baseband Processing Using Analogue Resistive Switching Memory, this can lead to a $10^6$ improvement in energy efficiency for signal processing tasks.

Beyond Simple Math: Associative and Digital IMC

While analog IMC is powerful, it faces challenges with noise and precision. This has led researchers like Mariam Rakka and Ahmed M. Eltawil to explore Associative Processors. In SoftmAP, they show how bit-serial operations within SRAM can perform complex non-linear functions like Softmax with massive energy savings compared to high-end GPUs.

Similarly, Mohsen Imani and Farinaz Koushanfar proposed RAPIDNN, a fully digital PIM framework. By turning memory into a massive lookup table for pre-computed results, they bypass the need for power-hungry analog-to-digital converters while maintaining high compute density.

Why This Matters Now

The urgency of this research is driven by two frontiers: the Edge and Large Language Models (LLMs).

At the edge, devices like sensors and wearables have tiny power budgets. Petar Jokic and Luca Benini highlight in A Construction Kit for Efficient Low Power Neural Network Accelerator Designs that IMC is the only way to achieve the "TOPS/W" (Tera-Operations Per Watt) required for real-time intelligence on a battery.

For LLMs, the bottleneck is memory bandwidth. As Jinhao Li notes in Large Language Model Inference Acceleration, the massive size of these models makes data movement the primary limit on speed. By integrating computation directly into the storage structures, we can finally move past the von Neumann bottleneck and enable the next generation of ubiquitous, efficient AI.

Papers in this version

Showing 60 of 255 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 293 papers
  2. Version 1 · 2026-10-04 · 255 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_in_memory_computing_processing_in_memory_fac16158,
  title = {in-memory computing (processing-in-memory)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/in-memory-computing-processing-in-memory/rdr_fac161581ddd42f6a54cc39f7e5efc49},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_collapsing_the_memory_wall_with_in_memor_fac16158_v1,
  title = {Collapsing the Memory Wall with In-Memory Computing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/collapsing-the-memory-wall-with-in-memory-computing/rdr_fac161581ddd42f6a54cc39f7e5efc49/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}