In traditional computer architectures, memory and processing are strictly separated. To perform a simple addition or multiplication, data must travel from a memory unit, across a bus, into the central processing unit (CPU) or graphics processing unit (GPU), and then back to memory. This constant shuffling of data creates a massive energy and latency bottleneck known as the von Neumann bottleneck.
For deep learning workloads, this bottleneck is devastating. Deep neural networks require billions of matrix-vector multiplications (VMMs), meaning that processors spend far more energy moving weight matrices across wires than actually computing products. As highlighted by Mostafa Rahimi Azghadi et al. in Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications, this data movement overhead severely limits our ability to deploy "always-on" intelligent systems in power-constrained environments like edge medical devices.
Analog In-Memory Computing (AIMC) fundamentally re-imagines this paradigm. Instead of moving data to the processor, AIMC performs mathematical operations directly inside the memory array itself. By mapping numbers to physical properties like electrical conductance and voltage, AIMC executes massive parallel multiplications using the fundamental laws of physics.
Computing with Physics: Ohm's and Kirchhoff's Laws
To understand how an analog memory array can compute, imagine a grid of wires called a crossbar array. The vertical wires represent the columns, and the horizontal wires represent the rows. At every intersection of a row and a column sits a programmable, non-volatile memory element (such as a memristor, phase-change memory cell, or resistive RAM cell) whose electrical conductance can be precisely adjusted.
Inputs (Voltages)
V₁ -----[ G₁₁ ]-----[ G₁₂ ]-----
| |
V₂ -----[ G₂₁ ]-----[ G₂₂ ]-----
| |
v v
Output₁ Output₂
(Current) (Current)
Let's look at how this structure naturally computes a dot product—the core operation of matrix multiplication. Suppose we want to multiply an input vector $\mathbf{x}$ by a weight matrix $\mathbf{W}$ to get an output vector $\mathbf{y}$.
-
Multiplication via Ohm's Law: We program the matrix weights $W_{ij}$ into the crossbar array as the physical conductance values ($G_{ij}$) of the memory devices. We then feed the input vector elements $x_i$ into the rows simultaneously as analog voltages ($V_i$). According to Ohm's Law ($I = V \cdot G$), the current flowing through any individual device at intersection $(i, j)$ is exactly the product of the input voltage and the device's conductance: $$I_{ij} = V_i \cdot G_{ij}$$
-
Summation via Kirchhoff's Current Law: Because all the devices in a single column are connected to the same vertical wire, their individual currents naturally merge. Kirchhoff's Current Law states that the total current exiting at the bottom of a column wire is simply the sum of all currents flowing into it: $$I_j = \sum_i V_i \cdot G_{ij}$$
By reading the total current out of each column, we obtain the result of the entire matrix-vector multiplication in a single step. As Zhengwu Liu et al. demonstrate in Neural signal analysis with memristor arrays towards high-efficiency brain–machine interfaces, this physical mapping reduces the time complexity of a VMM operation to a constant factor ($O(1)$), completely bypassing the sequential loops and clock cycles required by digital processors.
The Hardware Substrates: Current vs. Charge
The physical implementation of AIMC generally falls into two categories depending on how the state is stored and manipulated:
- Current-Domain Resistive Crossbars: Devices like Resistive RAM (ReRAM) and Phase-Change Memory (PCM) act as variable resistors. They alter their atomic structure (e.g., by forming conductive filaments or changing crystalline phases) to retain a specific conductance value even when powered off.
- Charge-Domain Capacitive Arrays: Instead of measuring continuous current through a resistor, some architectures use standard memory cells paired with tiny capacitors. For example, Yan He in CAP-RAM: A Charge-Domain In-Memory Computing 6T-SRAM for Accurate and Precision-Programmable CNN Inference details an approach where multiplication is performed via voltage sampling, and accumulation occurs through charge sharing across local capacitors. This charge-domain approach offers superior linearity and is less sensitive to the thermal noise that can plague current-based resistive arrays.
Overcoming Real-World Non-Idealities
While computing at the speed of physics sounds ideal, analog hardware introduces unique physical challenges that digital systems do not face. Researchers have had to develop clever hardware-algorithm co-designs to make AIMC practical.
Handling Negative Numbers (Four-Quadrant Multiplication)
Real-world neural networks require both positive and negative weights and inputs. However, physical electrical conductance can never be negative.
To solve this, traditional analog accelerators had to split operations into multiple sequential phases, adding latency. To bypass this limitation, Manuel Le Gallo et al. developed a technique detailed in Demonstration of 4-quadrant analog in-memory matrix multiplication in a single modulation. By applying dual-polarity voltages simultaneously across a single-phase modulation scheme, the system can compute fully signed four-quadrant matrix multiplications in a single read cycle, cutting energy and latency overhead in half.
Turning Noise into a Feature
Analog hardware is inherently noisy. Device-to-device variations and stochastic drift mean that a programmed conductance value might fluctuate slightly over time. While this is a liability for deterministic digital logic, it can be an asset for probabilistic algorithms.
In Exploiting Oxide Based Resistive RAM Variability for Bayesian Neural Network Hardware Design, Akul Malhotra et al. show how to turn this hardware "bug" into a feature. They exploit the natural cycle-to-cycle stochasticity of ReRAM devices to serve as a built-in random number generator, performing the complex Gaussian sampling required by Bayesian Neural Networks directly in hardware without any software overhead.
The Flexibility Gap
Analog crossbars excel at dense, uniform matrix operations, but modern neural networks often include highly specialized, sparse, or depthwise convolutional layers that do not map cleanly to a rigid grid.
To prevent these specialized layers from bottlenecking the entire system, the authors of End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet? propose a hybrid architecture. By pairing an analog in-memory accelerator (dedicated to heavy, dense VMM tasks) with a cluster of highly flexible digital RISC-V cores (which handle specialized layers and data formatting), they achieve massive end-to-end energy savings without sacrificing the flexibility needed for state-of-the-art neural network architectures.
Key Foundational Work
Memristor-based Synaptic Networks and Logical Operations Using In-Situ Computing (2011) — contributed 3 concept elements to this direction.