Analog, In-memory Compute Architectures for Artificial Intelligence

Patrick Bowen, Guy Regev, Nir Regev, Bruno Pedroni, Edward Hanson, Yiran ChenarXiv 2023 · DOI · arXiv

Modern artificial intelligence is currently constrained by a fundamental "energy crisis." In conventional von Neumann architectures, the energy required to shuttle data between the processor and memory—often termed the "data movement tax"—frequently dwarfs the energy consumed by the actual arithmetic operations. This paper provides a rigorous analytical framework to evaluate how in-memory computing (IMC) and analog architectures can circumvent these bottlenecks as neural networks continue to scale.

The Challenge of Data Movement

Traditional CPUs and GPUs struggle with the "memory wall," where the separation of compute and storage creates a massive energy overhead. While Digital In-Memory Computing (DIM), such as systolic arrays or TPUs, attempts to mitigate this by performing calculations near the data, it remains limited by the energy costs of digital logic gates.

Analog architectures offer a different path. By utilizing the physical properties of devices (e.g., Ohm's law for multiplication or Kirchhoff's law for summation), these systems can perform massive linear operations—convolutions and matrix multiplications—with significantly lower energy footprints. However, these systems introduce new overheads, primarily the energy required for Digital-to-Analog Converters (DACs) and Analog-to-Digital Converters (ADCs).

Analog In-Memory Architecture

Optical 4F Systems: A 3D Scaling Advantage

A key contribution of this research is the analysis of Optical 4F systems. Unlike planar (2D) analog chips like Silicon Photonics or ReRAM crossbars, which are limited by surface area for inputs, Optical 4F systems utilize the third dimension. By propagating light through lenses to perform Fourier transforms in 3D space, these volumetric processors can handle millions of parallel inputs.

Optical 4F Convolutional System

The researchers demonstrate a profound "Scaling Advantage": in analog systems, the effective energy per operation ($e_{op}$) scales inversely with the processor size ($1/N$). While digital systems scale linearly (twice the math costs twice the energy), analog systems, once configured, perform calculations almost for free through physical propagation. This suggests that as AI problems grow larger, analog systems actually become more efficient.

Methodology and Benchmarking

To validate these theoretical models, the authors used architectural parameters from industry-standard Convolutional Neural Networks (CNNs), including DenseNet201, GoogLeNet, ResNet152, VGG16/19, and YOLOv3. By extracting median layer sizes and arithmetic intensities, they modeled performance across four architectures: standard CPUs, Digital In-Memory (DIM), Silicon Photonics (SP), and Optical 4F systems.

Key Results

The study's findings indicate that at a 32nm technology node, Optical 4F systems can achieve efficiencies approximately 100x greater than standard CPUs. While specialized digital IMC might reach ~5 TOPS/W, the analytical models predict that optical 4F systems can reach significantly higher efficiencies as they scale to larger input sizes.

Limitations and Future Outlook

Despite the efficiency gains, analog computing faces hurdles:

  1. Precision Penalty: ADC/DAC energy costs grow exponentially with bit precision ($2^B$), making high-precision scientific computing difficult.
  2. Hardware Maturity: Manufacturing high-resolution Spatial Light Modulators (SLMs) or metasurfaces at scale remains a challenge.
  3. Programmability: Current models focus on inference (forward-propagation). Adapting these "eigenspace" processors for general-purpose training (backpropagation) is an area for future exploration.

In conclusion, by moving beyond 2D silicon constraints to volumetric optical processing, AI hardware may finally break the efficiency plateaus of modern digital architectures.

Summary updated

BibTeX
@misc{bowen2023analog,
  author = {Patrick Bowen and Guy Regev and Nir Regev and Bruno Pedroni and Edward Hanson and Yiran Chen},
  title = {{Analog, In-memory Compute Architectures for Artificial Intelligence}},
  year = {2023},
  eprint = {2302.06417},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.06417},
}
Figure