Processing-In-Memory Acceleration of Convolutional Neural Networks for Energy-Effciency, and Power-Intermittency Resilience

Arman Roohi, Shaahin Angizi, Deliang Fan, Ronald F. DeMara

Introduction: The Power and Memory Wall

Modern AI faces a significant "power and memory wall." In traditional architectures like CPUs and GPUs, data must constantly travel between the processor and memory. This movement consumes significantly more energy and time than the actual computation, especially for deep Convolutional Neural Networks (CNNs) with millions of parameters. As shown in Figure 1, convolutional layers dominate the execution time in both CPU and GPU environments, accounting for 75-84% of the total workload.

Convolutional layers represent the primary computational bottleneck in standard architectures.

Furthermore, the rise of the Internet of Things (IoT) introduces the challenge of "power intermittency." Many IoT devices rely on energy harvesting (e.g., solar or vibration) and frequently lose power. Standard processors lose their progress during these shutdowns, wasting energy by constantly restarting tasks. This paper introduces a solution that addresses both the data movement bottleneck and the power failure problem simultaneously.

Proposed Architecture: SOT-MRAM PIM

The researchers propose a Processing-In-Memory (PIM) architecture based on Spin-Orbit Torque Magnetic RAM (SOT-MRAM). Unlike standard RAM, SOT-MRAM is non-volatile and can be modified to perform logic operations directly on the bitlines.

Technical Innovations

  • Bit-wise Convolution: Instead of high-precision floating-point math, the system uses low bit-width quantization. This allows complex multiplications to be replaced by simple bit-wise AND operations and bit-counting.
  • AND-Accumulation Method: Rather than using slow serial counters, the authors designed a hardware-conforming accumulation phase using 4:2 Compressors and Adaptive Shift Registers (ASR). This allows the system to count "1"s in a single clock cycle.
  • Non-Volatile Full Adder (NV-FA): By integrating non-volatile flip-flops into the adder circuits, the system can "checkpoint" its state. If power fails, the system resumes exactly where it left off once power returns.
The bit-wise convolution process: mapping quantized input and kernel data into the memory array for parallel AND operations and accumulation.

Key Insights: Algorithmic-Hardware Co-design

The core innovation lies in the co-design of algorithms and hardware. By quantizing the network to low bit-widths (e.g., 1-bit weights and 4-bit activations), the mathematical operations become simple enough to execute entirely within the memory's internal wiring.

Traditional PIM designs often use serial counters to sum results, creating a new bottleneck. This architecture utilizes a compressor tree within the memory sub-array to enable massive parallelism. Furthermore, because SOT-MRAM is naturally non-volatile, the memory serves as both storage and the computational substrate. Making the logic gates non-volatile creates a "normally-off" machine with near-zero standby power.

Evaluation and Results

The architecture was evaluated using the SVHN (Street View House Numbers) and ImageNet (AlexNet) benchmarks, comparing the SOT-MRAM design against traditional CMOS-only ASICs and ReRAM (Resistive RAM) accelerators.

  • Energy Efficiency: The design is ~5.4× more energy-efficient than ReRAM-based accelerators and ~9.7× more efficient than traditional CMOS-only approaches.
  • Performance: It achieves a 9× speedup over ReRAM and a 13.5× speedup over CMOS-only designs.
  • Memory Footprint: Bit-wise quantization reduced memory storage requirements by 11.7× compared to full-precision models. For AlexNet, the memory requirement dropped to ~40MB.
  • Accuracy: Despite heavy quantization, inference accuracy remained high, with only a ~0.7% drop for SVHN at specific bit-widths.

Limitations and Future Outlook

While highly efficient, write operations in MRAM remain energy-intensive compared to reads. Future work aims to reduce write energy by tuning the thermal barriers of Magnetic Tunnel Junctions (MTJs) and optimizing circuit overhead by selectively storing only critical bits (like "carry" bits) during power failures. Scaling the architecture to deeper networks like ResNet or Transformers is also a key objective.

Summary updated

BibTeX
@inproceedings{roohi2019processing,
  author = {Arman Roohi and Shaahin Angizi and Deliang Fan and Ronald F. DeMara},
  title = {{Processing-In-Memory Acceleration of Convolutional Neural Networks for Energy-Effciency, and Power-Intermittency Resilience}},
  booktitle = {ISQED},
  year = {2019},
  doi = {10.1109/isqed.2019.8697572},
}
Figure