Abstract
Analogue memory-based Deep Neural Networks (DNNs) provide energy-efficiency and per-area throughput gains relative to state-of-the-art digital counterparts such as graphic processing units (GPUs). Recent advances focus largely on hardware-aware algorithmic training and improvements in circuits, architectures, and memory device characteristics. Optimal translation of software-trained weights into analogue hardware weights---given the plethora of complex memory non-idealities---represents an equally important goal in realizing the full potential of this technology. We report a generalized computational framework that automates the process of crafting complex weight programming strategies for analogue memory-based DNNs, in order to minimize accuracy degradations during inference, particularly over time. This framework is agnostic to DNN structure and is shown to generalize well across Long Short-Term Memory (LSTM), Convolution Neural Networks (CNNs), and Transformer networks. Being a highly-flexible numerical heuristic, our approach can accommodate arbitrary device-level complexity, and is thus broadly applicable to a variety of analogue memories and their continually evolving device characteristics. Interestingly, this computational technique is capable of optimizing inference accuracy without the need to run inference simulations or evaluate large training, validation, or test datasets. Lastly, by quantifying the limit of achievable inference accuracy given imperfections in analogue memory, weight programming optimization represents a unique and foundational tool for enabling analogue memory-based DNN accelerators to reach their full inference potential.
Introduction
Deep Neural Networks (DNNs) are traditionally executed on digital hardware, such as CPUs and GPUs, which are constrained by the "von Neumann bottleneck"—the energy-intensive movement of data between memory and processors. Analogue in-memory computing (AIMC) offers a promising alternative by performing computations directly within memory arrays (e.g., Phase-Change Memory or ReRAM) using Ohm’s and Kirchhoff’s laws.
However, physical analogue devices introduce significant non-idealities that can degrade model performance:
- Programming Error: Difficulty in setting a device to a precise target conductance.
- Read Noise: Electrical fluctuations that introduce uncertainty into calculations.
- Conductance Drift: Temporal changes in the physical state of the memory, causing weights to decay and accuracy to drop over time.
While hardware-aware training (HAT) mitigates some of these issues, it often falls short of maintaining long-term accuracy. This research addresses the "translation" problem: how to optimally map mathematical software weights to a complex set of physical devices to minimize errors over time.
Methodology: Differential Weight Evolution (DWE)
The researchers developed the Differential Weight Evolution (DWE) framework to automate the creation of weight programming strategies.
Weight Slicing and Optimization
Each synaptic weight is represented by four physical conductances: a "Most Significant Pair" (MSP) for the bulk weight and a "Least Significant Pair" (LSP) for fine-tuning. Because hardware models are noisy and non-convex, standard gradient descent is ineffective. Instead, DWE uses an evolutionary heuristic to find the optimal mapping strategy.
The Proxy Metric
A key innovation is the use of a time-averaged normalized mean-squared-error (MSE) metric as a proxy for inference accuracy. This allows the framework to optimize programming strategies without the computational burden of running full inference simulations on massive datasets. The researchers demonstrated that minimizing this mathematical proxy correlates perfectly with high inference fidelity.
Dimensionality Reduction
To ensure the search remains tractable for models with millions of weights, the framework optimizes for a small set of discretized weight intervals. It then employs linear interpolation and symmetry to cover the entire weight distribution.
Experimental Validation
The framework's "structure-agnostic" nature was validated across three distinct architectures and datasets:
- LSTM: Evaluated using the Penn Treebank dataset.
- CNN (ResNet-32): Evaluated using the CIFAR-10 dataset.
- Transformer (BERT-base): Evaluated using the MNLI (Multi-Genre Natural Language Inference) dataset.
Results and Discussion
The DWE framework yielded several critical insights:
- MSP/LSP Dynamics: The optimizer revealed a counter-intuitive preference for using the LSP whenever possible. This is because errors in the MSP are amplified by a significance factor ($F$), whereas LSP errors remain relatively small.
- Drift Mitigation: By anticipating device drift over a one-month period, the framework selects initial programming targets that maintain weight fidelity for longer durations.
- Performance Gains: The optimized strategy reduced hardware weight errors by 39% immediately after programming and by 17% after one month compared to naive mapping strategies.
- Iso-accuracy: For all tested models, the framework achieved inference accuracy levels very close to those of perfect digital floating-point models.
Conclusion
The DWE framework provides a foundational tool for maximizing the potential of analogue memory-based DNN accelerators. By quantifying the limits of achievable accuracy given specific hardware imperfections, it allows engineers to objectively evaluate new materials and device characteristics. Future work may explore local optimization for individual crossbar arrays and the integration of energy consumption constraints into the cost function to balance accuracy with power efficiency.
Summary updated