Abstract
In-Memory Computing (IMC) platforms such as analog crossbars are gaining focus as they facilitate the acceleration of low-precision Deep Neural Networks (DNNs) with high area- & compute-efficiencies. However, the intrinsic non-idealities in crossbars, which are often non-deterministic and non-linear, degrade the performance of the deployed DNNs. In addition to quantization errors, most frequently encountered non-idealities during inference include crossbar circuit-level parasitic resistances and device-level non-idealities such as stochastic read noise and temporal drift. In this work, our goal is to closely examine the distortions caused by these non-idealities on the dot-product operations in analog crossbars and explore the feasibility of a nearly training-less solution via crossbar-aware fine-tuning of batchnorm parameters in real-time to mitigate the impact of the non-idealities. This enables reduction in hardware costs in terms of memory and training energy for IMC noise-aware retraining of the DNN weights on crossbars.
Introduction
The demand for efficient Deep Neural Network (DNN) inference on edge devices has led to the rise of In-Memory Computing (IMC). By utilizing analog memristive crossbars (e.g., RRAM, PCM, or FeFET), IMC architectures can perform matrix-vector multiplications directly within the memory array. This eliminates the energy-intensive data movement between memory and processor, effectively bypassing the "memory wall." However, the transition from digital precision to analog physics introduces significant hardware-level non-idealities that can severely degrade model accuracy.
The Challenge of Hardware Non-idealities
Analog crossbars suffer from three primary categories of noise and distortion:
- Stochastic Read Noise: Random fluctuations occurring during each read operation, introducing non-deterministic errors.
- Temporal Drift: A phenomenon where the electrical conductance of memristive devices shifts over time, leading to a gradual decay of the stored weights.
- Parasitic Resistance: Circuit-level resistance in the interconnects and wires of the crossbar that distorts the voltage drops and current sums, leading to non-linearities in the dot-product operations.
Traditionally, these issues are addressed through hardware-aware retraining, where the entire model is retrained to adapt to the specific noise profile of the hardware. While effective, this process is computationally expensive, requiring significant memory to store gradients and substantial energy—resources that are often scarce on edge devices.
Proposed Solution: Batchnorm Fine-tuning
This research proposes a "nearly training-less" alternative: Batch Normalization (BN) fine-tuning. In a standard DNN, BN layers normalize activations using a running mean and variance, followed by a learnable linear transformation using scale ($\gamma$) and shift ($\beta$) parameters.
The authors explore three levels of mitigation:
- No Mitigation: The model is deployed directly onto the noisy hardware.
- BN Adaptation: Only the running mean and variance are updated using a small set of calibration data, without any backpropagation.
- BN Fine-tuning: The weights of the convolutional layers are frozen (mapped to the analog crossbars), and only the digital $\gamma$ and $\beta$ parameters are optimized via backpropagation.
By focusing only on the BN parameters, the system avoids the need to calculate or store gradients for millions of weights, drastically reducing the hardware overhead.
Quantifying Distortion via PCA
To understand why BN fine-tuning is effective, the researchers employed Principal Component Analysis (PCA) to visualize the distortion of weights. When plotting ideal weights against their noisy hardware counterparts, the distribution typically forms a "dumbbell" shape. The second principal component (PC2) serves as a metric for the non-linear shift caused by noise. The study found that BN fine-tuning remains highly effective as long as the PC2 value remains below a critical threshold (approximately $1.0 \times 10^{-2}$). Beyond this point, the non-linear distortion becomes too severe for the linear BN parameters to compensate.
Experimental Results
The approach was evaluated using VGG16 and ResNet-18 architectures on the CIFAR10 and TinyImagenet datasets across various memristive technologies (FeFET, PCM, RRAM).
Accuracy Recovery
On FeFET-based hardware with significant read noise ($\sigma=0.15$), VGG16 accuracy initially plummeted. However, BN fine-tuning successfully recovered the accuracy to approximately 87%, nearly reaching the noise-free baseline. The method proved particularly effective against temporal drift, as drift often acts as a scaling factor—a distortion that BN layers are inherently designed to handle.
Hardware Efficiency
The primary advantage of BN fine-tuning is its efficiency:
- Memory Savings: Compared to full weight retraining, BN fine-tuning achieved a ~95% reduction in memory overhead.
- Energy Savings: The approach resulted in a ~52% reduction in training energy per epoch.
Device Resilience
The study also highlighted that the choice of memristive device matters. Devices with higher ON/OFF ratios, such as certain PCM configurations, demonstrated greater inherent resilience to noise, providing a stronger signal relative to the noise floor.
Conclusion and Limits
While BN fine-tuning offers a powerful and efficient solution for mitigating hardware noise, it is not universal. In scenarios with extreme parasitic resistance—particularly in larger crossbars (e.g., 128x128)—the resulting non-linearities can exceed the compensatory capabilities of BN layers. Future work will involve implementing these techniques on physical hybrid analog-digital chips to validate these simulation-based findings in real-world environments. Nevertheless, this "nearly training-less" approach represents a significant step toward making high-efficiency IMC hardware viable for long-term, reliable edge AI deployment.
Summary updated