Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

Geng Yuan, Zhiheng Liao, Xiaolong Ma, Yuxuan Cai, Zhenglun Kong, Xuan Shen, Jingyan Fu, Zhengang Li, Chengming Zhang, Hongwu Peng, Ning Liu, Ao Ren, Jinhui Wang, Yanzhi Wang

Abstract

Resistive RAM (ReRAM) is a promising technology for low-power Edge AI, but it suffers from high rates of "stuck-at" hardware faults. This paper introduces a novel approach that combines unstructured weight pruning and a differential crossbar mapping scheme to mask these faults. By representing pruned weights as the difference between two high-conductance cells, the method effectively masks "stuck-on" faults, which are the most prevalent defect type. Evaluation on CIFAR-10 and ImageNet shows that this combined method improves fault tolerance by an order of magnitude without requiring per-device retraining or additional hardware overhead.

Introduction

Deep Neural Networks (DNNs) are computationally intensive, posing significant challenges for deployment on resource-constrained Edge devices. ReRAM crossbars offer a solution via efficient in-memory analog matrix-vector multiplication. However, ReRAM cells are prone to "stuck-at-faults" (SAFs), where conductance is permanently high (stuck-on) or low (stuck-off). Existing solutions, such as per-device retraining or remapping, are not scalable for mass production. This work proposes a "universal" fault-tolerance method that works across all chips without individual tuning.

Methodology

The authors propose a software-hardware co-design approach consisting of two main components:

  1. Hierarchical Progressive Pruning: Unstructured pruning is used to create zeros in the weight matrix. A "stuck-off" (low conductance) fault at a pruned location causes no error because the intended value is already zero. The authors use a progressive algorithm to find the optimal sparsity ratio that maximizes fault tolerance without compromising baseline accuracy.
  2. Differential Crossbar Mapping: Hardware data indicates that stuck-on faults are 5.2x more common than stuck-off faults. The authors represent a weight $w$ as the difference between two cells ($w_a - w_b$). When a weight is pruned to 0, both $w_a$ and $w_b$ are set to their maximum conductance (1), resulting in $1 - 1 = 0$. Consequently, a stuck-on fault (a cell stuck at 1) perfectly overlaps with the intended state, masking the defect.
Comparison of structured and unstructured pruning techniques and their mapping to a 2D weight format.

Experimental Results

The method was evaluated on CIFAR-10 and ImageNet using ResNet18, as well as MS COCO using YOLOv4. A key insight is the existence of a "sweet spot" for sparsity: if pruning is too low, there are insufficient zeros to mask faults; if too high, the remaining weights become overly sensitive to any unmasked faults.

The proposed method tolerates failure rates nearly an order of magnitude higher than traditional mapping. For CIFAR-10 at a 0.001 failure rate, accuracy drops only 0.2%, compared to 1.2% for traditional methods. As shown in the performance curves, the proposed method maintains high accuracy even at failure rates where baseline models suffer catastrophic failure.

Accuracy vs. failure rate for CIFAR10 and ImageNet, comparing the proposed method against the original baseline.

Conclusion

The synergy between pruning and differential mapping provides a robust, zero-overhead solution for ReRAM-based AI. By repurposing pruning as a reliability tool, the authors enable high-performance Edge AI on faulty hardware without the need for expensive per-device calibration. Future work may extend this synergy to other analog defects such as read/write noise and conductance drift.

Summary updated

BibTeX
@inproceedings{yuan2021improving,
  author = {Geng Yuan and Zhiheng Liao and Xiaolong Ma and Yuxuan Cai and Zhenglun Kong and Xuan Shen and Jingyan Fu and Zhengang Li and Chengming Zhang and Hongwu Peng and Ning Liu and Ao Ren and Jinhui Wang and Yanzhi Wang},
  title = {{Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI}},
  booktitle = {ISQED},
  year = {2021},
  doi = {10.1109/isqed51717.2021.9424332},
}
Figure