FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Kai Zhao, Sheng Di, Sihuan Li, Xin Liang, Yujia Zhai, Jieyang Chen, Kaiming Ouyang, Franck Cappello, Zizhong Chen

TL;DR

This paper introduces FT-CNN, a novel framework for making Convolutional Neural Networks (CNNs) resilient to soft errors, which are common in modern hardware. It proposes a set of Algorithm-Based Fault Tolerance (ABFT) schemes that can be applied to any CNN convolution implementation, offering a good balance between error correction capabilities and runtime overhead.

What Problem Does This Paper Address?

CNNs are increasingly used in safety-critical applications like autonomous vehicles and high-performance scientific simulations. However, these applications are vulnerable to "soft errors" – transient faults caused by factors like cosmic rays or voltage fluctuations. These errors can lead to Silent Data Corruption (SDC), where the CNN produces incorrect results without any indication of failure, potentially causing catastrophic consequences. Traditional fault tolerance methods like Error-Correcting Code (ECC) or instruction duplication are often too costly in terms of performance or memory overhead for CNNs, and existing ABFT techniques are typically limited to specific matrix-matrix multiplication implementations, which may not be the most efficient for CNNs. This paper aims to provide an efficient and general solution for protecting CNN inference against soft errors.

Methods

The core of this work lies in developing new Algorithm-Based Fault Tolerance (ABFT) schemes specifically for CNN convolution operations. The authors propose four novel ABFT schemes:

  • Full Checksum (FC): Offers high error detection and correction capabilities by using checksums derived from both input feature maps (D) and kernels (W), as well as output checksums.
  • Row Checksum (RC) and Column Checksum (CIC): These are symmetric schemes that provide good error correction for errors within a row or column of the output, respectively. They offer a better balance between overhead and protection than FC.
  • Checksum-of-Checksum (CoC): This scheme uses checksums derived from other checksums, offering lower overhead but also lower error correction capabilities. It's particularly useful for error detection.

A key innovation is the multischeme workflow that intelligently combines these schemes. This workflow prioritizes error detection using the low-overhead CoC-D scheme. If an error is detected, it attempts correction using CoC, then RC/CIC, and finally FC as a last resort for high correction capability. This layered approach aims to minimize overhead by only invoking more expensive correction schemes when necessary. The authors also extend their schemes to support bias operations, grouped convolutions, and backpropagation.

Datasets

The paper evaluates its proposed FT-CNN framework using the ImageNet dataset. This is a standard benchmark for image classification tasks and is used here to test the performance and effectiveness of the fault tolerance schemes on popular CNN models.

Key Insights and Ideas

  • Convolution is the Bottleneck: The paper correctly identifies convolution layers as the most computationally intensive part of CNNs, making them the primary target for fault tolerance.
  • Generality is Key: Unlike previous ABFT methods tied to specific matrix multiplication implementations, the proposed schemes are designed to work with any convolution implementation, making them broadly applicable across different CNN frameworks and hardware accelerators.
  • Layerwise Optimization is Crucial: The effectiveness of RC and CIC schemes varies significantly across different layers of a CNN. The multischeme workflow's ability to dynamically decide whether to enable RC/CIC on a per-layer basis is a critical insight for minimizing overhead.
  • Trade-offs are Inevitable: There's a clear trade-off between error correction capability and runtime overhead. The CoC scheme offers low overhead but limited correction, while FC provides high correction at a higher cost. The multischeme workflow effectively navigates this trade-off.

Key Results

  • Low Overhead: The FT-CNN framework achieves very low runtime overhead, typically 4% to 8% in both error-free and erroneous scenarios across various CNN models (AlexNet, VGG-19, ResNet-18, YOLOv2) when using MKL-DNN implementations.
  • Effective Error Correction: The multischeme workflow, particularly with layerwise RC/CIC optimization, significantly reduces the error correction overhead, sometimes by 40% to 60%.
  • MM-based ABFT Inefficiency: Traditional MM-based ABFT methods show prohibitively high overheads (up to 60%) when applied to CNN convolutions, highlighting the need for the proposed general ABFT schemes.
  • Scalability: The parallel performance evaluation demonstrates excellent scalability, with soft error protection overhead remaining low even when scaling to multiple nodes.

Limitations and Future Work

  • Fault Model: The analysis primarily considers at most one fault per convolution. While the authors mention that multiple faults can be handled by recomputing layers, the efficiency of this approach for very high fault rates isn't deeply explored.
  • Specific Hardware: While the schemes are architecture-independent, their performance might vary on specialized hardware accelerators.
  • Future Work: The authors plan to extend their implementation to more CNN frameworks and design architecture-specific optimizations for GPUs, FPGAs, and AI accelerators. They also aim to explore further optimizations for different hardware.

Summary updated

Abstract

Convolutional neural networks (CNNs) are becoming more and more important for solving challenging and critical problems in many fields. CNN inference applications have been deployed in safety-critical systems, which may suffer from soft errors caused by high-energy particles, high temperature, or abnormal voltage. Of critical importance is ensuring the stability of the CNN inference process against soft errors. Traditional fault tolerance methods are not suitable for CNN inference because error-correcting code is unable to protect computational components, instruction duplication techniques incur high overhead, and existing algorithm-based fault tolerance (ABFT) techniques cannot protect all convolution implementations. In this paper, we focus on how to protect the CNN inference process against soft errors as efficiently as possible, with the following three contributions. (1) We propose several systematic ABFT schemes based on checksum techniques and analyze their fault protection ability and runtime thoroughly.Unlike traditional ABFT based on matrix-matrix multiplication, our schemes support any convolution implementations. (2) We design a novel workflow integrating all the proposed schemes to obtain a high detection/correction ability with limited total runtime overhead. (3) We perform our evaluation using ImageNet with well-known CNN models including AlexNet, VGG-19, ResNet-18, and YOLOv2. Experimental results demonstrate that our implementation can handle soft errors with very limited runtime overhead (4%~8% in both error-free and error-injected situations).
BibTeX
@article{zhao2021ft,
  author = {Kai Zhao and Sheng Di and Sihuan Li and Xin Liang and Yujia Zhai and Jieyang Chen and Kaiming Ouyang and Franck Cappello and Zizhong Chen},
  title = {{FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks}},
  journal = {IEEE TPDS},
  year = {2021},
  doi = {10.1109/tpds.2020.3043449},
}

Loading related content...

Figure