Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs

Syed Qutub, Florian Geissler, Yang Peng, Ralf Grafe, Michael Paulitsch, Gereon Hinz, Alois Knoll

TL;DR

Standard performance metrics like Average Precision (AP) fail to capture the true safety risks posed by hardware faults in object detection systems. This paper introduces IVMOD, a new image-level metric that reveals how a single bit flip in a neural network's memory can lead to catastrophic "ghost" objects or complete blindness in autonomous systems.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are the eyes of autonomous vehicles and robots. While researchers have spent years hardening these models against blurry images or adversarial attacks, they often overlook a threat from within: hardware faults.

Natural phenomena like cosmic radiation or electrical leakage can cause "bit flips" (changing a 0 to a 1) in a system's memory. In a safety-critical environment, we need to know if these faults cause Silent Data Corruption (SDC)—where the model produces a dangerously wrong output without the system realizing anything is broken.

The authors argue that standard metrics like Average Precision (AP) are inadequate for safety. AP is an object-level average; it might not change much if a fault causes 100 "ghost" cars to appear in a single frame, yet that single frame could cause an autonomous vehicle to slam on its brakes unnecessarily.

Methods

The researchers performed extensive fault injection campaigns on three popular object detection architectures: YOLOv3, RetinaNet, and Faster R-CNN. They simulated both transient faults (random bit flips during a single inference) and permanent faults (bits "stuck" at 0 or 1).

To better estimate safety impact, they proposed IVMOD (Image-wise Vulnerability Metric for Object Detection). Unlike AP, IVMOD:

  1. Evaluates at the image level: It counts how many images suffer from corrupted detections.
  2. Uses a Hungarian association algorithm: This provides a more robust way to match predicted boxes to ground truth, independent of the model's confidence scores.
  3. Analyzes Severity: It measures the "area occupancy" of errors—how much of the drivable space is covered by false detections ($A_{FPblob}$) or how much of a real object is "erased" ($A_{FNblob}$).
Figure 1: Examples of the impact of a single neuron bit flip (at bit position b and layer index l, see image insets). TPs are marked by green, FPs by red and FNs by blue rectangles, comparing the fault-free (top) and the faulty (bottom) predictions. In example (a) multiple FPs are generated right in front of the ego vehicle, while in (b) all previous detections are erased due to the fault.

Datasets

The study utilized three major benchmarks:

  • COCO2017: A standard for general object detection.
  • KITTI: A popular dataset for autonomous driving.
  • Lyft Level 5 Perception Dataset: Crucial for this study because it provides video sequences, allowing the authors to track how permanent hardware faults persist over time across multiple frames.

Key Insights and Ideas

  • The AP Illusion: AP can be misleading. In some tests, a model's AP dropped significantly due to faults in only 3.2% of images. Conversely, for Faster R-CNN, AP barely moved even when the model started missing a third of all objects.
  • Exponent Bits are Critical: In FP32 (single-precision floating point), flipping a bit in the exponent has a much more devastating effect than flipping a bit in the mantissa. These flips can scale a weight or activation to astronomical values, effectively "breaking" the layer's logic.
  • Architecture Matters: YOLOv3 and RetinaNet are prone to generating massive amounts of high-confidence "ghost" objects. Faster R-CNN is more robust against ghost objects but is highly susceptible to "blindness," where it simply stops seeing real objects.
  • Temporal Persistence: Permanent faults don't just affect one frame; they create "ghost" blobs that follow the camera, potentially confusing downstream tracking modules.
Tracking how a permanent fault creates a persistent "ghost" blob across sequential video frames.

Key Results

  • Catastrophic Failures: A single bit flip in YOLOv3 can generate over 100 false positives in one image or cause the model to lose 90% of its true detections.
  • Area Occupancy: Permanent stuck-at-1 faults in weights were found to be particularly severe, creating ghost detections that cover up to 83% of the image area.
  • Vulnerability Rates: Depending on the model, the probability of a single bit flip causing a safety-relevant SDC event ranges from 0.4% to 4.2%. Weight faults were generally twice as likely to cause issues as neuron (activation) faults.

Limitations and Future Work

The authors focused their fault injections specifically on convolutional layers, noting that fully connected layers showed less impact in their preliminary tests. They also assumed that faults would successfully propagate to the application layer.

A promising direction for future work is the development of tracking-based defenses. Since the paper shows that fault-induced errors often have distinct temporal signatures (like "ghosts" that don't move naturally), a smart tracking module could potentially identify and filter out these hardware-induced hallucinations before they reach the vehicle's decision-making engine.

Summary updated

Abstract

Object detection neural network models need to perform reliably in highly dynamic and safety-critical environments like automated driving or robotics. Therefore, it is paramount to verify the robustness of the detection under unexpected hardware faults like soft errors that can impact a systems perception module. Standard metrics based on average precision produce model vulnerability estimates at the object level rather than at an image level. As we show in this paper, this does not provide an intuitive or representative indicator of the safety-related impact of silent data corruption caused by bit flips in the underlying memory but can lead to an over- or underestimation of typical fault-induced hazards. With an eye towards safety-related real-time applications, we propose a new metric IVMOD (Image-wise Vulnerability Metric for Object Detection) to quantify vulnerability based on an incorrect image-wise object detection due to false positive (FPs) or false negative (FNs) objects, combined with a severity analysis. The evaluation of several representative object detection models shows that even a single bit flip can lead to a severe silent data corruption event with potentially critical safety implications, with e.g., up to (much greater than) 100 FPs generated, or up to approx. 90% of true positives (TPs) are lost in an image. Furthermore, with a single stuck-at-1 fault, an entire sequence of images can be affected, causing temporally persistent ghost detections that can be mistaken for actual objects (covering up to approx. 83% of the image). Furthermore, actual objects in the scene are continuously missed (up to approx. 64% of TPs are lost). Our work establishes a detailed understanding of the safety-related vulnerability of such critical workloads against hardware faults.
BibTeX
@inproceedings{qutub2022hardware,
  author = {Syed Qutub and Florian Geissler and Yang Peng and Ralf Grafe and Michael Paulitsch and Gereon Hinz and Alois Knoll},
  title = {{Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs}},
  booktitle = {SAFECOMP},
  year = {2022},
}
Figure