TL;DR
Standard performance metrics like Average Precision (AP) fail to capture the true safety risks posed by hardware faults in object detection systems. This paper introduces IVMOD, a new image-level metric that reveals how a single bit flip in a neural network's memory can lead to catastrophic "ghost" objects or complete blindness in autonomous systems.
What Problem Does This Paper Address?
Deep Neural Networks (DNNs) are the eyes of autonomous vehicles and robots. While researchers have spent years hardening these models against blurry images or adversarial attacks, they often overlook a threat from within: hardware faults.
Natural phenomena like cosmic radiation or electrical leakage can cause "bit flips" (changing a 0 to a 1) in a system's memory. In a safety-critical environment, we need to know if these faults cause Silent Data Corruption (SDC)—where the model produces a dangerously wrong output without the system realizing anything is broken.
The authors argue that standard metrics like Average Precision (AP) are inadequate for safety. AP is an object-level average; it might not change much if a fault causes 100 "ghost" cars to appear in a single frame, yet that single frame could cause an autonomous vehicle to slam on its brakes unnecessarily.
Methods
The researchers performed extensive fault injection campaigns on three popular object detection architectures: YOLOv3, RetinaNet, and Faster R-CNN. They simulated both transient faults (random bit flips during a single inference) and permanent faults (bits "stuck" at 0 or 1).
To better estimate safety impact, they proposed IVMOD (Image-wise Vulnerability Metric for Object Detection). Unlike AP, IVMOD:
- Evaluates at the image level: It counts how many images suffer from corrupted detections.
- Uses a Hungarian association algorithm: This provides a more robust way to match predicted boxes to ground truth, independent of the model's confidence scores.
- Analyzes Severity: It measures the "area occupancy" of errors—how much of the drivable space is covered by false detections ($A_{FPblob}$) or how much of a real object is "erased" ($A_{FNblob}$).
Datasets
The study utilized three major benchmarks:
- COCO2017: A standard for general object detection.
- KITTI: A popular dataset for autonomous driving.
- Lyft Level 5 Perception Dataset: Crucial for this study because it provides video sequences, allowing the authors to track how permanent hardware faults persist over time across multiple frames.
Key Insights and Ideas
- The AP Illusion: AP can be misleading. In some tests, a model's AP dropped significantly due to faults in only 3.2% of images. Conversely, for Faster R-CNN, AP barely moved even when the model started missing a third of all objects.
- Exponent Bits are Critical: In FP32 (single-precision floating point), flipping a bit in the exponent has a much more devastating effect than flipping a bit in the mantissa. These flips can scale a weight or activation to astronomical values, effectively "breaking" the layer's logic.
- Architecture Matters: YOLOv3 and RetinaNet are prone to generating massive amounts of high-confidence "ghost" objects. Faster R-CNN is more robust against ghost objects but is highly susceptible to "blindness," where it simply stops seeing real objects.
- Temporal Persistence: Permanent faults don't just affect one frame; they create "ghost" blobs that follow the camera, potentially confusing downstream tracking modules.
Key Results
- Catastrophic Failures: A single bit flip in YOLOv3 can generate over 100 false positives in one image or cause the model to lose 90% of its true detections.
- Area Occupancy: Permanent stuck-at-1 faults in weights were found to be particularly severe, creating ghost detections that cover up to 83% of the image area.
- Vulnerability Rates: Depending on the model, the probability of a single bit flip causing a safety-relevant SDC event ranges from 0.4% to 4.2%. Weight faults were generally twice as likely to cause issues as neuron (activation) faults.
Limitations and Future Work
The authors focused their fault injections specifically on convolutional layers, noting that fully connected layers showed less impact in their preliminary tests. They also assumed that faults would successfully propagate to the application layer.
A promising direction for future work is the development of tracking-based defenses. Since the paper shows that fault-induced errors often have distinct temporal signatures (like "ghosts" that don't move naturally), a smart tracking module could potentially identify and filter out these hardware-induced hallucinations before they reach the vehicle's decision-making engine.
Summary updated