Abstract
Reliability assessment of Deep Neural Networks (DNNs) in safety-critical applications requires scalable fault injection tools. We present enpheeph, an open-source framework specifically designed for Spiking Neural Networks (SNNs) and compressed (quantized/pruned) DNNs. enpheeph achieves high scalability, simulating hundreds of thousands of concurrent faults with less than 20% runtime overhead—a 10x improvement over existing tools—making it a vital resource for certifying AI in safety-critical systems.
Introduction
As Deep Neural Networks move into safety-critical domains like autonomous driving, their reliability becomes a life-or-death issue. Standards like ISO 26262 demand extremely low failure rates. However, as hardware technology scales down, the risk of "multiple upset events"—where many faults occur simultaneously—increases. Existing fault injection tools struggle with scalability, as their runtime overhead often grows exponentially with the number of concurrent faults. Furthermore, modern efficiency techniques like quantization, pruning, and SNNs are not well-supported by traditional frameworks.
Methodology
The enpheeph framework is built to be hardware-agnostic and highly customizable. Unlike previous tools that might require heavy modifications to a model's underlying code, enpheeph uses a modular API to instrument the network dynamically.
Key technical highlights include:
- Execution Hooks: Instead of manually backing up and restoring massive tensors, enpheeph inserts "hooks" into the model's execution flow to apply fault masks on the fly.
- Optimized Masking: Fault masks are expanded to match target tensor shapes efficiently, allowing for massive parallel fault injection on GPUs.
- Device Autonomy: The framework automatically detects the hardware (CPU vs. GPU) and employs optimized back-end libraries to minimize latency.
Experimental Evaluation
The framework was evaluated using standard benchmarks to ensure broad relevance:
- CIFAR10: Used for training and testing standard DNN architectures like VGG11 and ResNet18.
- DVS Gesture Dataset: A specialized dataset for event-based vision, used to evaluate the resiliency of Spiking Neural Networks.
Key Insights and Results
Scalability and Performance
enpheeph exhibits less than 20% runtime overhead when injecting 100,000 concurrent faults. This is at least 10x lower than previous state-of-the-art frameworks. While tools like TensorFI2 show exponential increases in runtime as the number of faults grows, enpheeph maintains a nearly linear trend.
SNN Resiliency
One of the most striking findings is that SNNs are significantly more resilient than standard DNNs. Because SNNs process information as a stream of events over time, they possess a form of "temporal redundancy" that allows them to maintain accuracy even when many internal signals are corrupted. Significant accuracy drops only appeared at fault rates orders of magnitude higher than those that crippled standard DNNs.
Vulnerability in Compressed Networks
In quantized networks, the impact of a fault is highly dependent on bit position. Flipping a Most Significant Bit (MSB) causes massive accuracy drops, while flips in Least Significant Bits (LSBs) are often naturally attenuated. Additionally, pruning (removing synapses or neurons) changes the "fault surface" of a network, requiring specialized tools like enpheeph to accurately map real-world behavior.
Conclusion
enpheeph provides a scalable solution for evaluating the reliability of modern, optimized neural architectures. By supporting SNNs and compressed models with minimal overhead, it enables the rigorous testing required for safety-critical AI. Future work aims to extend support for sparse tensors and optimize execution for specialized hardware like TPUs.
Summary updated