Abstract
The AEDAM project aims to transform software transient-error detection through configurable specialized detectors and quantitative characterization of hardware resilience. UT Austin's contributions focus on the Hamartia open-source error injection framework and the Nested Monte Carlo (NMC) methodology. These tools enable the study of detector composition, allowing researchers to efficiently protect whole programs against transient hardware faults.
Introduction: The Reliability Challenge at Exascale
As computing moves toward the exascale, "soft errors"—transient hardware faults such as bit-flips caused by cosmic rays or voltage fluctuations—pose a significant threat to system reliability. To mitigate these risks, researchers employ error injection to evaluate how systems fail and how various detectors (e.g., software assertions or hardware residue checkers) perform.
However, current tools face a critical "Fidelity vs. Speed" trade-off. Simple bit-flip models are fast but inaccurate, while high-fidelity Register-Transfer Level (RTL) simulations are often too slow for large-scale applications. Furthermore, the "Masking Problem" in traditional Monte Carlo simulations leads to thousands of wasted runs where injected errors have no impact on the final output.
Methodology: High-Fidelity Injection and Efficient Sampling
The Hamartia Framework
The UT Austin team developed Hamartia, a fully open-source suite that integrates high-fidelity hardware error models with Intel’s Pin dynamic binary instrumentation platform. Hamartia utilizes on-demand, just-in-time logic simulation of arithmetic unit RTL models. This approach allows the framework to simulate how a specific circuit fails based on real-time input values, bridging the gap between speed and fidelity.
Nested Monte Carlo (NMC)
To address the masking problem, the researchers introduced the Nested Monte Carlo (NMC) methodology. Instead of executing a full application for every injection, NMC utilizes "micro-rollbacks." If an error is immediately masked or caught by a local detector, the framework re-executes only the affected instruction or function until an unmasked, undetected error is produced.
Key Insights and Results
Performance and Scalability
Hamartia demonstrates superior performance compared to PinFi, previously the fastest injection framework, while providing significantly higher modeling fidelity. A key advantage is its fixed overhead; regardless of application duration, Hamartia maintains a consistent slowdown (approximately 4x), ensuring scalability for long-running HPC applications.
The Necessity of High Fidelity
A major finding of the project is that while low-fidelity bit-flip models may suffice for general sensitivity analysis, they are inadequate for evaluating specific error detectors. Because detectors are often tailored to specific circuits and data values, high-fidelity RTL models are essential to accurately measure their effectiveness.
Synergy in Detector Composition
The study highlights that software and hardware detectors "compose" effectively. By combining simple software assertions with basic hardware detectors, researchers achieved protection levels equivalent to much more complex and computationally expensive standalone hardware solutions.
As shown in the CLAMR benchmarks, composing detectors (e.g., RTL+) significantly reduces the ratio of Silent Data Corruption (SDC) compared to baseline configurations, demonstrating the power of multi-layered defense.
Conclusion and Future Work
The AEDAM project has successfully delivered tools that reduce the simulation time required for error injection campaigns by 10x to 100x. Future work will focus on expanding support for floating-point hardware detectors and conducting broader composition studies in increasingly complex system scenarios.
Summary updated