Introduction
Blockchains are frequently marketed as the ultimate fault-tolerant systems, designed to operate across a global, decentralized network to eliminate single points of failure. However, high-profile outages in major networks suggest a significant gap between theoretical resilience and real-world performance. While existing benchmarks focus on "happy path" metrics like peak throughput and latency under ideal conditions, they often fail to capture how these systems behave when nodes crash or network partitions occur.
To address this, researchers have introduced STABL, a benchmarking framework and a novel "sensitivity" metric designed to quantify blockchain dependability. By evaluating five major protocols—Algorand, Aptos, Avalanche, Redbelly, and Solana—the study reveals that many modern blockchains struggle to recover from transient failures, with some suffering total liveness failures.
Quantifying Dependability: The Sensitivity Metric
The core contribution of this work is the Sensitivity Metric, which provides a formal way to measure how much performance degrades in response to environmental variations. Inspired by economic distribution functions, the metric calculates the difference between two Cumulative Distribution Functions (CDFs) of transaction latencies: the "baseline" (normal operation) and the "altered" environment (under failure).
The "sensitivity score" is represented by the area between these two curves. A low score indicates high resilience, while a high score suggests fragility. An infinite score denotes a "liveness" failure, where the system stops processing transactions entirely and fails to recover within the test window.
Experimental Methodology
The researchers developed the STABL (Sensitivity Testing and Analysis for BLockchain) tool, built upon the DIABLO benchmark. STABL uses "observer nodes" to orchestrate specific failure patterns across a distributed system of 15 nodes (10 blockchain nodes and 5 clients). The protocols tested include Algorand (v3.22.0), Aptos (v1.9.3), Avalanche (C-Chain v1.10.18), Redbelly (v0.36.2), and Solana (v1.18.1).
The framework evaluates four primary failure modes:
- Crashes: Permanent node deactivation.
- Transient Failures: Temporary node deactivation followed by a restart.
- Partitions: Simulated network splits using
netfilter. - Byzantine Behavior: Testing system response to redundant requests required for secure client implementations.
Comparative Analysis of Results
The experimental results highlight stark differences in how modern blockchains handle adversity. As shown in the sensitivity comparisons below, while most systems handle simple crashes reasonably well, transient failures and network partitions prove far more disruptive.
Key Findings
- The Liveness Trap: Both Solana and Avalanche experienced total liveness failures (infinite sensitivity) during transient failure and partition tests. In Solana, a bug in the "Epoch Accounts Hash" calculation caused nodes to panic when attempting to catch up. In Avalanche, aggressive throttling mechanisms intended for protection actually prevented nodes from reaching consensus once they fell behind.
- Transient vs. Permanent Failures: Surprisingly, many blockchains handle a permanent node death better than a node that leaves and attempts to rejoin. Algorand and Redbelly demonstrated the best recoverability, while Aptos recovered but suffered a permanent performance drop, failing to clear its transaction backlog.
- Leaderless Resilience: Redbelly, which utilizes a leaderless consensus algorithm, proved to be the most resilient. Unlike leader-based systems (e.g., Solana), Redbelly avoids the "snowball effect" of performance degradation when a specific proposer node goes offline.
- Redundancy as an Optimizer: In Byzantine settings, where clients send requests to multiple nodes for security, Redbelly and Avalanche actually saw performance improvements. This redundancy allowed clients to bypass slow or throttled nodes effectively.
Conclusion and Future Work
The STABL framework reveals that current blockchain marketing often overstates real-world resilience. While leaderless protocols like Redbelly show high dependability, popular leader-based systems remain vulnerable to localized failures that can trigger network-wide outages.
Future research will focus on scaling these tests to hundreds of nodes to better simulate global network topologies and evaluating how complex smart contract executions affect recovery dynamics. The authors hope the community will adopt STABL to provide more transparent, standardized dependability data for emerging blockchain protocols.
Summary updated