BlockPerf: A Hybrid Blockchain Emulator/Simulator Framework

Julien Polge, Sankalp Ghatpande, Sylvain Kubler, Jérémy Robert, Yves Le TraonIEEE Access 2021 · OpenReview

Abstract

Blockchain is increasingly used for registering, authenticating and validating digital assets (financial assets, real estate, etc.) and transactions, governing interactions, recording data and managing identification among multiple parties in a trusted, decentralized, and secure manner. Today, a large variety of blockchain technologies is expanding in order to fulfill technical and non-technical needs and requirements. Within this context, determining and most importantly evaluating the characteristics/performance of a given blockchain platform is crucial for system designers before deploying it. A number of blockchain simulators have been proposed in the literature over the past few years, as reviewed in this paper, but are often limited in several respects (lack of extensibility, do not allow for evaluating all aspects of a blockchain ...). This paper extends and improves a state-of-the-art simulator (BlockSim) into a new simulator called “BlockPerf” to overcome those limitations. Both simulators are compared based on a real-life (benchmarking) Bitcoin scenario, whose results show that BlockPerf provides more realistic results than BlockSim, improving by around ≈ 50% (in average) the outcomes.

Introduction

Choosing the right blockchain platform is a high-stakes decision for system designers, yet evaluating how a specific platform will perform at scale is notoriously difficult. Researchers typically face a "Goldilocks" problem: full-scale testbeds using hundreds of real machines are prohibitively expensive and complex to manage, while pure simulators (such as BlockSim) are fast but often oversimplify reality. Specifically, pure simulators frequently assume a perfectly connected network, ignoring the latency, packet loss, and dynamic topology changes that occur in real-world deployments. This gap leads to non-optimal performance results that fail to meet requirements once the system is live.

Mapping of existing standards to the six-layer blockchain model.

The BlockPerf Framework

To bridge the gap between simulation and reality, the authors introduce BlockPerf, which extends the existing BlockSim tool into a hybrid architecture. The framework adopts a six-layer abstraction model to categorize blockchain functions: Application, Contract, Incentive, Consensus, Node/Data, and Network.

The six-layer blockchain abstraction model and associated performance metrics.

Hybrid Methodology

The "hybrid" nature of BlockPerf comes from how it handles these layers:

  • Simulation (Upper Layers): The Application, Incentive, Consensus, and Node/Data layers are simulated using statistical models and Python code. This allows for rapid testing of different reward structures or consensus rules without the overhead of a full deployment.
  • Emulation (Network Layer): Unlike traditional simulators, BlockPerf uses real physical/distributed nodes to handle the Network layer. It implements actual P2P protocols (like GETADDR and ADDR messages) and manages real-world wallet addresses.

By running upper-layer logic on top of a real-world distributed network, BlockPerf captures authentic communication delays and network congestion that pure mathematical models miss.

Experimental Validation

To validate BlockPerf, the authors built a real-life Bitcoin benchmarking testbed using bitcoind. The setup included 63 nodes (23 full nodes and 40 light nodes) distributed across three geographic locations: Ireland, Luxembourg, and India. This private network was operated for 11 days, generating random transactions to collect ground-truth data on throughput, block validation times, and fork occurrences. This real-world data was then compared against results generated by both BlockSim and BlockPerf.

Key Results and Insights

BlockPerf demonstrated a massive leap in accuracy over previous state-of-the-art tools:

  • Overall Realism: BlockPerf provided results that were, on average, 50% more realistic than BlockSim when compared to the 11-day Bitcoin testbed.
  • Layer-Specific Gains: Accuracy at the Node/Data layer improved by 39%, while the Network layer saw a 55% improvement. This confirms that ignoring network-level changes is the primary source of error in traditional simulators.
  • Predicting Forks: BlockPerf was significantly better at predicting "stale blocks" and chain forks. Because it uses a real network, it naturally experiences the propagation delays that cause two miners to find a block simultaneously.
  • Throughput and Speed: While BlockSim tended to overestimate transaction capacity, BlockPerf’s results closely tracked the 1.5 - 2.5 Tx/s observed in the real testbed. Furthermore, BlockPerf ran 33 times faster than real-time, offering a strong middle ground between the speed of pure simulation and the accuracy of full emulation.

Conclusion

BlockPerf addresses the critical need for realistic blockchain performance evaluation. By combining high-level simulation with low-level network emulation, it provides system designers with a tool that is both practical and accurate. Future work will focus on implementing the Contract Layer (Smart Contracts) and adapting the framework to non-linear architectures, such as DAG-based chains.

Summary updated

Abstract

Blockchain is increasingly used for registering, authenticating and validating digital assets (financial assets, real estate, etc.) and transactions, governing interactions, recording data and managing identification among multiple parties in a trusted, decentralized, and secure manner. Today, a large variety of blockchain technologies is expanding in order to fulfill technical and non-technical needs and requirements. Within this context, determining and most importantly evaluating the characteristics/performance of a given blockchain platform is crucial for system designers before deploying it. A number of blockchain simulators have been proposed in the literature over the past few years, as reviewed in this paper, but are often limited in several respects (lack of extensibility, do not allow for evaluating all aspects of a blockchain ...). This paper extends and improves a state-of-the-art simulator (BlockSim) into a new simulator called “BlockPerf” to overcome those limitations. Both simulators are compared based on a real-life (benchmarking) Bitcoin scenario, whose results show that BlockPerf provides more realistic results than BlockSim, improving by around ≈ 50% (in average) the outcomes.
BibTeX
@inproceedings{polge2021blockperf,
  author = {Julien Polge and Sankalp Ghatpande and Sylvain Kubler and Jérémy Robert and Yves Le Traon},
  title = {{BlockPerf: A Hybrid Blockchain Emulator/Simulator Framework}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=B-VVaMphXx9},
}
Figure