Just Accepted
Exploring Scalability of BFT Blockchain Protocols through Network Simulations
Abstract
Novel Byzantine fault-tolerant (BFT) state machine replication protocols improve scalability for their practical use in distributed ledger technology, where hundreds of replicas must reach consensus. Evaluating the performance of BFT protocol implementations requires careful evaluation. We propose a new methodology using scalable network simulations to predict BFT protocol performance. Our simulation architecture allows for the integration of existing BFT implementations without modification or re-implementation, offering a cost-effective alternative to large-scale cloud experiments. We validate our method by comparing simulation results with real-world cloud deployments, showing that simulations can accurately predict performance at larger scales when network limitations dominate.
In our study, we applied this methodology to assess the performance of several “blockchain-generation” BFT protocols, including HotStuff, Kauri, Narwhal & Tusk, and Bullshark, under realistic network conditions (with constrained 25 Mbit/s bandwidth) and induced faults. Kauri emerges as the top performer, achieving 6742 operations per second (op/s) with 128 replicas, outperforming BullShark (2318 op/s) and Tusk (1952 op/s). HotStuff, using secp256k1 and BLS signatures, reaches 494 op/s and 707 op/s, respectively, demonstrating the efficiency of BLS-signature aggregation for saving bandwidth. This study demonstrates that state-of-the-art asynchronous BFT protocols can achieve competitive throughput in large-scale, real-world scenarios.
References
[1]
Mohammad Javad Amiri, Chenyuan Wu, Divyakant Agrawal, Amr El Abbadi, Boon Thau Loo, and Mohammad Sadoghi. 2024. The Bedrock of Byzantine Fault Tolerance: A Unified Platform for {BFT} Protocols Analysis, Implementation, and Experimentation. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). 371–400.
[2]
Yusuke Aoki, Kai Otsuki, Takeshi Kaneko, Ryohei Banno, and Kazuyuki Shudo. 2019. Simblock: A blockchain network simulator. In IEEE Conf. on Computer Communications Workshops. IEEE Comp. Soc., Washington, DC, USA, 325–329.
[3]
Balaji Arun and Binoy Ravindran. 2020. DuoBFT: Resilience vs. Efficiency Trade-off in Byzantine Fault Tolerance. preprint arXiv:2010.01387(2020).
[4]
Hagit Attiya, Constantin Enea, and Shafik Nassar. 2023. Faithful simulation of randomized bft protocols on block dags. Cryptology ePrint Archive(2023).
[5]
Shehar Bano, Alberto Sonnino, Andrey Chursin, Dmitri Perelman, Zekun Li, Avery Ching, and Dahlia Malkhi. 2022. Twins: BFT Systems Made Robust. In 25th Int. Conf. on Principles of Distributed Systems, Vol. 217. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl, Germany, 7:1–7:29.
[6]
Christian Berger, Hans P. Reiser, and Alysson Bessani. 2021. Making Reads in BFT State Machine Replication Fast, Linearizable, and Live. In 40th Int. Symp. on Reliable Distributed Systems. IEEE Comp. Soc., Washington, DC, USA, 1–12.
[7]
Christian Berger, Hans P. Reiser, João Sousa, and Alysson Neves Bessani. 2022. AWARE: Adaptive wide-area replication for fast and resilient Byzantine consensus. IEEE Transactions on Dependable and Secure Computing 19, 3 (2022), 1605–1620.
[8]
Christian Berger, Signe Schwarz-Rüsch, Arne Vogel, Kai Bleeke, Leander Jehl, Hans P. Reiser, and Rüdiger Kapitza. 2023. SoK: Scalability Techniques for BFT Consensus. In IEEE International Conference on Blockchain and Cryptocurrency (ICBC). IEEE.
[9]
Christian Berger, Sadok Ben Toumia, and Hans P Reiser. 2022. Does my bft protocol implementation scale?. In Proceedings of the 3rd International Workshop on Distributed Infrastructure for the Common Good. 19–24.
[10]
Christian Berger, Sadok Ben Toumia, and Hans P Reiser. 2023. Scalable Performance Evaluation of Byzantine Fault-Tolerant Systems Using Network Simulation. In 2023 IEEE 28th Pacific Rim International Symposium on Dependable Computing (PRDC). IEEE, 180–190.
[11]
Alysson Bessani, João Sousa, and Eduardo EP Alchieri. 2014. State machine replication for the masses with BFT-SMaRt. In 44th Annu. IEEE/IFIP Int. Conf. on Dependable Systems and Networks (DSN). IEEE Comp. Soc., Washington, DC, USA, 355–362.
[12]
Dan Boneh, Ben Lynn, and Hovav Shacham. 2004. Short signatures from the Weil pairing. Journal of cryptology 17(2004), 297–319.
[13]
Daniel Cason, Enrique Fynn, Nenad Milosevic, Zarko Milosevic, Ethan Buchman, and Fernando Pedone. 2021. The design, architecture and performance of the Tendermint Blockchain Network. In 40th Int. Symp. on Reliable Distributed Systems. IEEE Comp. Soc., Washington, DC, USA, 23–33.
[14]
Miguel Castro and Barbara Liskov. 1999. Practical Byzantine Fault Tolerance. In OSDI. USENIX Association, Berkeley, CA, USA, 173–186.
[15]
Allen Clement, Edmund Wong, Lorenzo Alvisi, Mike Dahlin, Mirco Marchetti, et al. 2009. Making Byzantine fault tolerant systems tolerate Byzantine faults. In Proceedings of the 6th USENIX symposium on Networked systems design and implementation. The USENIX Association.
[16]
Tyler Crain, Christopher Natoli, and Vincent Gramoli. 2021. Red belly: A secure, fair and scalable open blockchain. In IEEE Symp. on Security and Privacy. IEEE Comp. Soc., Washington, DC, USA, 466–483.
[17]
George Danezis, Lefteris Kokoris-Kogias, Alberto Sonnino, and Alexander Spiegelman. 2022. Narwhal and tusk: a dag-based mempool and efficient bft consensus. In Proceedings of the Seventeenth European Conference on Computer Systems. 34–50.
[18]
Jérémie Decouchant, David Kozhaya, Vincent Rahli, and Jiangshan Yu. 2022. DAMYSUS: streamlined BFT consensus leveraging trusted components(EuroSys ’22). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3492321.3519568
[19]
Jérémie Decouchant, Burcu Kulahcioglu Ozkan, and Yanzhuo Zhou. 2023. Liveness Checking of the HotStuff Protocol Family. In The 28th Pacific Rim International Symposium on Dependable Computing (PRDC). IEEE, 168–179. https://doi.org/10.1109/PRDC59308.2023.00029
[20]
Tobias Distler. 2021. Byzantine fault-tolerant state-machine replication from a systems perspective. ACM Computing Surveys (CSUR) 54, 1 (2021), 1–38.
[21]
Cynthia Dwork, Nancy Lynch, and Larry Stockmeyer. 1988. Consensus in the presence of partial synchrony. Journal of the ACM (JACM) 35, 2 (1988), 288–323.
[22]
Carlos Faria and Miguel Correia. 2019. BlockSim: blockchain simulator. In IEEE Int. Conf. on Blockchain. IEEE Comp. Soc., Washington, DC, USA, 439–446.
[23]
Rati Gelashvili, Lefteris Kokoris-Kogias, Alberto Sonnino, Alexander Spiegelman, and Zhuolun Xiang. 2022. Jolteon and ditto: Network-adaptive efficient consensus with asynchronous fallback. In International Conference on Financial Cryptography and Data Security. Springer, 296–315.
[24]
Arthur Gervais, Ghassan O Karame, Karl Wüst, Vasileios Glykantzis, Hubert Ritzdorf, and Srdjan Capkun. 2016. On the security and performance of proof of work blockchains. In ACM SIGSAC CCS. ACM, New York, NY, 3–16.
[25]
Paulo Gouveia, João Neves, Carlos Segarra, Luca Liechti, Shady Issa, Valerio Schiavoni, and Miguel Matos. 2020. Kollaps: decentralized and dynamic topology emulation. In 15th European Conf. on Computer Systems. ACM, New York, NY, 1–16.
[26]
Vincent Gramoli, Rachid Guerraoui, Andrei Lebedev, Chris Natoli, and Gauthier Voron. 2023. Diablo: A benchmark suite for blockchains. In EuroSys’23. 540–556.
[27]
Divya Gupta, Lucas Perronne, and Sara Bouchenak. 2016. BFT-Bench: Towards a practical evaluation of robustness and effectiveness of BFT protocols. In Distributed Applications and Interoperable Systems: 16th IFIP WG 6.1 International Conference, DAIS 2016, Held as Part of the 11th International Federated Conference on Distributed Computing Techniques, DisCoTec 2016, Heraklion, Crete, Greece, June 6-9, 2016, Proceedings 16. Springer, 115–128.
[28]
Suyash Gupta, Sajjad Rahnama, Jelle Hellings, and Mohammad Sadoghi. 2020. Resilientdb: Global scale resilient blockchain fabric. arXiv preprint arXiv:2002.00160(2020).
[29]
Raluca Halalai, Thomas A Henzinger, and Vasu Singh. 2011. Quantitative evaluation of BFT protocols. In 8th Int. Conf. on Quantitative Evaluation of Systems. IEEE Comp. Soc., Washington, DC, USA, 255–264.
[30]
Nikhil Handigol, Brandon Heller, Vimalkumar Jeyakumar, Bob Lantz, and Nick McKeown. 2012. Reproducible network experiments using container-based emulation. In 8th Int. Conf. on Emerging networking experiments and technologies. ACM, New York, NY, 253–264.
[31]
Yahya Hassanzadeh-Nazarabadi, Misha Rybalov, and Khalil Claybon. 2023. BFT Testing Framework for Flow Blockchain. In International Congress on Blockchain and Applications. Springer, 338–347.
[32]
Rob Jansen, Jim Newsome, and Ryan Wails. 2022. Co-opting Linux Processes for High-Performance Network Simulation. In USENIX ATC 22. USENIX Association, Berkeley, CA, USA, 327–350.
[33]
Ramakrishna Kotla, Lorenzo Alvisi, Mike Dahlin, Allen Clement, and Edmund Wong. 2007. Zyzzyva: speculative Byzantine fault tolerance. In the 21st ACM SIGOPS SOSP. ACM, New York, NY, 45–58.
[34]
Leslie Lamport. 1998. The Part-Time Parliament. ACM Transactions on Computer Systems 16, 2 (1998), 133–169.
[35]
Bob Lantz, Brandon Heller, and Nick McKeown. 2010. A network in a laptop: rapid prototyping for software-defined networks. In 9th ACM SIGCOMM Workshop on Hot Topics in Networks. ACM, New York, NY, 1–6.
[36]
Peilun Li, Guosai Wang, Xiaoqi Chen, Fan Long, and Wei Xu. 2020. Gosig: a scalable and high-performance Byzantine consensus for consortium blockchains. In 11th ACM Symp. on Cloud Computing. ACM, New York, NY, 223–237.
[37]
Boon Thau Loo, Tyson Condie, Joseph M Hellerstein, Petros Maniatis, Timothy Roscoe, and Ion Stoica. 2005. Implementing declarative overlays. In 12th ACM SOSP. ACM, New York, NY, 75–90.
[38]
Andrew Miller and Rob Jansen. 2015. Shadow-Bitcoin: Scalable Simulation via Direct Execution of Multi-Threaded Applications. In 8th Workshop on Cyber Security Experimentation and Test. USENIX Association, Berkeley, CA, USA.
[39]
Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. https://bitcoin.org/bitcoin.pdf
[40]
Ray Neiheiser, Miguel Matos, and Luís Rodrigues. 2021. Kauri: Scalable BFT consensus with pipelined tree-based dissemination and aggregation. In ACM SIGOPS 28th SOSP. ACM, New York, NY, 35–48.
[41]
Martin Nischwitz, Marko Esche, and Florian Tschorsch. 2021. Bernoulli meets pbft: Modeling BFT protocols in the presence of dynamic failures. In 16th Conference on Computer Science and Intelligence Systems. IEEE Comp. Soc., Washington, DC, USA, 291–300.
[42]
Diego Ongaro and John Ousterhout. 2014. In search of an understandable consensus algorithm. In 2014 USENIX annual technical conference (USENIX ATC 14). 305–319.
[43]
George F Riley and Thomas R Henderson. 2010. The ns-3 network simulator. In Modeling and tools for network simulation. Springer, Cham, 15–34.
[44]
Team Rocket, Maofan Yin, Kevin Sekniqi, Robbert van Renesse, and Emin Gün Sirer. 2019. Scalable and probabilistic leaderless BFT consensus through metastability. arXiv preprint arXiv:1906.08936(2019).
[45]
Signe Rüsch, Kai Bleeke, and Rüdiger Kapitza. 2019. Themis: An Efficient and Memory-Safe BFT Framework in Rust: Research Statement. In 3rd Workshop on Scalable and Resilient Infrastructures for Distributed Ledgers. ACM, New York, NY, 9–10.
[46]
Fred B Schneider. 1990. Implementing fault-tolerant services using the state machine approach: A tutorial. ACM Computing Surveys (CSUR) 22, 4 (1990), 299–319.
[47]
Fábio Silva, Ana Alonso, José Pereira, and Rui Oliveira. 2020. A comparison of message exchange patterns in BFT protocols. In IFIP Int. Conf. on Distributed Applications and Interoperable Systems. Springer, 104–120.
[48]
Atul Singh, Tathagata Das, Petros Maniatis, Peter Druschel, and Timothy Roscoe. 2008. BFT Protocols Under Fire. In NSDI, Vol. 8. USENIX Association, Berkeley, CA, USA, 189–204.
[49]
João Sousa and Alysson Bessani. 2015. Separating the WHEAT from the Chaff: An Empirical Design for Geo-Replicated State Machines. In 34th IEEE Symp. on Reliable Distributed Systems. IEEE Comp. Soc., Washington, DC, USA, 146–155.
[50]
Alexander Spiegelman, Neil Giridharan, Alberto Sonnino, and Lefteris Kokoris-Kogias. 2022. Bullshark: Dag bft protocols made practical. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2705–2718.
[51]
Chrysoula Stathakopoulou, Tudor David, and Marko Vukolić. 2019. Mir-bft: High-throughput BFT for blockchains. arXiv preprint arXiv:1906.05552(2019).
[52]
Xiao Sui, Sisi Duan, and Haibin Zhang. 2022. Marlin: Two-Phase BFT with Linearity. In Proc. of the 52nd Annual IEEE/IFIP Int. Conf. on Dependable Systems and Networks (DSN). IEEE, 54–66.
[53]
Marko Vukolić. 2015. The quest for scalable blockchain fabric: Proof-of-work vs. BFT replication. In Int. Workshop on Open Problems in Network Security. Springer, Cham, 112–125.
[54]
Bozhi Wang, Shiping Chen, Lina Yao, and Qin Wang. 2020. ChainSim: A P2P Blockchain Simulation Framework. In CCF China Blockchain Conf.Springer, Singapore, 1–16.
[55]
Jitao Wang, Bo Zhang, Kai Wang, Yuzhou Wang, and Weili Han. 2024. BFTDiagnosis: An automated security testing framework with malicious behavior injection for BFT protocols. Computer Networks (2024), 110404.
[56]
Ping-Lun Wang, Tzu-Wei Chao, Chia-Chien Wu, and Hsu-Chun Hsiao. 2022. Tool: An Efficient and Flexible Simulator for Byzantine Fault-Tolerant Protocols. In 52th Annu. IEEE/IFIP Int. Conf. on Dependable Systems and Networks. IEEE Comp. Soc., Washington, DC, USA, 287–294.
[57]
Philip Wette, Martin Dräxler, Arne Schwabe, Felix Wallaschek, Mohammad Hassan Zahraee, and Holger Karl. 2014. Maxinet: Distributed emulation of software-defined networks. In IFIP Networking Conf.IEEE Comp. Soc., Washington, DC, USA, 1–9.
[58]
Levin N Winter, Florena Buse, Daan De Graaf, Klaus Von Gleissenthall, and Burcu Kulahcioglu Ozkan. 2023. Randomized Testing of Byzantine Fault Tolerant Algorithms. Proceedings of the ACM on Programming Languages 7, OOPSLA1(2023), 757–788.
[59]
Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. 2018. HotStuff: BFT consensus in the lens of blockchain. arXiv preprint arXiv:1803.05069(2018).
[60]
Maofan Yin, Dahlia Malkhi, Michael K Reiter, Guy Golan Gueta, and Ittai Abraham. 2019. HotStuff: BFT consensus with linearity and responsiveness. In ACM Symposium on Principles of Distributed Computing (PODC). 347–356.
Index Terms
- Exploring Scalability of BFT Blockchain Protocols through Network Simulations
Recommendations
BFT in Blockchains: From Protocols to Use Cases
A blockchain is a distributed system that achieves strong security guarantees in storing, managing, and processing data. All blockchains achieve a common goal: building a decentralized system that provides a trustworthy service in an untrustworthy ...
Does my BFT protocol implementation scale?
DICG '22: Proceedings of the 3rd International Workshop on Distributed Infrastructure for the Common GoodThe novel blockchain generation of Byzantine fault-tolerant (BFT) state machine replication (SMR) protocols focuses on scalability and performance to meet the requirements of distributed ledger technology (DLT), e.g., decentralization and geographic ...
Autobahn: Seamless high speed BFT
SOSP '24: Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems PrinciplesToday's practical, high performance Byzantine Fault Tolerant (BFT) consensus protocols operate in the partial synchrony model. However, existing protocols are inefficient when deployments are indeed partially synchronous. They deliver either low latency ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
EISSN:1433-299X
Table of ContentsCopyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 21 August 2024
Accepted: 16 July 2024
Revised: 14 June 2024
Received: 15 January 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 175Total Downloads
- Downloads (Last 12 months)175
- Downloads (Last 6 weeks)61
Reflects downloads up to 30 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in