The Journal of Supercomputing

, Volume 75, Issue 11, pp 7337–7365 | Cite as

Hierarchical Byzantine fault-tolerance protocol for permissioned blockchain systems

  • Quang Tung ThaiEmail author
  • Jong-Chul Yim
  • Tae-Whan Yoo
  • Hyun-Kyung Yoo
  • Ji-Young Kwak
  • Sun-Me Kim


Emerging blockchain technology has introduced a new challenge to the distributed system research: Can Byzantine fault-tolerance protocols scale up to, for example, hundreds of nodes? In this work, we introduce HiBFT, a hierarchical Byzantine fault-tolerance protocol to address the problem. The core idea is to divide replicas into groups and exchange consensus messages among groups, thus avoiding the necessity of message broadcasting. The motivation for such approach bases on the hierarchical property of network architecture in permissioned block chains, our target deployments. HiBFT works very much in the same way as the classical Practical Byzantine Fault-Tolerance protocol. However, it replaces the concept of physical replica with a logical one that represents a replica group. As such, protocol message complexity can be reduced from \(O(N^2)\) to \(O(s^2)\) where N and s are the total number of replicas and the number of groups. Additionally, using threshold signature scheme for representing a logical group results in two important improvements: The cost of signature verification is significantly reduced at each replica; blocks can be secured more effectively in terms of signature size. Our protocol guarantees safety and liveness under partially synchronous assumption with a correctness proof. Our experiment results show that the protocol can scale up to hundred of nodes.


Consensus Blockchain State machine replication Replicated systems Fault tolerance Byzantine failures 



This work was supported by the ICT R&D program of MSICT/IITP. [2017-0-00045, Hyper-connected Intelligent Infrastructure Technology Development]


  1. 1.
    Abd-El-Malek M, Ganger GR, Goodson GR, Reiter MK, Wylie JJ (2005) Fault-scalable Byzantine fault-tolerant services. ACM SIGOPS Oper Syst Rev 39:59–74CrossRefGoogle Scholar
  2. 2.
    Amir Y, Danilov C, Dolev D, Kirsch J, Lane J, Nita-Rotaru C, Olsen J, Zage D (2010) Steward: Scaling Byzantine fault-tolerant replication to wide area networks. IEEE Trans Dependable Secure Comput 7(1):80–93CrossRefGoogle Scholar
  3. 3.
    Androulaki E, Barger A, Bortnikov V, Cachin C, Christidis K, De Caro A, Enyeart D, Ferris C, Laventman G, Manevich, Y et al (2018) Hyperledger fabric: a distributed operating system for permissioned blockchains. In: Proceedings of the Thirteenth EuroSys Conference. ACM, p 30Google Scholar
  4. 4.
    Baker J, Bond C, Corbett JC, Furman JJ, Khorlin A, Larson J, Leon J-M, Li Y, Lloyd A, Yushprakh V (2011) Megastore: Providing scalable, highly available storage for interactive services. In: Proceedings of the Conference on Innovative Data system Research (CIDR), pp 223–234Google Scholar
  5. 5.
    Behl J, Distler T, Kapitza R (2017) Hybrids on steroids: SGX-based high performance BFT. In: Proceedings of the Twelfth European Conference on Computer Systems, pp 222–237. ACMGoogle Scholar
  6. 6.
    Ben-Or M (1983) Another advantage of free choice (extended abstract): Completely asynchronous agreement protocols. In: Proceedings of the second annual ACM symposium on principles of distributed computing. ACM, pp 27–30Google Scholar
  7. 7.
    Bracha G (1984) An asynchronous [(n-1)/3]-resilient consensus protocol. In: Proceedings of the third annual ACM symposium on Principles of distributed computing. ACM, pp 154–162Google Scholar
  8. 8.
    Buchman E (2016) Tendermint: Byzantine fault tolerance in the age of blockchains. Ph.D. thesisGoogle Scholar
  9. 9.
    Burrows M (2006) The chubby lock service for loosely-coupled distributed systems. In: Proceedings of the 7th symposium on operating systems design and implementation. USENIX Association, pp 335–350Google Scholar
  10. 10.
    Cachin C, Kursawe K, Shoup V (2005) Random oracles in constantinople: practical asynchronous byzantine agreement using cryptography. J Cryptol 18(3):219–246MathSciNetCrossRefGoogle Scholar
  11. 11.
    Castro M, Liskov B et al (1999) Practical Byzantine fault tolerance. OSDI 99:173–186Google Scholar
  12. 12.
    Chun B-G, Maniatis P, Shenker S, Kubiatowicz J (2007) Attested append-only memory: making adversaries stick to their word. ACM SIGOPS Oper Syst Rev 41:189–204CrossRefGoogle Scholar
  13. 13.
    Corbett JC, Dean J, Epstein M, Fikes A, Frost C, Furman JJ, Ghemawat S, Gubarev A, Heiser C, Hochschild P et al (2013) Spanner: Google’s globally distributed database. ACM Trans Computer Syst (TOCS) 31(3):8CrossRefGoogle Scholar
  14. 14.
    Correia M, Ferreira Neves N, Lung LC, Veríssimo P (2005) Low complexity Byzantine-resilient consensus. Distrib Comput 17(3):237–249CrossRefGoogle Scholar
  15. 15.
    Cowling J, Myers D, Liskov B, Rodrigues R, Shrira L (2006) HQ replication: a hybrid quorum protocol for Byzantine fault tolerance. In: Proceedings of the 7th symposium on Operating systems design and implementation. USENIX Association, pp 177–190Google Scholar
  16. 16.
    Duan S, Peisert S, Levitt KN (2014) hbft:speculative byzantine fault tolerance with minimum cost. IEEE Trans Dependable Secure Comput 12(1):58–70CrossRefGoogle Scholar
  17. 17.
    Dwork C, Lynch N, Stockmeyer L (1988) Consensus in the presence of partial synchrony. J ACM (JACM) 35(2):288–323MathSciNetCrossRefGoogle Scholar
  18. 18.
    Ethereum Foundation. Go ethreum (2018). Accessed 15 Oct 2018
  19. 19.
    Fischer MJ, Lynch NA, Paterson MS (1985) Impossibility of distributed consensus with one faulty process. J ACM (JACM) 32(2):374–382MathSciNetCrossRefGoogle Scholar
  20. 20.
    Hunt P, Konar M, Junqueira FP, Reed B (2010) Zookeeper: Wait-free coordination for internet-scale systems. In USENIX Annual Technical Conference, vol 8. Boston, MA, USA, p 9Google Scholar
  21. 21.
    Johnson D, Menezes A, Vanstone S (2001) The elliptic curve digital signature algorithm (ECDSA). Int J Inf Secur 1(1):36–63CrossRefGoogle Scholar
  22. 22.
    Kotla R, Alvisi L, Dahlin M, Clement A, Wong E (2007) Zyzzyva: speculative Byzantine fault tolerance. ACM SIGOPS Oper Syst Rev 41:45–58CrossRefGoogle Scholar
  23. 23.
    Lamport L (1998) The part-time parliament. ACM Trans Comput Syst (TOCS) 16(2):133–169CrossRefGoogle Scholar
  24. 24.
    Lamport L et al (2001) Paxos made simple. ACM Sigact News 32(4):18–25Google Scholar
  25. 25.
    Lamport L, Shostak R, Pease M (1982) The Byzantine generals problem. ACM Trans Program Lang Syst (TOPLAS) 4(3):382–401CrossRefGoogle Scholar
  26. 26.
    Levin D, Douceur JR, Lorch JR, Moscibroda T (2009) TrInc: Small trusted hardware for large distributed systems. NSDI 9:1–14Google Scholar
  27. 27.
    Malkhi D, Reiter M (1998) Byzantine quorum systems. Distrib Comput 11(4):203–213CrossRefGoogle Scholar
  28. 28.
    Mao Y, Junqueira FP, Marzullo K (2008) Mencius: building efficient replicated state machines for WANs. In: Proceedings of OSDI’08, USENIX Conference on Operating Systems Design and Implementation, USENIX, pp 369–384Google Scholar
  29. 29.
    Miller A, Xia Y, Croman K, Shi E, Song D (2016) The honey badger of BFT protocols. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, pp 31–42Google Scholar
  30. 30.
    Nakamoto S (2008) Bitcoin: a peer-to-peer electronic cash system. Technical reportGoogle Scholar
  31. 31.
    Oki BM, Liskov BH (1988) Viewstamped replication: a new primary copy method to support highly-available distributed systems. In: Proceedings of the seventh annual ACM symposium on principles of distributed computing. ACM, pp 8–17Google Scholar
  32. 32.
    Ongaro D, Ousterhout JK (2014) In search of an understandable consensus algorithm. In: USENIX Annual Technical Conference, pp 305–319Google Scholar
  33. 33.
    Rabin MO (1983) Randomized Byzantine generals. In: 24th annual symposium on foundations of computer science (SFCS 1983). IEEE, pp 403–409Google Scholar
  34. 34.
    Schneider FB (1990) Implementing fault-tolerant services using the state machine approach: a tutorial. ACM Comput Surv (CSUR) 22(4):299–319CrossRefGoogle Scholar
  35. 35.
    Shamir A (1979) How to share a secret. Commun ACM 22(11):612–613MathSciNetCrossRefGoogle Scholar
  36. 36.
    Shoup V (2000) Practical threshold signatures. In: Advances in cryptology—EUROCRYPT 2000. Springer, pp 207–220Google Scholar
  37. 37.
    Tschorsch F, Scheuermann B (2016) Bitcoin and beyond: a technical survey on decentralized digital currencies. IEEE Commun Surv Tutor 18(3):2084–2123CrossRefGoogle Scholar
  38. 38.
    Underwood S (2016) Blockchain beyond bitcoin. Commun ACM 59(11):15–17CrossRefGoogle Scholar
  39. 39.
    Veronese GS, Correia M, Bessani AN, Lung LC 2009) Spin one’s wheels? byzantine fault tolerance with a spinning primary. In: 2009 28th IEEE International Symposium on Reliable Distributed Systems. IEEE, pp 135–144Google Scholar
  40. 40.
    Veronese GS, Correia M, Bessani AN, Lung LC (2010) EBAWA: efficient byzantine agreement for wide-area networks. In: 2010 IEEE 12th international symposium on high assurance systems engineering. IEEE, pp 10–19Google Scholar
  41. 41.
    Veronese GS, Correia M, Bessani AN, Lung LC, Verissimo P (2013) Efficient byzantine fault-tolerance. IEEE Trans Comput 62(1):16–30MathSciNetCrossRefGoogle Scholar
  42. 42.
    Vukolić M (2015) The quest for scalable blockchain fabric: Proof-of-work vs. BFT replication. In: International Workshop on Open Problems in Network Security. Springer, pp 112–125Google Scholar
  43. 43.
    Vukolić M (2017) Rethinking permissioned blockchains. In: Proceedings of the ACM Workshop on Blockchain, Cryptocurrencies and Contracts. ACM, pp 3–7Google Scholar
  44. 44.
    Wood G (2014) ETHEREUM: a secure decentralised generalised transaction ledger, Technical Report.
  45. 45.
    Yin J, Martin J-P, Venkataramani A, Alvisi L, Dahlin M (2003) Separating agreement from execution for byzantine fault tolerant services. ACM SIGOPS Oper Syst Rev 37(5):253–267CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Quang Tung Thai
    • 1
    Email author
  • Jong-Chul Yim
    • 1
  • Tae-Whan Yoo
    • 1
  • Hyun-Kyung Yoo
    • 1
  • Ji-Young Kwak
    • 1
  • Sun-Me Kim
    • 1
  1. 1.Electronics and Telecommunications Research InstituteDaejeonSouth Korea

Personalised recommendations