Abstract
Scalability is one of the most relevant features of today’s data management systems. In order to achieve high scalability and availability, recent distributed key-value stores refrain from costly replica coordination when processing requests. However, these systems typically do not perform well under churn. In this paper, we propose DataFlagons, a large-scale key-value store that integrates epidemic dissemination with a probabilistic total order broadcast algorithm. By ensuring that all replicas process requests in the same order, DataFlagons provides probabilistic strong data consistency while achieving high scalability and robustness under churn.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Our system was named DataFlagons to convey an improvement over DataFlasks, as flagons are arguably more robust and consistent containers than flasks.
- 2.
- 3.
- 4.
Note that this is a high-level quantitative comparison based on prior studies [8], as a thorough experimental analysis for robustness is currently lacking in the literature.
- 5.
Our implementation of DataFlasks, as well as that of the simulator used in the experiments, are detailed in Sect. 4.
- 6.
We omit a description of the flow for a get request, as it is similar to that of a put.
- 7.
For messages with the same logical clock, DataFlagons orders them in ascending order of the ids of their broadcasting nodes.
- 8.
References
Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. 26(2), 4:1–4:26 (2008)
Cooper, B.F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.A., Puz, N., Weaver, D., Yerneni, R.: Pnuts: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1(2), 1277–1288 (2008)
Corbett, J.C., Dean, J., Epstein, M., Fikes, A., Frost, C., Furman, J.J., Ghemawat, S., Gubarev, A., Heiser, C., Hochschild, P., Hsieh, W., Kanthak, S., Kogan, E., Li, H., Lloyd, A., Melnik, S., Mwaura, D., Nagle, D., Quinlan, S., Rao, R., Rolig, L., Saito, Y., Szymaniak, M., Taylor, C., Wang, R., Woodford, D.: Spanner: Google’s globally distributed database. ACM Trans. Comput. Syst. 31(3), 251–264 (2013)
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. ACM SIGOPS Oper. Syst. Rev. 41(6), 205–220 (2007)
Eyal, I., Gencer, A.E., Sirer, E.G., Van Renesse, R.: Bitcoin-NG: a scalable blockchain protocol. In: NSDI 2016. USENIX Association (2016)
Koldehofe, B.: Simple gossiping with balls and bins. Stud. Inform. Univ. 3(1), 43–60 (2004)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. ACM SIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010)
Lourenço, J.R., Cabral, B., Carreiro, P., Vieira, M., Bernardino, J.: Choosing the right NoSQL database for the job: a quality attribute evaluation. J. Big Data 2(1), 18 (2015)
Maia, F., Matos, M., Vilaça, R., Pereira, J., Oliveira, R., Riviere, E.: Dataflasks: epidemic store for massive scale systems. In: SRDS 2014. IEEE (2014)
Matos, M., Mercier, H., Felber, P., Oliveira, R., Pereira, J.: EpTO: an epidemic total order algorithm for large-scale distributed systems. In: Middleware 2015. ACM (2015)
Rhea, S., Geels, D., Roscoe, T., Kubiatowicz, J.: Handling churn in a DHT. In: Proceedings of the Annual Conference on USENIX Annual Technical Conference, ATC 2004, p. 10. USENIX Association, Berkeley (2004)
Vogels, W.: Eventually consistent. Commun. ACM 52(1), 40–44 (2009)
Voulgaris, S., Gavidia, D., van Steen, M.: CYCLON: inexpensive membership management for unstructured P2P overlays. J. Netw. Syst. Manag. 13(2), 197–217 (2005)
Acknowledgments
The authors would like to thank the anonymous reviewers for their valuable feedback. This work was partially supported by Project “TEC4Growth - Pervasive Intelligence, Enhancers and Proofs of Concept with Industrial Impact” (NORTE-01-0145-FEDER-000020), financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme within project “POCI-01-0145-FEDER-006961”, and by National Funds through the Portuguese funding agency, FCT – Fundação para a Ciência as part of project “UID/EEA/50014/2013”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 IFIP International Federation for Information Processing
About this paper
Cite this paper
Ribeiro, J., Machado, N., Maia, F., Matos, M. (2018). Totally Ordered Replication for Massive Scale Key-Value Stores. In: Bonomi, S., Rivière, E. (eds) Distributed Applications and Interoperable Systems. DAIS 2018. Lecture Notes in Computer Science(), vol 10853. Springer, Cham. https://doi.org/10.1007/978-3-319-93767-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-93767-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93766-3
Online ISBN: 978-3-319-93767-0
eBook Packages: Computer ScienceComputer Science (R0)
