Skip to main content
SpringerLink
Log in

SD2DS-Based Datastore for Large Files

  • Conference paper
  • First Online:
Proceedings of the 2015 Federated Conference on Software Development and Object Technologies (SDOT 2015)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 511))

Included in the following conference series:

Abstract

We are introducing Scalable Distributed Two-Layer Datastore. The system that is an efficient solution while storing relatively big multimedia files. In the article we are focusing on storing high-resolution photos. We are introducing some of the key implementation concepts as well as the careful evaluation. We are comparing our solution with two of the most recognizable data storing systems: MongoDB and Memcached.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sadalage, P.J., Fowler, M.: NoSQL distilled: a brief guide to the emerging world of polyglot persistence. Pearson Education, London (2012)

    Google Scholar 

  2. DataStax, “DataStax Documentation Apache CassandraTM 2.1.” http://docs.datastax.com/en/cassandra/2.1/. Accessed 14 Apr 2015

  3. Quora, “Is HBase appropriate for indexed blob storage in HDFS?” https://www.quora.com/Is-HBase-appropriate-for-indexed-blob-storage-in-HDFS. Accessed 7 Oct 2015

  4. Sapiecha, K., Łukawski, G.: Scalable distributed two–layer data structures (SD2DS). IJDST 4, 15–30 (2013)

    Google Scholar 

  5. Krechowicz, A., Deniziak, S., Bedla, M., Chrobot, A., Łukawski, G.: Scalable distributed two-layer block based datastore. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9573, pp. 302–311. Springer, Heidelberg (2016). doi:10.1007/978-3-319-32149-3_29

    Chapter  Google Scholar 

  6. Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. (TOCS) 26(2), 4 (2008)

    Article  Google Scholar 

  7. DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. In: ACM SIGOPS Operating Systems Review, vol. 41, no. 6, pp. 205–220. ACM (2007)

    Google Scholar 

  8. MongoDB, “The MongoDB 3.0 Manual.” http://docs.mongodb.org/manual/. Accessed 14 Apr 2015

  9. Bronson, N., Amsden, Z., Cabrera, G., Chakka, P., Dimov, P., Ding, H., Ferris, J., Giardullo, A., Kulkarni, S., Li, H.C., et al.: Tao: facebook’s distributed data store for the social graph. In: USENIX Annual Technical Conference, pp. 49–60 (2013)

    Google Scholar 

  10. Memcached, “Memcached – A Distributed Memory Object Caching System.” http://memcached.org. Accessed 13 Apr 2015

  11. Chidambaram, V., Ramamurthi, D.: “Performance analysis of mem- cached.” http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.409.411&rep=rep1&type=pdf. Accessed 13 Apr 2015

  12. Carra, D., Michiardi, P.: Memory partitioning in memcached: an experimental performance analysis. In: ICC 2014, IEEE International Conference on Communications, 10–14 June 2014, Sydney, Australia, June 2014. http://wwweurecom.fr/publication/4320

  13. Memcached, “Timeouts.” https://code.google.com/p/memcached/wiki/Timeouts. Accessed 5 Oct 2015

  14. Chu, S.: “Memcachedb: The Complete Guide.” http://memcachedb.org/memcachedb-guide-1.0.pdf. Accessed 13 Apr 2015

  15. Krechowicz, A., Deniziak, S., Łukawski, G., Bedla, M.: Preserving data consistency in scalable distributed two layer data structures. Beyond Databases, Architectures and Structures. Communications in Computer and Information Science, vol. 521, pp. 126–135. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18422-7_11

    Google Scholar 

  16. Litwin, W., Neimat, M.-A., Schneider, D.A.: LH* — a scalable, distributed data structure. ACM Trans. Database Syst. 21(4), 480–525 (1996). http://citeseer.ist.psu.edu/litwin96lh.html

    Article  Google Scholar 

  17. Litwin, W., Neimat, M.-A., Schneider, D.: RP*: a family of order preserving scalable distributed data structures. In: Proceedings of the Twentieth International Conference on Very Large Databases, Santiago, Chile, pp. 342–353 (1994). citeseer.ist.psu.edu/736278.html

  18. Litwin, W., Neimat, M.-A., Schneider, D.: Rp*: a family of order preserving scalable distributed data structures. In: VLDB, vol. 94, pp. 12–15 (1994)

    Google Scholar 

  19. Litwin, W.: Linear hashing: a new tool for file and table addressing. In: VLDB 1980: Proceedings of the Sixth International Conference on Very Large Data Bases. VLDB Endowment, pp. 212–223 (1980)

    Google Scholar 

  20. Litwin, W.: Trie hashing. In: Proceedings of the 1981 ACM SIGMOD International Conference on Management of Data, pp. 19–29. ACM (1981)

    Google Scholar 

  21. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 143–154. ACM (2010)

    Google Scholar 

  22. Fei-Fei, R.F.L., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR 2004, IEEE Workshop on Generative-Model Based Vision (2004)

    Google Scholar 

  23. Griffin, G., Holub, A., Perona, P.: “Caltech-256 Object Category Dataset,” California Institute of Technology, Technical report CNS-TR-2007- 001 (2007). http://authors.library.caltech.edu/7694

  24. Łukawski, G., Sapiecha, K.: Fault tolerant record placement for decentralized SDDS LH*. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J. (eds.) PPAM 2007. LNCS, vol. 4967, pp. 312–320. Springer, Heidelberg (2008). doi:10.1007/978-3-540-68111-3_33

    Chapter  Google Scholar 

  25. Janowski, L., Kozowski, P., Baran, R., Romaniak, P., Gowacz, A., Rusc, T.: Quality assessment for a visual and automatic license plate recognition. Multimedia Tools Appl. 68(1), 23–40 (2014)

    Article  Google Scholar 

  26. Deniziak, R., Bak, S., Czarnecki, R.: Synthesis of real-time cloud applications for internet of things. Turk. J. Electr. Eng. Comput. Sci. 7719, 35–49 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kielce University of Technology, Kielce, Poland

    Adam Krechowicz, Arkadiusz Chrobot, Stanisław Deniziak & Grzegorz Łukawski

Authors
  1. Adam Krechowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arkadiusz Chrobot
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stanisław Deniziak
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Grzegorz Łukawski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Krechowicz .

Editor information

Editors and Affiliations

  1. Faculty of Management Science and Informatics, University of Zilina, Zilina, Slovakia

    Jan Janech

  2. Faculty of Management Science and Informatics, University of Zilina, Zilina, Slovakia

    Jozef Kostolny

  3. Institute of Metrology, Electronics, Zielona Góra, Poland

    Tomasz Gratkowski

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Krechowicz, A., Chrobot, A., Deniziak, S., Łukawski, G. (2017). SD2DS-Based Datastore for Large Files. In: Janech, J., Kostolny, J., Gratkowski, T. (eds) Proceedings of the 2015 Federated Conference on Software Development and Object Technologies. SDOT 2015. Advances in Intelligent Systems and Computing, vol 511. Springer, Cham. https://doi.org/10.1007/978-3-319-46535-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46535-7_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46534-0

  • Online ISBN: 978-3-319-46535-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation