Skip to main content
SpringerLink
Log in
SpringerLink
Log in

SHMemCache: Enabling Memcached on the OpenSHMEM Global Address Model

  • Conference paper
  • First Online:
OpenSHMEM and Related Technologies. Enhancing OpenSHMEM for Hybrid Environments (OpenSHMEM 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10007))

Included in the following conference series:

Abstract

Memcached is a popular key-value memory store for big data applications. Its performance is directly related to the underlying run-time systems including the communication protocols. OpenSHMEM is a strong run-time system that supports data access to both local and remote memory through a simple shared-memory addressing model. In view of the communication compatibilities between Memcached and OpenSHMEM, we propose to integrate the programmability and portability of OpenSHMEM for supporting Memcached on a wide variety of HPC systems. In this paper, we present the design and implementation of SHMemCache, an OpenSHMEM-based communication conduit for Memcached, which can expand the deployment scope of Memcached to various leadership facilities with OpenSHMEM run-time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Apache Hadoop Project. http://hadoop.apache.org/

  2. Libmemcached. http://libmemcached.org/libMemcached.html

  3. Memcached. https://memcached.org/downloads

  4. OpenMPI. https://www.open-mpi.org/

  5. Aguilera, M.K., Merchant, A., Shah, M., Veitch, A., Karamanolis, C.: Sinfonia: a new paradigm for building scalable distributed systems. ACM SIGOPS Oper. Syst. Rev. 41, 159–174 (2007)

    Article  Google Scholar 

  6. Andersen, D.G., Franklin, J., Kaminsky, M., Phanishayee, A., Tan, L., Vasudevan, V.: Fawn: a fast array of wimpy nodes. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 1–14. ACM (2009)

    Google Scholar 

  7. Appavoo, J., Waterland, A., Da Silva, D., Uhlig, V., Rosenburg, B., Van Hensbergen, E., Stoess, J., Wisniewski, R., Steinberg, U.: Providing a cloud network infrastructure on a supercomputer. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp. 385–394. ACM (2010)

    Google Scholar 

  8. Corbett, J.C., Dean, J., Epstein, M., Fikes, A., Frost, C., Furman, J.J., Ghemawat, S., Gubarev, A., Heiser, C., Hochschild, P., et al.: Spanner: Googles globally distributed database. ACM Trans. Comput. Syst. (TOCS) 31(3), 8 (2013)

    Article  Google Scholar 

  9. Dragojević, A., Narayanan, D., Castro, M., Hodson, O.: Farm: fast remote memory. In: 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2014), pp. 401–414 (2014)

    Google Scholar 

  10. Fan, B., Andersen, D.G., Kaminsky, M.: Memc3: Compact and concurrent memcache with dumber caching and smarter hashing. Presented as part of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2013), pp. 371–384 (2013)

    Google Scholar 

  11. Greenberg, H., Bent, J., Grider, G.: Mdhim: a parallel key/value framework for HPC. In: 7th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 2015) (2015)

    Google Scholar 

  12. Issa, J., Figueira, S.: Hadoop and memcached: performance and power characterization and analysis. J. Cloud Comput. 1(1), 1–20 (2012)

    Article  Google Scholar 

  13. Jose, J., Subramoni, H., Luo, M., Zhang, M., Huang, J., Wasi-ur Rahman, M., Islam, N.S., Ouyang, X., Wang, H., Sur, S., et al.: Memcached design on high performance RDMA capable interconnects. In: 2011 International Conference on Parallel Processing (ICPP), pp. 743–752. IEEE (2011)

    Google Scholar 

  14. Kapoor, R., Porter, G., Tewari, M., Voelker, G.M., Vahdat, A.: Chronos: predictable low latency for data center applications. In: Proceedings of the Third ACM Symposium on Cloud Computing, p. 9. ACM (2012)

    Google Scholar 

  15. Lim, H., Fan, B., Andersen, D.G., Kaminsky, M.: Silt: a memory-efficient, high-performance key-value store. In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, pp. 1–13. ACM (2011)

    Google Scholar 

  16. Mao, Y., Kohler, E., Morris, R.T.: Cache craftiness for fast multicore key-value storage. In: Proceedings of the 7th ACM European Conference on Computer Systems, pp. 183–196. ACM (2012)

    Google Scholar 

  17. Mitchell, C., Geng, Y., Li, J.: Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In: USENIX Annual Technical Conference, pp. 103–114 (2013)

    Google Scholar 

  18. Narula, N., Cutler, C., Kohler, E., Morris, R.: Phase reconciliation for contended in-memory transactions. In: 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2014), pp. 511–524 (2014)

    Google Scholar 

  19. Ruan, W., Vyas, T., Liu, Y., Spear, M.: Transactionalizing legacy code: an experience report using GCC and memcached. ACM SIGARCH Comput. Architect. News 42, 399–412 (2014)

    Google Scholar 

  20. Thomson, A., Diamond, T., Weng, S.-C., Ren, K., Shao, P., Abadi, D.J.: Calvin: fast distributed transactions for partitioned database systems. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, pp. 1–12. ACM (2012)

    Google Scholar 

  21. Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 18–32. ACM (2013)

    Google Scholar 

  22. Wang, Y., Meng, X., Zhang, L., Tan, J.: C-hint: an effective and reliable cache management for RDMA-accelerated key-value stores. In: Proceedings of the ACM Symposium on Cloud Computing, pp. 1–13. ACM (2014)

    Google Scholar 

  23. Wang, Y., Zhang, L., Tan, J., Li, M., Gao, Y., Guerin, X., Meng, X., Meng, S.: Hydradb: a resilient RDMA-driven key-value middleware for in-memory cluster computing. In: SC 2015, p. 22. ACM (2015)

    Google Scholar 

  24. Wei, X., Shi, J., Chen, Y., Chen, R., Chen, H.: Fast in-memory transaction processing using RDMA and HTM. In: Proceedings of the 25th Symposium on Operating Systems Principles, pp. 87–104. ACM (2015)

    Google Scholar 

  25. Wu, X., Zhang, L., Wang, Y., Ren, Y., Hack, M., Jiang, S.: Zexpander: a key-value cache with both high performance and fewer misses. In: Proceedings of the Eleventh European Conference on Computer Systems, p. 14. ACM (2016)

    Google Scholar 

  26. Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M.J., Shenker, S., Stoica, I.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, NSDI 2012, p. 2. USENIX Association, Berkeley (2012)

    Google Scholar 

Download references

Acknowledgment

We are very thankful for the insightful comments from the anonymous reviewers. This work was supported in part by a contract from Oak Ridge National Laboratory to Florida State University.

Author information

Authors and Affiliations

  1. Oak Ridge National Laboratory, Florida State University, Tallahassee, USA

    Huansong Fu, Kunal SinghaRoy, Manjunath Gorentla Venkata, Yue Zhu & Weikuan Yu

Authors
  1. Huansong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kunal SinghaRoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manjunath Gorentla Venkata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yue Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weikuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huansong Fu .

Editor information

Editors and Affiliations

  1. Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

    Manjunath Gorentla Venkata

  2. Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

    Neena Imam

  3. Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

    Swaroop Pophale

  4. Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

    Tiffany M. Mintz

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Fu, H., SinghaRoy, K., Venkata, M.G., Zhu, Y., Yu, W. (2016). SHMemCache: Enabling Memcached on the OpenSHMEM Global Address Model. In: Gorentla Venkata, M., Imam, N., Pophale, S., Mintz, T. (eds) OpenSHMEM and Related Technologies. Enhancing OpenSHMEM for Hybrid Environments. OpenSHMEM 2016. Lecture Notes in Computer Science(), vol 10007. Springer, Cham. https://doi.org/10.1007/978-3-319-50995-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50995-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50994-5

  • Online ISBN: 978-3-319-50995-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation