Skip to main content
SpringerLink
Log in
Book cover

Resource Management in Utility and Cloud Computing pp 57–77Cite as

Flexible Resource Sharing Management

  • Chapter
  • First Online:

  • 823 Accesses

Part of the book series: SpringerBriefs in Computer Science ((BRIEFSCOMPUTER))

Abstract

This chapter presents CloudBay, an online resource trading and leasing platform for multi-party resource sharing. It is a proof-of-concept design bridging the gap between resource providers and resource customers. With the help of CloudBay, the untapped computing power privately owned by multiple organizations is unleashed. The design and implementation of the CloudBay project presents the most challenge to our exploration of cost-effective resource management strategy design. Following a market-oriented design principle, CloudBay provides an abstraction of a shared virtual resource space across multiple administration domains, and features enhanced functionalities for scalable and automatic resource management and efficient service provisioning. CloudBay distinguishes itself from existing research and contributes in mainly two aspects. First, it leverages scalable network virtualization and self-configurable virtual appliances to facilitate resource federation and parallel application deployment. Second, CloudBay adopts an eBay-style transaction model that supports differentiated services with different levels of job priorities. For cost-sensitive users, CloudBay implements an efficient matchmaking algorithm based on the auction theory and enables opportunistic resource access through preemptive service scheduling. The proposed CloudBay platform stands between HPC service sellers and buyers, and offers a comprehensive solution for resource advertising and stitching, transaction management, and application-to-infrastructure mapping. In this chapter, we present the design details of CloudBay, and briefly discuss lessons learnt and challenges encountered in the implementation process.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The buy-request can be viewed as a special case of bid-request where users are willing to pay a fixed predefined amount.

  2. 2.

    We simply append the same text to generate larger size of input.

References

  1. Abbes, H., Cerin, C., Jemni, M.: Bonjourgrid: Orchestration of multi-instances of grid middlewares on institutional desktop grids. In: Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing (IPDPS’09), pp. 1–8 (2009)

    Google Scholar 

  2. Archer. www.archer-project.org

  3. Ausubel, L.M.: An efficient ascending-bid auction for multiple objects. American Economic Review 94(5), 1452–1475 (2004)

    Google Scholar 

  4. Bailey Lee, C., Schwartzman, Y., Hardy, J., Snavely, A.: Are user runtime estimates inherently inaccurate? In: Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing (JSSPP’04), pp. 253–263 (2005)

    Google Scholar 

  5. Ben-Yehuda, M., Day, M.D., Dubitzky, Z., Factor, M., Har’El, N., Gordon, A., Liguori, A., Wasserman, O., Yassour, B.A.: The turtles project: Design and implementation of nested virtualization. In: USENIX OSDI’10 (2010)

    Google Scholar 

  6. Buyya, R., Abramson, D., Giddy, J.: Nimrod/g: an architecture for a resource management and scheduling system in a global computational grid. In: Proceedings of the fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region, pp. 283–289 (2000)

    Google Scholar 

  7. Carroll, T.E., Grosu, D.: Incentive compatible online scheduling of malleable parallel jobs with individual deadlines. In: Proceedings of the 2010 39th International Conference on Parallel Processing (ICPP’10), pp. 516–524 (2010)

    Google Scholar 

  8. Cirne, W., Brasileiro, F., Andrade, N., Costa, L., Andrade, A., Novaes, R., Mowbray, M.: Labs of the world, unite!!! Journal of Grid Computing 4(3), 225–246 (2006)

    Google Scholar 

  9. Cloud Exchange. Http://www.cloudexchange.org/

  10. CloudNet. Available: http://www.cloud-net.org/

  11. Condor Project Home Page. http://research.cs.wisc.edu/condor/ (2009)

  12. Di, S., Wang, C.L., Zhang, W., Cheng, L.: Probabilistic best-fit multi-dimensional range query in self-organizing cloud. In: Proceedings of the 2011 International Conference on Parallel Processing (ICPP’11), pp. 763–772 (2011)

    Google Scholar 

  13. Django. https://www.djangoproject.com/ (2012)

  14. FutureGrid. Available: http://futuregrid.org/ (2012)

  15. Ganguly, A., Agrawal, A., Boykin, O.P., Figueiredo, R.: IP over P2P: Enabling self-configuring virtual IP networks for grid computing. In: Proceedings of the 20th international conference on parallel&distributed processing (IPDPS’06) (2006)

    Google Scholar 

  16. HPC on AWS. http://aws.amazon.com/hpc-applications/ (2012)

  17. Kale, L., Kumar, S., Potnuru, M., DeSouza, J., Bandhakavi, S.: Faucets: efficient resource allocation on the computational grid. In: Proceedings of the 2004 International Conference on Parallel Processing (ICPP’04), pp. 396–405 (2004)

    Google Scholar 

  18. Krieger, O., McGachey, P., Kanevsky, A.: Enabling a marketplace of clouds: VMware’s vCloud director. SIGOPS Oper. Syst. Rev. 44, 103–114 (2010)

    Google Scholar 

  19. Krishnan, S., Tatineni, M., Baru, C.: myhadoop - hadoop-on-demand on traditional hpc resources. Tech. Rep. SDSC TR-2011-2, San Diego Supercomputer Center, University of California San Diego (2011)

    Google Scholar 

  20. Lai, K., Rasmusson, L., Adar, E., Zhang, L., Huberman, B.A.: Tycoon: An implementation of a distributed, market-based resource allocation system. Multiagent Grid Syst. 1, 169–182 (2005)

    Google Scholar 

  21. Lawson, B.G., Smirni, E.: Multiple-queue backfilling scheduling with priorities and reservations for parallel systems. SIGMETRICS Perform. Eval. Rev. 29, 40–47 (2002)

    Google Scholar 

  22. Marshall, P., Keahey, K., Freeman, T.: Improving utilization of infrastructure clouds. In: Proceedings of the 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid’11), pp. 205–214 (2011)

    Google Scholar 

  23. Nakamoto, S.: Bitcoin: A Peer-to-Peer Electronic Cash System. http://bitcoin.org/bitcoin.pdf (2010)

  24. Regev, O., Nisan, N.: The POPCORN marketan online market for computational resources. In: Proceedings of the first international conference on Information and computation economies (ICE’98), pp. 148–157 (1998)

    Google Scholar 

  25. Shudler, S., Amar, L., Barak, A., Mu’alem, A.: The effects of untruthful bids on user utilities and stability in computing markets. In: proceedings of the 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing (CCGrid’10), pp. 205–214 (2010)

    Google Scholar 

  26. Simpy. Available: http://simpy.sourceforge.net/ (2012)

  27. Stokely, M., Winget, J., Keyes, E., Grimes, C., Yolken, B.: Using a market economy to provision compute resources across planet-wide clusters. In: Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing (IPDPS 2009), pp. 1–8 (2009)

    Google Scholar 

  28. Teo, Y.M., Mihailescu, M.: A strategy-proof pricing scheme for multiple resource type allocations. In: Proceedings of the 2009 International Conference on Parallel Processing (ICPP’09), pp. 172–179 (2009)

    Google Scholar 

  29. Waldspurger, C., Hogg, T., Huberman, B., Kephart, J., Stornetta, W.: Spawn: a distributed computational economy. IEEE Transactions on Software Engineering 18(2), 103–117 (1992)

    Google Scholar 

  30. Wolinsky, D., Figueiredo, R.: Experiences with self-organizing, decentralized grids using the grid appliance. In: Proceedings of the 20th international symposium on High performance distributed computing (HPDC’11) (2011)

    Google Scholar 

  31. Wolinsky, D.I., Figueiredo, R.: Grid appliance user interface. URL: http://www.grid-appliance.org (2009)

  32. Wolinsky, D.I., Liu, Y., Juste, P.S., Venkatasubramanian, G., Figueiredo, R.: On the design of scalable, self-configuring virtual networks. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC’09), pp. 13:1–13:12 (2009)

    Google Scholar 

  33. Wolski, R., Plank, J.S., Brevik, J., Bryan, T.: G-commerce: Market formulations controlling resource allocation on the computational grid. In: Proceedings of the 15th International Parallel & Distributed Processing Symposium (IPDPS 2001), pp. 46–53 (2001)

    Google Scholar 

  34. Xu, Z., Di, S., Zhang, W., Cheng, L., Wang, C.L.: Wavnet: Wide-area network virtualization technique for virtual private cloud. In: Proceedings of the 2011 International Conference on Parallel Processing (ICPP’11), pp. 285–294 (2011)

    Google Scholar 

  35. Zaharia, M., Konwinski, A., Joseph, A.D., Katz, R., Stoica, I.: Improving mapreduce performance in heterogeneous environments. In: OSDI’08, pp. 29–42 (2008)

    Google Scholar 

  36. Zhao, H.: Exploring Cost-Effective Resource Management Strategies in the Age of Utility Computing. Ph.D. thesis, University of Florida, Gainesville, FL, USA (2013)

    Google Scholar 

  37. Zhao, H., Yu, Z., Tiwari, S., Mao, X., Lee, K., Wolinsky, D., Li, X., Figueiredo, R.: Cloudbay: Enabling an online resource market place for open clouds. In: IEEE Fifth International Conference on Utility and Cloud Computing (UCC’12), pp. 135–142 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Florida, Gainesville, FL, USA

    Han Zhao & Xiaolin Li

Authors
  1. Han Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaolin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2013 The Author(s)

About this chapter

Cite this chapter

Zhao, H., Li, X. (2013). Flexible Resource Sharing Management. In: Resource Management in Utility and Cloud Computing. SpringerBriefs in Computer Science. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8970-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-8970-2_4

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-8969-6

  • Online ISBN: 978-1-4614-8970-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation