Journal of Grid Computing

, Volume 11, Issue 1, pp 63–81 | Cite as

Cloud Platform Datastore Support

  • Navraj Chohan
  • Chris Bunch
  • Chandra Krintz
  • Navyasri Canumalla
Article

Abstract

There are many datastore systems to choose from that differ in many ways including public versus private cloud support, data management interfaces, programming languages, supported feature sets, fault tolerance, consistency guarantees, configuration, and their deployment processes. In this paper, we focus on technologies for structured data access (database/datastore systems) in cloud systems. Our goal is to simplify the use of datastore systems through automation and to facilitate their empirical evaluation using real world applications. To enable this, we provide a cloud platform abstraction layer that decouples a data access API from its implementation. Applications that use this API can use any datastore that “plugs into” our abstraction layer, thus enabling application portability. We use this layer to extend the functionality of multiple datastores without modifying the datastores directly. Specifically, we provide support for ACID transaction semantics for popular key-value stores (none of which provide this feature). We integrate this layer into the AppScale cloud platform—an open-source cloud platform that executes cloud applications written in Python, Java, and Go, over virtualized cluster resources and infrastructures-as-a-service (Eucalyptus, OpenStack, and Amazon EC2). We use this system to investigate the impact of extending disparate datastores via the application portability layer with distributed transaction support.

Keywords

Cloud computing Cloud platform Datastore Key-value structured storage Online transaction processing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amazon Web Services home page: http://aws.amazon.com/. Accessed 1 Aug 2011
  2. 2.
    Baker, J., Bond, C., Corbett, J., Furman, J., Larson, A.K.J., Leon, J., Li, Y., Lloyd, A., Yushprakh, V.: Megastore: providing scalable, highly available storage for interactive services. In: Conference on Innovative Data Systems Research (CIDR), pp. 223–234 (2011)Google Scholar
  3. 3.
    Bernstein, P.A., Goodman, N.: Concurrency control in distributed database systems. ACM Comput. Surv. 13(2), 185–221 (1981)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman, Boston (1987)Google Scholar
  5. 5.
    Bunch, C., Chohan, N., Krintz, C., Chohan, J., Kupferman, J., Lakhina, P., Li, Y., Nomura, Y.: An evaluation of distributed datastores using the AppScale cloud platform. In: IEEE International Conference on Cloud Computing (2010)Google Scholar
  6. 6.
    Bunch, C., Chohan, N., Krintz, C., Shams, K.: Neptune: a domain specific language for deploying hpc software on cloud platforms. In: International Workshop on Scientific Cloud Computing, pp. 59–68 (2011)Google Scholar
  7. 7.
    Burrows, M.: The chubby lock service for loosely-coupled distributed systems. In: OSDI’06: Seventh Symposium on Operating System Design and Implementation (2006)Google Scholar
  8. 8.
    Cassandra: http://cassandra.apache.org/. Accessed 1 Aug 2011
  9. 9.
    Chandra, T., Griesemer, R., Redstone, J.: Paxos made live—an engineering perspective. In: PODC ’07: 26th ACM Symposium on Principles of Distributed Computing (2007)Google Scholar
  10. 10.
    Chang, F., Dean, J., Ghemawat, S., Hsieh, W., Wallach, D., Burrows, M., Chandra, T., Fikes, A., Gruber, R.: Bigtable: a distributed storage system for structured data. In: Symposium on Operating System Design and Implementation (2006)Google Scholar
  11. 11.
    Chang, F., Dean, J., Ghemawat, S., Hsieh, W., Wallach, D., Burrows, M., Chandra, T., Fikes, A., Gruber, R.: Bigtable: a distributed storage system for structured data. In: Proceedings of 7th Symposium on Operating System Design and Implementation (OSDI), pp. 205–218 (2006)Google Scholar
  12. 12.
    Chohan, N., Bunch, C., Nomura, Y., Krintz, C.: Database-agnostic transaction support for cloud infrastructures. In: IEEE International Conference on Cloud Computing (2011)Google Scholar
  13. 13.
    Cooper, B.F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.-A., Puz, N., Weaver, D., Yerneni, R.: PNUTS: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1(2), 1277–1288 (2008)Google Scholar
  14. 14.
    DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: Symposium on Operating System Principles (2007)Google Scholar
  15. 15.
    ejabberd: http://ejabberd.im. Accessed 1 Aug 2011
  16. 16.
    Eucalyptus home page: http://eucalyptus.cs.ucsb.edu/. Accessed 1 Aug 2011
  17. 17.
    Hadoop Distributed File System: http://hadoop.apache.org. Accessed 1 Aug 2011
  18. 18.
    Hadoop MapReduce: http://hadoop.apache.org/. Accessed 1 Aug 2011
  19. 19.
    HAProxy: http://haproxy.1wt.eu. Accessed 1 Aug 2011
  20. 20.
    HBase; http://hadoop.apache.org/hbase/. Accessed 1 Aug 2011
  21. 21.
    Hypertable: http://hypertable.org. Accessed 1 Aug 2011
  22. 22.
    Kernel based virtual machine: http://www.linux-kvm.org/. Accessed 1 Aug 2011
  23. 23.
    Kola, G., Kosar, T., Livny, M.: Faults in large distributed systems and what we can do about them. Lect. Notes Comput. Sci. 3648, 442–453 (2005)CrossRefGoogle Scholar
  24. 24.
    Kossmann, D., Kraska, T., Loesing, S.: An evaluation of alternative architectures for transaction processing in the cloud. In: International Conference on Management of Data, pp. 579–590 (2010)Google Scholar
  25. 25.
    Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM Trans. Database Syst. 6(2), 213–226 (1981)CrossRefGoogle Scholar
  26. 26.
    Lamport, L.: The part-time parliament. ACM Trans. Comput. Syst. 16(2), 133–169 (1998)CrossRefGoogle Scholar
  27. 27.
    memcached: http://memcached.org. Accessed 1 Aug 2011
  28. 28.
    MemcacheDB: http://memcachedb.org/. Accessed 1 Aug 2011
  29. 29.
    MongoDB: http://mongodb.org/. Accessed 1 Aug 2011
  30. 30.
    MySQL Cluster: http://www.mysql.com/cluster. Accessed 1 Aug 2011
  31. 31.
    Nginx: http://www.nginx.net. Accessed 1 Aug 2011
  32. 32.
    Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The eucalyptus open-source cloud-computing system. In: 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID) (2009)Google Scholar
  33. 33.
    OpenStack: http://openstack.org. Accessed 1 Aug 2011
  34. 34.
    Peng, D., Dabek, F.: Large-scale incremental processing using distributed transactions and notifications. In: Symposium on Operating System Design and Implementation (2010)Google Scholar
  35. 35.
    Protocol Buffers. Google’s Data Interchange Format: http://code.google.com/p/protobuf. Accessed 1 Aug 2011
  36. 36.
    Redis: http://redis.io. Accessed 1 Aug 2011
  37. 37.
    SimpleDB: http://aws.amazon.com/simpledb/. Accessed 1 Aug 2011
  38. 38.
    Voldemort: http://project-voldemort.com/. Accessed 1 Aug 2011
  39. 39.
    Wei, Z., Pierre, G., Chi, C.-H.: Scalable transactions for web applications in the cloud. In: Proceedings of the Euro-Par Conference, Delft, The Netherlands (2009). http://www.globule.org/publi/STWAC_europar2009.html. Accessed 1 Aug 2011
  40. 40.
    What is Google App Engine? http://code.google.com/appengine/docs/whatisgoogleappengine.html. Accessed 1 Aug 2011
  41. 41.
    XenSource: http://www.xensource.com/. Accessed 1 Aug 2011
  42. 42.
    ZooKeeper: http://hadoop.apache.org/zookeeper. Accessed 1 Aug 2011

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Navraj Chohan
    • 1
  • Chris Bunch
    • 1
  • Chandra Krintz
    • 1
  • Navyasri Canumalla
    • 1
  1. 1.Computer Science DepartmentUniversity of California, Santa BarbaraSanta BarbaraUSA

Personalised recommendations