Skip to main content
SpringerLink
Log in

Scalability Evaluation of Big Data Processing Services in Clouds

  • Conference paper
  • First Online:
Book cover Benchmarking, Measuring, and Optimizing (Bench 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11459))

Included in the following conference series:

  • 1167 Accesses

Abstract

Currently, many cloud providers deploy their big data processing systems as cloud services, which helps users conveniently manage and process their data in clouds. Among different service providers’ big data processing services, how to evaluate and compare their scalability is an interesting and challenging work. Most traditional benchmark tools focus on performance evaluation of big data processing systems, such as aggregated throughput and IOPS, but fail to conduct a quantitative analysis of their scalability. In this paper, we propose a measurement methodology to quantify the scalability of big data processing services, which makes the cloud services scalability comparable. We conduct a group of comparative experiments on AliCloud E-MapReduce and Baidu MRS, and collect their respective scalability characteristics under Hadoop and Spark workloads. The scalability characteristics observed in our work could help cloud users choose the best cloud service platform to set up an optimized big data processing system to achieve their specific goals more successfully.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hadoop. http://hadoop.apache.org/

  2. Spark. https://spark.apache.org/

  3. Amazon EMR. https://aws.amazon.com/cn/emr/

  4. AliCloud E-MapReduce. https://www.aliyun.com/product/emapreduce?utm_medium=text&utm_source=baidu&utm_campaign=emr&utm_content=se_331947

  5. Baidu BMR. https://cloud.baidu.com/product/bmr.html?track=cp:nsem|pf:pc|pp:bmr|pu:brand|ci:|kw:50293

  6. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: SoCC, pp. 143–154 (2010)

    Google Scholar 

  7. Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. Spec. Interest Group Oper. Syst. Oper. Syst. Rev. 44(2), 35–40 (2010)

    Google Scholar 

  8. George, L.: HBase - The Definitive Guide. O’Reilly, Newton (2011)

    Google Scholar 

  9. Cooper, B.F., et al.: PNUTS: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1(2), 1277–1288 (2008)

    Article  Google Scholar 

  10. Shi, Y., Meng, X., Zhao, J., Hu, X., Liu, B., Wang, H.: Benchmarking cloud-based data management systems. In: Proceedings of the Second International Workshop on Cloud Data Management, pp. 47–54. ACM (2010)

    Google Scholar 

  11. Ferdman, M., et al.: Clearing the clouds: a study of emerging scale-out workloads on modern hardware. In: ACM SIGARCH Computer Architecture News, vol. 40, pp. 37–48. ACM (2012)

    Google Scholar 

  12. Jia, Z., et al.: Understanding big data analytics workloads on modern processors. IEEE Trans. Parallel Distrib. Syst. 28(6), 1797–1810 (2017)

    Article  Google Scholar 

  13. Jia, Z., Wang, L., Zhan, J., Zhang, L., Luo, C.: Characterizing data analysis workloads in data centers. In: IISWC, pp. 66–76. IEEE (2013)

    Google Scholar 

  14. Huang, S., Huang, J., Dai, J., Xie, T., Huang, B.: The HiBench benchmark suite: characterization of the MapReduce-based data analysis. In: 2010 IEEE 26th International Conference on Data Engineering Workshops (ICDEW), pp. 41–51. IEEE (2010)

    Google Scholar 

  15. Gray, J.: Graysort benchmark. Sort Benchmark. http://sortbenchmark.org

  16. Luo, C., et al.: CloudRank-D: benchmarking and ranking cloud computing systems for data processing applications. Front. Comput. Sci. 6(4), 347–362 (2012)

    Article  MathSciNet  Google Scholar 

  17. Jia, Z., et al.: The implications of diverse applications and scalable data sets in benchmarking big data systems. In: Rabl, T., Poess, M., Baru, C., Jacobsen, H.-A. (eds.) WBDB -2012. LNCS, vol. 8163, pp. 44–59. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-53974-9_5

    Chapter  Google Scholar 

  18. Baru, C., Bhandarkar, M., Nambiar, R., Poess, M., Rabl, T.: Benchmarking big data systems and the bigdata top100 list. Big Data 1(1), 60–64 (2013)

    Article  Google Scholar 

  19. Dede, E., Fadika, Z., Govindaraju, M., Ramakrishnan, L.: Benchmarking MapReduce implementations under different application scenarios. Future Gener. Comput. Syst. 36, 389–399 (2014)

    Article  Google Scholar 

  20. Ming, Z., et al.: BDGS: a scalable big data generator suite in big data benchmarking. arXiv preprint arXiv:1401.5465 (2014)

  21. Pavlo, A., et al.: A comparison of approaches to large-scale data analysis. In: Special Interest Group on Management Of Data, pp. 165–178. ACM (2009)

    Google Scholar 

  22. Rizzelli, G., Maier, G., Quagliotti, M., Schiano, M., Pattavina, A.: Assessing the scalability of next-generation wavelength switched optical networks. J. Lightwave Technol. 32(12), 2263–2270 (2014)

    Article  Google Scholar 

  23. Badia, S., Martín, A.F., Principe, J.: Implementation and scalability analysis of balancing domain decomposition methods. Arch. Comput. Methods Eng. 20(3), 239–262 (2013)

    Article  MathSciNet  Google Scholar 

  24. Gunther, N., Puglia, P., Tomasette, K.: Hadoop superlinear scalability. Queue 13(5), 20 (2015)

    Article  Google Scholar 

  25. Gao, J., Pattabhiraman, P., Bai, X., Tsai, W.T.: Saas performance and scalability evaluation in clouds. In: 2011 IEEE 6th International Symposium on Service Oriented System Engineering (SOSE), pp. 61–71. IEEE (2011)

    Google Scholar 

  26. Jiang, C., Han, G., Lin, J., Jia, G., Shi, W., Wan, J.: Characteristics of co-allocated online services and batch jobs in internet data centers: a case study from alibaba cloud. IEEE Access 7, 22495–22508 (2019)

    Article  Google Scholar 

  27. Jiang, C., et al.: Energy efficiency comparison of hypervisors. Sustain. Comput.: Inf. Syst. 22, 311–321 (2019)

    Google Scholar 

  28. Jiang, C., et al.: Interdomain I/O optimization in virtualized sensor networks. Sensors 18(12), 4395 (2018)

    Article  Google Scholar 

  29. Qiu, Y., Jiang, C., Wang, Y., Ou, D., Li, Y., Wan, J.: Energy aware virtual machine scheduling in data centers. Energies 12(4), 646 (2019)

    Article  Google Scholar 

  30. Terasort. https://hadoop.apache.org/docs/current/api/org/apache/hadoop/examples/terasort/package-summary.html

  31. WordCount. https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Example:_WordCount_v1.0

  32. OMalley, O.: Terabyte sort on apache Hadoop. Yahoo, pp. 1–3, May 2008. http://sortbenchmark.org/Yahoo-Hadoop.pdf

Download references

Acknowledgement

This work is supported by Natural Science Foundation of China (No. 61472109, No. 61572163 and No. 61472112) and Key Research and Development Program of Zhejiang Province (No. 2018C01098,2019C01059 and 2019C03134). This work is also supported in part by National Science Foundation (NSF) grant CNS-1205338 and CNS-1561216, and by the Introduction of Innovative R&D team program of Guangdong Province (No. 201001D0104726115). This work is supported by Alibaba Group through Alibaba Innovative Research (AIR) Program. This work is partially supported by Visiting Scholarship of Teachers’ Professional Development Program (No. FX2018050).

Author information

Authors and Affiliations

  1. Key Laboratory of Complex Systems Modeling and Simulation, Ministry of Education, Hangzhou Dianzi University, Hangzhou, 310037, China

    Xin Zhou, Congfeng Jiang, Yeliang Qiu, Tiantian Fan & Yumei Wang

  2. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310037, China

    Xin Zhou, Congfeng Jiang, Yeliang Qiu, Tiantian Fan & Yumei Wang

  3. College of Big Data and Software Engineering, Zhejiang Wanli University, Ningbo, China

    Liangbin Zhang

  4. School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou, 310023, China

    Jian Wan

  5. Department of Computer Science, Wayne State University, Detroit, MI, 48202, USA

    Weisong Shi

Authors
  1. Xin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Congfeng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yeliang Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tiantian Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yumei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liangbin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jian Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Weisong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Congfeng Jiang .

Editor information

Editors and Affiliations

  1. Chinese Academy of Sciences, Beijing, China

    Chen Zheng

  2. Chinese Academy of Sciences, Beijing, China

    Jianfeng Zhan

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, X. et al. (2019). Scalability Evaluation of Big Data Processing Services in Clouds. In: Zheng, C., Zhan, J. (eds) Benchmarking, Measuring, and Optimizing. Bench 2018. Lecture Notes in Computer Science(), vol 11459. Springer, Cham. https://doi.org/10.1007/978-3-030-32813-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-32813-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32812-2

  • Online ISBN: 978-3-030-32813-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation