Abstract
Many Internet-scale services are emerging and developing quickly in recent years, and they may have different performance goals. It is a challenge for a data center to deploy these services and satisfy their performance goals. Servers in a data center are usually heterogeneous, which makes it more sophisticated to efficiently schedule jobs in a cluster. This paper analyzed workload data from publicly available Google cluster traces, and explored two types of heterogeneity from these traces: machine heterogeneity and workload heterogeneity. Based on analysis results, we proposed a heterogeneity model for dynamic capacity provisioning problem in a cluster to deal with these Internet-scale services.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmad F, Chakradhar ST, Raghunathan A, Vijaykumar TN (2012) Tarazu: optimizing map reduce on heterogeneous clusters. In: Proceedings of ASPLOS 2012, 3–7 Mar 2012, London, UK
Nathuji R, Isci C, Gorbatov E (2007) Exploiting platform heterogeneity for power efficient data centers. In: Proceedings of the IEEE international conference on autonomic computing (ICAC), June 2007, Florida, USA
Chun B, Iannaccone G, Iannaccone G, Katz R, Lee G, Niccolini L (2009) An energy case for hybrid data centers. In: Proceedings of HotPower 2009, 10 Oct 2009, Big Sky, MT, USA
Chen Y, Ganapathi AS, Griffith R, Katz RH (2010) Analysis and lessons from a publicly available google cluster trace. Technical Report UCB/EECS-2010-95, 2010, UC Berkeley, USA
Kavulya S, Tan J, Gandhi R, Narasimhan P (2010) An analysis of traces from a production map reduce cluster. In: Proceedings of IEEE/ACM conference on cluster, cloud and grid computing (CCGrid), May 2010, Melbourne, Australia
Zhang Q, Hellerstein J, Boutaba R (2011) Characterizing task usage shapes in google compute clusters. In: Proceedings of LADIS, 2–3 Sept 2011, Washington, USA
Heath T, Diniz B, Carrera EV, Jr. Meira W, Bianchini R (2005) Energy conservation in heterogeneous server clusters. In: Proceedings of the tenth ACM SIGPLAN symposium on principles and practice of parallel programming, 15–17 June 2005, Chicago, USA
Krioukov A, Mohan P, Alspaugh S, Keys L, Culler D, and Katz R (2011) NAPSAC: design and implementation of a power-proportional web cluster, ACM SIGCOMM computer communication review, vol 41(1), pp 102–108
Zhan J, Wang L, Li X, Shi W, Weng C, Zhang W, Zang X (2012) Cost-aware cooperative resource provisioning for heterogeneous workloads in data centers accepted by IEEE transactions on computers, May 2012
Koller R, Verma A, Neogi A (2010) WattApp: an application aware power meter for shared data centers. In: Proceeding of the 7th international conference on autonomic computing, 07–11 June 2010, Washington, DC, USA
Googleclusterdata–google workloads (2011) http://code.google.com/p/googleclusterdata/
Zhang Q, Zhani MF, Zhang S, Zhu Q, Boutaba R, Hellerstein JL (2012) Dynamic energy-aware capacity provisioning for cloud computing environments. In: Proceedings of IEEE/ACM international conference on autonomic computing, Sept 2012, California, USA
Garg S, Sundaram S, Patel HD (2011) Robust heterogeneous data center design: a principled approach. SIGMETRICS Perf Eval Rev 39(3):28–30
Acknowledgments
Special thanks to Qi Zhang, Mohamed Faten Zhani, Prof. Boutaba in University of Waterloo for their kind help. And this work is supported by Program for Changjiang Scholars and Innovative Research Team in University No.IRT1012.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, S., Liu, Y. (2013). Analysis and Modeling of Heterogeneity from Google Cluster Traces. In: Lu, W., Cai, G., Liu, W., Xing, W. (eds) Proceedings of the 2012 International Conference on Information Technology and Software Engineering. Lecture Notes in Electrical Engineering, vol 210. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34528-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-34528-9_16
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34527-2
Online ISBN: 978-3-642-34528-9
eBook Packages: EngineeringEngineering (R0)