Abstract
The need for low latency analysis of high velocity real time continuous data streams has led to the emergence of Stream Processing Systems (SPSs). Contemporary SPSs allow a stream processing application to be hosted on Cloud infrastructures and dynamically scaled so as to adapt to the fluctuating data rates. However, the run time scalability incorporated in these SPSs are in their early adaptations and are based on simple local/global threshold based controls. This work studies the issues with the local and global auto scaling techniques that may lead to performance inefficiencies in real time traffic analysis on Cloud platforms and presents an efficient hybrid auto scaling strategy StreamScale which addresses the identified issues. The proposed StreamScale auto-scaling algorithm accounts for the gaps in the local/global scaling approaches and effectively identifies (de)parallelization opportunities in stream processing applications for maintaining QoS at reduced costs. Simulation based experimental evaluation on representative stream application topologies indicate that the proposed StreamScale auto-scaling algorithm exhibits better performance in comparison to both local and global auto-scaling approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, D.J., Ahmad, Y., Balazinska, M., Çetintemel, U., Cherniack, M., Hwang, J.-H., Lindner, W., Maskey, A., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y. and Zdonik, S.B.: The design of the Borealis stream processing engine. In: CIDR, pp. 277–289 (2005)
Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., Widom, J.: STREAM: the stanford stream data manager. IEEE Data Eng. Bull. 26, 19–26 (2003)
Biem, A., Bouillet, E., Feng, H.: IBM infosphere streams for scalable, real-time, intelligent transportation services. In: SIGMOD 2010: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 1093–1103 (2010)
Toshniwal, A., Donham, J., Bhagat, N., Mittal, S., Ryaboy, D., Taneja, S., Shukla, A., Ramasamy, K., Patel, J.M., Kulkarni, S., Jackson, J., Gade, K., Fu, M.: Storm@twitter. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data - SIGMOD 2014, pp. 147–156 (2014)
Google Compute Engine. https://developers.google.com/compute/pricing
Gulisano, V., Jiménez-Peris, R., Patiño-Mart́nez, M., Soriente, C., Valduriez, P.: StreamCloud: an elastic and scalable data streaming system. IEEE Trans. Parallel Distrib. Syst. 23(12), 2351–2365 (2012)
Iosup, A., Yigitbasi, N., Epema, D.: On the performance variability of production cloud services. In: Proceedings - 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2011, pp. 104–113 (2011)
Jain, N., Amini, L., Andrade, H., King, R.: Design, implementation, and evaluation of the linear road benchmark on the stream processing core. In: Proceedings, pp. 431–442 (2006)
Kumbhare, A.G., Member, S., Simmhan, Y., Member, S.: Reactive resource provisioning heuristics for dynamic dataflows on cloud infrastructure. IEEE Trans. Cloud Comput. 3(2), 105–118 (2015)
Satzger, B., Hummer, W., Leitner, P., Dustdar, S.: Esc: towards an elastic stream computing platform for the cloud. In: Proceedings - 2011 IEEE 4th International Conference on Cloud Computing, CLOUD 2011, pp. 348–355 (2011)
Schad, J., Dittrich, J., Quiané-Ruiz, J.-A.: Runtime measurements in the cloud observing analyzing and reducing variance. Proc. VLDB Endow. 3(1–2), 460–471 (2010)
StreamBase Complex Event Processing – Overview—TIBCO: 2013. http://www.tibco.com/products/event-processing/complex-event-processing/streambase-complex-event-processing
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sahni, J., Vidyarthi, D.P. (2017). Scalable Online Analytics on Cloud Infrastructures. In: Singh, M., Gupta, P., Tyagi, V., Sharma, A., Ören, T., Grosky, W. (eds) Advances in Computing and Data Sciences. ICACDS 2016. Communications in Computer and Information Science, vol 721. Springer, Singapore. https://doi.org/10.1007/978-981-10-5427-3_43
Download citation
DOI: https://doi.org/10.1007/978-981-10-5427-3_43
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-5426-6
Online ISBN: 978-981-10-5427-3
eBook Packages: Computer ScienceComputer Science (R0)