Abstract
In such continuously changing era when large chunks of data are generated at every moment, data analysis is performed for business predictions. The processing of such data is very difficult to be handled in serialized manner. To avoid such constraint, we opt for parallel processing. The term big data refers to the large and complex data chunks which cannot be processed using day-to-day processing software because of their limitations. And also, in the existing environment where the big data are processed, the system is controlled by an admin, i.e., the processor is not automated. In this system, we propose to develop an automated engine that will receive the dataset and the requirements for the output as input from the user, and the engine will process that chunk of data without involvement of an admin according to the need of the user and the output will be generated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Ithiel de Sola, P.O.O.L.: Technologies of Freedom. Harvard University Press (1983)
Morris, R.J., Truskowski, B.J.: The evolution of storage systems. IBM Syst. J. 42(2), 205–217 (2003)
Frank, E., Hall, M., Trigg, L., Holmes, G., Witten, I.H.: Data mining in bioinformatics using Weka. Bioinformatics 20(15), 2479–2481 (2004)
Eldawy, A., Mokbel, M.F.: Spatialhadoop: A mapreduce framework for spatial data. In: 2015 IEEE 31st International Conference on Data Engineering (ICDE), pp. 1352–1363. IEEE (2015)
Patil, T.R., Sherekar, S.S.: Performance analysis of Naive Bayes and J48 classification algorithm for data classification. Int. J. Comput. Sci. Appl. 6(2), 256–261 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Datta, L., Mukherjee, A., Kumar, C., Swarnalatha, P. (2019). An Automated Big Data Processing Engine. In: Satapathy, S., Bhateja, V., Das, S. (eds) Smart Intelligent Computing and Applications . Smart Innovation, Systems and Technologies, vol 105. Springer, Singapore. https://doi.org/10.1007/978-981-13-1927-3_29
Download citation
DOI: https://doi.org/10.1007/978-981-13-1927-3_29
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1926-6
Online ISBN: 978-981-13-1927-3
eBook Packages: EngineeringEngineering (R0)