Abstract
Organizations often end up with wasted space when handling datasets generated as code-application logs. Every dataset be it semi-structured, unstructured is monitored and insights are driven be it predictive, prescriptive or descriptive.
Now we often replicate data to an application space for analysis and these datasets are often cause a critical problem which is not cost effective. Using this paper we try to evaluate cost effective ways of doing decentralised in-situ and in-transit data analysis with the objective of providing business impact insights.
We also discuss techniques for queue management, scenario based hypothesis for various business requirements and the approach to achieve cost effective analysis mechanisms. Based on the scenarios, we also try to bring in the importance of the in-situ techniques as data movement and storage is itself energy hungry problem when it comes to simulation and analytics.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bitton, D., et al.: One platform for mining structured and unstructured data: dream or reality? In: Proceedings of the 32nd International Conference on Very Large Data Bases. VLDB Endowment (2006)
Faerber, F., et al.: Towards a web-scale data management ecosystem demonstrated by SAP HANA. In: 2015 IEEE 31st International Conference on Data Engineering. IEEE (2015)
Fang, H.: Managing data lakes in big data era: what’s a data lake and why has it became popular in data management ecosystem. In: 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER). IEEE (2015)
Tsai, C.-W., et al.: Big data analytics: a survey. J. Big Data 2(1), 21 (2015)
Shin, D.-H.: Demystifying big data: anatomy of big data developmental process. Telecommun. Policy 40(9), 837–854 (2016)
Bennett, J.C., et al.: Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. IEEE Computer Society Press (2012)
Samek, W., Wiegand, T., Müller, K.-R.: Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296 (2017)
Laney, D.: 3D data management: controlling data volume, velocity and variety. META Group Res. Note 6(70), 1 (2001)
Gandomi, A., Haider, M.: Beyond the hype: big data concepts, methods, and analytics. Int. J. Inf. Manag. 35(2), 137–144 (2015)
Zafar, R., et al.: Big data: the NoSQL and RDBMS review. In: 2016 International Conference on Information and Communication Technology (ICICTM). IEEE (2016)
Dixon, J.: Union of the State – A Data Lake Use Case. James Dixon’s Blog, 22 January 2015. jamesdixon.wordpress.com/2015/01/22/union-of-the-state-a-data-lake-use-case/
Hai, R., Geisler, S., Quix, C.: Constance: an intelligent data lake system. In: Proceedings of the 2016 International Conference on Management of Data. ACM (2016)
Gao, Y., Huang, S., Parameswaran, A.: Navigating the data lake with datamaran: automatically extracting structure from log datasets. In: Proceedings of the 2018 International Conference on Management of Data. ACM (2018)
Brown, N., et al.: In situ data analytics for highly scalable cloud modelling on Cray machines. Concurrency Comput.: Pract. Exp. 30(1), e4331 (2018)
Hashem, I.A.T., et al.: The rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015)
Danziger, P.: Big o notation. Source internet. http://www.scs.ryerson.ca/~mth110/Handouts/PD/bigO.pdf. Accessed Apr 2010
Batini, C., et al.: From data quality to big data quality. In: Big Data: Concepts, Methodologies, Tools, and Applications, pp. 1934–1956. IGI Global (2016)
Feldman, D., Schmidt, M., Sohler, C.: Turning big data into tiny data: constant-size coresets for k-means, PCA and projective clustering. In: Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics (2013)
Madnick, S., Zhu, H.: Improving data quality through effective use of data semantics. Data Knowl. Eng. 59(2), 460–475 (2006)
Chen, H., Chiang, R.H.L., Storey, V.C.: Business intelligence and analytics: From big data to big impact. MIS Q. 36(4), 1165–1188 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Mahanta, P., Mohamed, AG. (2020). A Hybrid Approach to Insightful Business Impacts. In: Debruyne, C., et al. On the Move to Meaningful Internet Systems: OTM 2019 Workshops. OTM 2019. Lecture Notes in Computer Science(), vol 11878. Springer, Cham. https://doi.org/10.1007/978-3-030-40907-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-40907-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-40906-7
Online ISBN: 978-3-030-40907-4
eBook Packages: Computer ScienceComputer Science (R0)