Abstract
In the multi-sensors environment, the crucial issue is collecting data from different sources. The paper considers the problem of data gathering from different data sources in the framework of proactive intelligent decision support systems design. The aim is to provide the invariate access to heterogeneous data stored in a data warehouse for further processing. There are two types of data sources are considered in the paper: machine or sensor data and video streams. We propose an ‘on-fly’ method of heterogeneous data fusion and preprocessing toward to minimisation of execution time of queries. A proposed method is implemented in the five-layer distributed architecture of the system based on Apache Kafka and Spark Streaming technology. The main conclusion is that in case of heterogeneous data (like video and loged data) and functional requirements for query execution over these data, the distributed data preprocessing might be efficient in comparison with batch processing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arnott, D., Pervan, G.: A critical analysis of decision support systems research revisited: the rise of design science. J. Inf. Technol. 29(4), 269–293 (2014)
Bengio, Y.: Practical recommendations for gradient-based training of deep architectures. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 437–478. Springer, Heidelberg (2012). doi:10.1007/978-3-642-35289-8_26
Engel, Y., Etzion, O.: Towards proactive event-driven computing. In: Proceedings of the 5th ACM International Conference on Distributed Event-Based System, pp. 125–136 (2011)
Bemporad, A., Morari, M.: Robust model predictive control: a survey. In: Garulli, A., Tesi, A. (eds.) Robustness in Identification and Control. LNCIS, vol. 245, pp. 207–226. Springer, London (1999). doi:10.1007/BFb0109870
Golubev, A., Chechetkin, I., Solnushkin, K.S., Sadovnikova, N., Parygin, D., Strategway, S.M.: Web solutions for building public transportation routes using big geodata analysis. In: 17th International Conference on Information Integration and Web-Based Applications and Services (2015)
Golubev, A., Chechetkin, I., Parygin, D., Sokolov, A., Shcherbakov, M.: Geospatial data generation and preprocessing tools for urban computing system development. Procedia Comput. Sci. 101, 217–226 (2016)
De Gooijer, J.G., Hyndman, R.J.: 25 years of time series forecasting. Int. J. Forecast. 22(3), 443–473 (2006)
HIPI Hadoop image processing interface. http://hipi.cs.virginia.edu/. Accessed 5 May 2017
Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines). https://engineering.linkedin.com/kafka/. Accessed 5 May 2017
Maarala, A.I., Rautiainen, M., Salmi, M., Pirttikangas, S., Riekki, J.: Low latency analytics for streaming traffic data with Apache Spark. In: 2015 IEEE International Conference on Big Data, pp. 2855–2858 (2015)
Sage, A.: Decision support systems. http://onlinelibrary.wiley.com/doi/10.1002/9780470172339.ch4/summary. Accessed
Tennenhouse, D.: Proactive computing. Commun. ACM 43(5), 43–50 (2000)
Tran V.P., Shcherbakov M., Nguyen T.A., Skorobogatchenko D.A.: A method for data acquisition and data fusion in intelligent proactive decision support systems. Neurocomputers (11), 40–44 (2016). (in Russian)
Tran, V.P., Shcherbakov, M., Nguyen, T.: A framework for event generator in proactive system design. In: 2016 7th International Conference on Information, Intelligence, Systems Applications (IISA), pp. 1–5 (2016)
Qin, S.J., Badgwell, T.: A survey of industrial model predictive control technology. Control Eng. Pract. 11(7), 733–764 (2003)
Acknowledgments
The reported study was partially supported by RFBR research projects 16-37-60066 mol_a_dk, and project MD-6964.2016.9.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Tran, V.P., Shcherbakov, M., Nguyen, T.A. (2017). Yet Another Method for Heterogeneous Data Fusion and Preprocessing in Proactive Decision Support Systems: Distributed Architecture Approach. In: Vishnevskiy, V., Samouylov, K., Kozyrev, D. (eds) Distributed Computer and Communication Networks. DCCN 2017. Communications in Computer and Information Science, vol 700. Springer, Cham. https://doi.org/10.1007/978-3-319-66836-9_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-66836-9_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66835-2
Online ISBN: 978-3-319-66836-9
eBook Packages: Computer ScienceComputer Science (R0)