Abstract
Kafka is a massively scalable publisher-subscriber architecture. Using the features of Kafka this research paper accounts to create an architecture composed of subscriber-publisher environment incorporated with streaming data analysis application like Kafka in order to grab information from IOT based devices and provide summarized data along with prediction to the designated consumers. Through the use of Kafka, streaming huge volumes of data can be accomplished within seconds. In this research paper, Kafka’s architecture have been integrated with machine learning algorithms to generate predictions on a variety of use cases which can be advantageous for the society. These algorithms have been examined on the basis of time to calculate speed of data production thereby providing an idea as to which algorithm is advantageous for anticipating the power consumption of a household.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apache Kafka. https://kafka.apache.org/
Zookeeper. http://zookeeper.apache.org/
Postscapes. https://www.postscapes.com/diy-apache-kafka/
Instructable. https://www.instructables.com/id/Subcriber-Publisher Environment/
TheVerge. https://www.theverge.com/ad/17604188/Big_data_analysis
Confluent. https://www.confluent.io/blog/using-apache-kafka-drive-cutting-edge-machine-learning
Towards Data Science. https://towardsdatascience.com/getting-started-with-apache-kafka-in-python-604b3250aa05
InfoQ. https://www.infoq.com/articles/traffic-data-monitoring-iot-kafka-and-spark-streaming/
Karapanagiotidis, P.: Literature review of modern times series forecasting methods. Int. J. Forecasting 27(4), 11–79 (2012)
Towards Data Science. https://towardsdatascience.com/putting-ml-in-production-using-apache-kafka-in-python-ce06b3a395c8
Rajkumar, L.R., Gagliardi, M.: The real-time publisher/subscriber inter-process communication model for distributed real-time systems: design and implementation. In: Proceedings of the IEEE Real-Time Technology and Applications Symposium (1995)
Fusco, G., Colombaroni, C., Comelli, L., Isaenko, N.: Short-term traffic predictions on large urban traffic networks: applications of network-based machine learning models and dynamic traffic assignment models. In: Proceedings of the 2015 IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS), Budapest, Hungary, 3–5 June 2015, pp. 93–101 (2015)
Machine Learning Mastery. https://machinelearningmastery.com/naive-methods-for-forecasting-household-electricity-consumption/
Albanese, D., Merler, G.S., Jurman, Visintainer, R.: MLPy: high-performance python package for predictive modeling. In: NIPS, MLOSS Workshop (2008)
Research Gate. https://www.researchgate.net/publication 330858371_Adaptive_Traffic_Management_System_Using_IoT_and_ Machine_Learning
Schaul, T.: PyBrain. J. Mach. Learn. Res. 743–746 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Talukdar, A. (2021). Analysis of Streaming Information Using Subscriber-Publisher Architecture. In: Singh, M., Kang, DK., Lee, JH., Tiwary, U.S., Singh, D., Chung, WY. (eds) Intelligent Human Computer Interaction. IHCI 2020. Lecture Notes in Computer Science(), vol 12615. Springer, Cham. https://doi.org/10.1007/978-3-030-68449-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-68449-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68448-8
Online ISBN: 978-3-030-68449-5
eBook Packages: Computer ScienceComputer Science (R0)