Abstract
Wireless Sensor Actor Networks (WSANs) consist of wireless network nodes, with the ability to sense events (sensors) and to perform actuations (actors) based on the sensing data collected by all sensors. This paper describes a design of a simulation system based on Deep Q-Network (DQN) for actor node mobility control in WSANs. DQN is a deep neural network structure used for estimation of Q value of the Q-learning technique. The proposed simulation system is implemented in Rust programming language. We evaluate the performance of the proposed system for different distributions of event placement considering three-dimensional environment. For this scenario, the simulation results show that for normal distribution of events actor nodes are connected in the best case.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akyildiz, I.F., Kasimoglu, I.H.: Wireless sensor and actor networks: research challenges. Ad Hoc Netw. 2(4), 351–367 (2004)
Krishnakumar, S.S., Abler, R.T.: Intelligent actor mobility in wireless sensor and actor networks. In: IFIP WG 6.8 1-st International Conference on Wireless Sensor and Actor Networks (WSAN 2007), pp. 13–22 (2007)
Sir, M.Y., Senturk, I.F., Sisikoglu, E., Akkaya, K.: An optimization-based approach for connecting partitioned mobile sensor/actuator networks. In: IEEE Conference on Computer Communications Workshops, pp. 525–530 (2011)
Younis, M., Akkaya, K.: Strategies and techniques for node placement in wireless sensor networks: a survey. Ad-Hoc Netw. 6(4), 621–655 (2008)
Liu, H., Chu, X., Leung, Y.-W., Du, R.: Simple movement control algorithm for bi-connectivity in robotic sensor networks. IEEE J. Sel. Areas Commun. 28(7), 994–1005 (2010)
Abbasi, A., Younis, M., Akkaya, K.: Movement-assisted connectivity restoration in wireless sensor and actor networks. IEEE Trans. Parallel Distrib. Syst. 20(9), 1366–1379 (2009)
Akkaya, K., Senel, F., Thimmapuram, A., Uludag, S.: Distributed recovery from network partitioning in movable sensor/actor networks via controlled mobility. IEEE Trans. Comput. 59(2), 258–271 (2010)
Costanzo, C., Loscri, V., Natalizio, E., Razafindralambo, T.: Nodes self-deployment for coverage maximization in mobile robot networks using an evolving neural network. Comput. Commun. 35(9), 1047–1055 (2012)
Li, Y., Li, H., Wang, Y.: Neural-based control of a mobile robot: a test model for merging biological intelligence into mechanical system. In: IEEE 7-th Joint International Information Technology and Artificial Intelligence Conference (ITAIC 2014), pp. 186–190 (2014)
Oda, T., Obukata, R., Ikeda, M., Barolli, L., Takizawa, M.: Design and implementation of a simulation system based on deep Q-network for mobile actor node control in wireless sensor and actor networks. In: The 31-th IEEE International Conference on Advanced Information Networking and Applications Workshops (IEEE WAINA 2017) (2017)
Oda, T., Kulla, E., Cuka, M., Elmazi, D., Ikeda, M., Barolli, L.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering different distributions of events. In: The 11-th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS 2017), pp. 36–49 (2017)
Oda, T., Elmazi, D., Cuka, M., Kulla, E., Ikeda, M., Barolli, L.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering three-dimensional environment. In: The 9-th International Conference on Intelligent Networking and Collaborative Systems (INCoS 2017), pp. 41–52 (2017)
Oda, T., Kulla, E., Katayama, K., Ikeda, M., Barolli, L.: A deep Q-network based simulation system for actor node mobility control in WSANs considering three-dimensional environment: a comparison study for normal and uniform distributions. In: The 12-th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS 2018), pp. 842–852 (2018)
Llaria, A., Terrasson, G., Curea, O., Jiménez, J.: Application of wireless sensor and actuator networks to achieve intelligent microgrids: a promising approach towards a global smart grid deployment. Appl. Sci. 6(3), 61–71 (2016)
Kulla, E., Oda, T., Ikeda, M., Barolli, L.: SAMI: a sensor actor network Matlab implementation. In: The 18-th International Conference on Network-Based Information Systems (NBiS 2015), pp. 554–560 (2015)
Kruger, J., Polajnar, D., Polajnar, J.: An open simulator architecture for heterogeneous self-organizing networks. In: Canadian Conference on Electrical and Computer Engineering, (CCECE 2006), pp. 754–757 (2006)
Akbas, M., Turgut, D.: APAWSAN: actor positioning for aerial wireless sensor and actor networks. In: IEEE 36-th Conference on Local Computer Networks (LCN 2011), pp. 563–570 (2011)
Akbas, M., Brust, M., Turgut, D.: Local positioning for environmental monitoring in wireless sensor and actor networks. In: IEEE 35-th Conference on Local Computer Networks (LCN 2010), pp. 806–813 (2010)
Melodia, T., Pompili, D., Gungor, V., Akyildiz, I.: Communication and coordination in wireless sensor and actor networks. IEEE Trans. Mob. Comput. 6(10), 1126–1129 (2007)
Gungor, V., Akan, O., Akyildiz, I.: A real-time and reliable transport (RT2) protocol for wireless sensor and actor networks. IEEE/ACM Trans. Netw. 16(2), 359–370 (2008)
Mo, L., Xu, B.: Node coordination mechanism based on distributed estimation and control in wireless sensor and actuator networks. J. Control Theory Appl. 11(4), 570–578 (2013)
Selvaradjou, K., Handigol, N., Franklin, A., Murthy, C.: Energy-efficient directional routing between partitioned actors in wireless sensor and actor networks. IET Commun. 4(1), 102–115 (2010)
Kantaros, Y., Zavlanos, M.M.: Communication-aware coverage control for robotic sensor networks. In: IEEE Conference on Decision and Control, pp. 6863–6868 (2014)
Melodia, T., Pompili, D., Akyldiz, I.: Handling mobility in wireless sensor and actor networks. IEEE Trans. Mob. Comput. 9(2), 160–173 (2010)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing Atari with deep reinforcement learning, pp. 1–9. arXiv:1312.5602v1 (2013)
Lei, T., Ming, L.: A robot exploration strategy based on Q-learning network. In: IEEE International Conference on Real-Time Computing and Robotics (RCAR 2016), pp. 57–62 (2016)
Riedmiller, M.: Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In: The 16-th European Conference on Machine Learning (ECML 2005). Lecture Notes in Computer Science, vol. 3720, pp. 317–328 (2005)
Lin, L.J.: Reinforcement learning for robots using neural networks. Technical report, DTIC Document (1993)
Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: The 2010 International Joint Conference on Neural Networks (IJCNN 2010), pp. 1–8 (2010)
Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)
The Rust Programming Language. https://www.rust-lang.org/. Accessed 14 Oct 2019
GitHub - rust-lang/rust: A safe, concurrent, practical language. https://github.com/rust-lang/. Accessed 14 Oct 2019
‘rust’ tag wiki - Stack Overflow. http://stackoverflow.com/tags/rust/info/. Accessed 14 Oct 2019
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: The 13-th International Conference on Artificial Intelligence and Statistics (AISTATS 2010), pp. 249–256 (2010)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: The 14-th International Conference on Artificial Intelligence and Statistics (AISTATS 2011), pp. 315–323 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Toyoshima, K., Oda, T., Hirota, M., Katayama, K., Barolli, L. (2020). A DQN Based Mobile Actor Node Control in WSAN: Simulation Results of Different Distributions of Events Considering Three-Dimensional Environment. In: Barolli, L., Okada, Y., Amato, F. (eds) Advances in Internet, Data and Web Technologies. EIDWT 2020. Lecture Notes on Data Engineering and Communications Technologies, vol 47. Springer, Cham. https://doi.org/10.1007/978-3-030-39746-3_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-39746-3_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39745-6
Online ISBN: 978-3-030-39746-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)