Skip to main content
SpringerLink
Log in

Design of a Deep Q-Network Based Simulation System for Actuation Decision in Ambient Intelligence

  • Conference paper
  • First Online:
Web, Artificial Intelligence and Network Applications (WAINA 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 927))

Abstract

Ambient Intelligence (AmI) deals with a new world of ubiquitous computing devices, where physical environments interact intelligently and unobtrusively with people. AmI environments can be diverse, such as homes, offices, meeting rooms, schools, hospitals, control centers, vehicles, tourist attractions, stores, sports facilities, and music devices. This paper presents design and implementation of a simulation system based on Deep Q-Network (DQN) for actuation decision in AmI. DQN is a deep neural network structure used for estimation of Q-value of the Q-learning method. We implemented the proposed simulating system by Rust programming language. We describe the design and implementation of the simulation system, and show some simulation results to evaluate its performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lindwer, M., Marculescu, D., Basten, T., Zimmermann, R., Marculescu, R., Jung, S., Cantatore, E.: Ambient intelligence visions and achievements: linking abstract ideas to real-world concepts. In: Design, Automation and Test in Europe Conference and Exhibition, pp. 10–15 (2003)

    Google Scholar 

  2. Obukata, R., Cuka, M., Elmazi, D., Oda, T., Ikeda, M., Barolli, L.: Design and evaluation of an ambient intelligence testbed for improving quality of life. Int. J. Space-Based Situated Comput. 7(1), 8–15 (2017)

    Article  Google Scholar 

  3. Gabel, O., Litz, L., Reif, M.: NCS testbed for ambient intelligence. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 1, pp. 115–120 (2005)

    Google Scholar 

  4. del Campo, I., Martinez, M.V., Echanobe, J., Basterretxea, K.: A hardware/software embedded agent for realtime control of ambient-intelligence environments. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–8 (2012)

    Google Scholar 

  5. Virtex 5 Family Overview, Xilinx Inc., San Jose, CA (2009)

    Google Scholar 

  6. Bernardos, A.M., Tarrio, P., Casar, J.R.: CASanDRA: a framework to provide context acquisition services and reasoning algorithms for ambient intelligence applications. In: International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 372–377 (2009)

    Google Scholar 

  7. Costanzo, C., Loscri, V., Natalizio, E., Razafindralambo, T.: Nodes self-deployment for coverage maximization in mobile robot networks using an evolving neural network. Comput. Commun. 35(9), 1047–1055 (2012)

    Article  Google Scholar 

  8. Li, Y., Li, H., Wang, Y.: Neural-based control of a mobile robot: a test model for merging biological intelligence into mechanical system. In: IEEE 7th Joint International Information Technology and Artificial Intelligence Conference (ITAIC-2014), pp. 186–190 (2014)

    Google Scholar 

  9. Oda, T., Obukata, R., Ikeda, M., Barolli, L., Takizawa, M.: Design and implementation of a simulation system based on deep Q-network for mobile actor node control in wireless sensor and actor networks. In: The 31th IEEE International Conference on Advanced Information Networking and Applications Workshops (IEEE WAINA-2017) (2017)

    Google Scholar 

  10. Oda, T., Elmazi, D., Cuka, M., Kulla, E., Ikeda, M., Barolli, L.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering three-dimensional environment. In: The 9th International Conference on Intelligent Networking and Collaborative Systems (INCoS-2017), pp. 41–52 (2017)

    Google Scholar 

  11. Acampora, G., Cook, D., Rashidi, P., Vasilakos, A.V.: A survey on ambient intelligence in health care. Proc. IEEE 101(12), 2470–2494 (2013)

    Article  Google Scholar 

  12. Aarts, E., Wichert, R.: Ambient Intelligence. Technology Guide, pp. 244–249 (2009)

    Google Scholar 

  13. Aarts, E., de Ruyter, B.: New research perspectives on ambient intelligence. J. Ambient Intell. Smart Environ. 1(1), 5–14 (2009)

    Google Scholar 

  14. Vasilakos, A., Pedrycz, W.: Ambient Intelligence, Wireless Networking, and Ubiquitous Computing. Artech House, Norwood (2006)

    Google Scholar 

  15. Sadri, F.: Ambient intelligence: a survey. ACM Comput. Surv. 43(4), 36:1–36:66 (2011)

    Article  Google Scholar 

  16. Hagras, H., Callaghan, V., Colley, M., Clarke, G., Pounds-Cornish, A., Duman, H.: Creating an ambient-intelligence environment using embedded agents. IEEE Intell. Syst. 19(6), 12–20 (2004)

    Article  Google Scholar 

  17. Ramos, C., Augusto, J.C., Shapiro, D.: Ambient intelligence-the next step for artificial intelligence. IEEE Intell. Syst. 23(2), 15–18 (2008)

    Article  Google Scholar 

  18. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)

    Article  Google Scholar 

  19. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv:1312.5602v1, pp. 1–9 (2013)

  20. Oda, T., Kulla, E., Cuka, M., Elmazi, D., Ikeda, M., Barolli, L.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering different distributions of events. In: The 11th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2017), pp. 36–49 (2017)

    Google Scholar 

  21. Oda, T., Kulla, E., Katayama, K., Ikeda, M., Barolli, L.: A deep Q-network based simulation system for actor node mobility control in WSANs considering three-dimensional environment: a comparison study for normal and uniform distributions. In: The 12th International Conference on Complex, Intelligent, and Software Intensive Systems, pp. 842–852 (2018)

    Google Scholar 

  22. Lei, T., Ming, L.: A robot exploration strategy based on Q-learning network. In: IEEE International Conference on Real-Time Computing and Robotics (RCAR-2016), pp. 57–62 (2016)

    Google Scholar 

  23. Riedmiller, M.: Neural fitted Q iteration - first experiences with a data efficient neural reinforcement learning method. In: The 16th European Conference on Machine Learning (ECML-2005), vol. 3720 of the series Lecture Notes in Computer Science, pp. 317–328 (2005)

    Google Scholar 

  24. Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: The 2010 International Joint Conference on Neural Networks (IJCNN-2010), pp. 1–8 (2010)

    Google Scholar 

  25. Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)

    Article  MathSciNet  Google Scholar 

  26. Lin, L.J.: Reinforcement learning for robots using neural networks. Technical report, DTIC Document (1993)

    Google Scholar 

  27. The Rust Programming Language (2007). https://www.rust-lang.org/

  28. GitHub - rust-lang/rust: a safe, concurrent, practical language (2007). https://github.com/rust-lang/

  29. ‘rust’ tag wiki - Stack Overflow (2007). http://stackoverflow.com/tags/rust/info/

  30. Ohno, Y.: Color rendering and luminous efficacy of white LED spectra. In: The 4th International Conference on Solid State Lighting, pp. 88–98 (2004)

    Google Scholar 

  31. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: The 13th International Conference on Artificial Intelligence and Statistics (AISTATS-2010), pp. 249–256 (2010)

    Google Scholar 

  32. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: The 14th International Conference on Artificial Intelligence and Statistics (AISTATS-2011), pp. 315–323 (2011)

    Google Scholar 

Download references

Acknowledgement

This work was supported by Faculty of Engineering, Okayama University of Science (OUS) Grant-in-Aid for Exploratory Research.

Author information

Authors and Affiliations

  1. Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama, 700–0005, Japan

    Tetsuya Oda, Chiaki Ueda, Ryo Ozaki & Kengo Katayama

Authors
  1. Tetsuya Oda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chiaki Ueda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryo Ozaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kengo Katayama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tetsuya Oda .

Editor information

Editors and Affiliations

  1. Department of Information and Communication Engineering, Fukuoka Institute of Technology, Fukuoka, Japan

    Leonard Barolli

  2. Department of Advanced Sciences, Hosei University, Koganei-Shi, Tokyo, Japan

    Makoto Takizawa

  3. Department of Computer Science, Technical University of Catalonia, Barcelona, Barcelona, Spain

    Fatos Xhafa

  4. Faculty of Business Administration, Rissho University, Tokyo, Japan

    Tomoya Enokido

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Oda, T., Ueda, C., Ozaki, R., Katayama, K. (2019). Design of a Deep Q-Network Based Simulation System for Actuation Decision in Ambient Intelligence. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds) Web, Artificial Intelligence and Network Applications. WAINA 2019. Advances in Intelligent Systems and Computing, vol 927. Springer, Cham. https://doi.org/10.1007/978-3-030-15035-8_34

Download citation

Publish with us

Policies and ethics

Search

Navigation