Action Prediction in Smart Home Based on Reinforcement Learning

  • Marwa HassanEmail author
  • Mirna AtiehEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8456)


This paper presents an “intelligent” environment that can be occupied by an elderly or handicapped person. It is characterized by its online learning and continuous adaptation based on a new algorithm called “Planning Q-learning Algorithm (PQLA)”. The user can make feedback promptly which simulates an algorithm that reconfigures the existing plans. The software adaptation is run under middleware “WCOMP” based on the aspect of assembly concept to adapt to the environmental changes.


Ambient computing Intelligent environment Online learning Reinforcement learning Software adaptation 


  1. 1.
    Augusto, J.C., Nugent, C.D. (eds.): Designing Smart Homes. LNCS (LNAI), vol. 4008. Springer, Heidelberg (2006)Google Scholar
  2. 2.
    Abowd, G., Mynatt, E.D.: Designing for human experience in smart environments. In: Cook, D.J., Das, S.K. (eds.) Smart Environments: Technology, Protocols and Applications, pp. 153–174. Willey, New York (2005)Google Scholar
  3. 3.
    Tigli, J.Y., Riveill, M., Rey, G., Lavirotte, S., Hourdin, V., Cheung-Foo-Woo, D., Callegari, E.: A middleware for ubiquitous computing: Wcomp. Research Report, Sophia Antipolis University – I3S Laboratory, Nice (2008)Google Scholar
  4. 4.
    Jara, A.J., Zamora, M.A., Skarmeta, A.F.: An architecture for ambient assisted living and health environments. In: Omatu, S., Rocha, M.P., Bravo, J., Fernández, F., Corchado, E., Bustillo, A., Corchado, J.M. (eds.) IWANN 2009, Part II. LNCS, vol. 5518, pp. 882–889. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Fontaine, E.: Programmation d’espace intelligent par l’utilisateur final. Ph.D. thesis, Brenoble University - Informatic Laboratory, France (2012)Google Scholar
  6. 6.
    Baldoni, R., Di Ciccio, C., Mecella, M., Patrizi, F., Querzoni, L., Santucci, G.: An embedded middleware platform for pervasive and immersive environments for-All. IEEE, Rome (2009)Google Scholar
  7. 7.
    Davis, G., Wiratunga, N., Taylor, B., Craw, S.: Matching SMARTHOUSE technology to needs of the elderly and disabled. In: ICCBR 03, Norway, pp. 29–36 (2004)Google Scholar
  8. 8.
    Dix, A., Finlay, J., Abowd, G., Beale, R.: Human–Computer Interaction, 3rd edn. Pearson/Prentice Hall, Harlow (2004)Google Scholar
  9. 9.
    Jakkula, V.R., Youngblood, G.M., Cook, D.J.: Identification of lifestyle behavior patterns with prediction of the happiness of an inhabitant in a smart home. AAAI, Boston (2006)Google Scholar
  10. 10.
    Sajal, K., Diane, J., Bhattacharya, A., Heierman, E., Tze-Yun, L.: The role of prediction algorithms in the MavHome smart home architecture. IEEE Wireless Commun. 9(6), 77–84 (2002)CrossRefGoogle Scholar
  11. 11.
    Rashidi, P., Cook, D.: An adaptive sensor mining model for pervasive computing applications. ACM, Cyprus (2004)Google Scholar
  12. 12.
    Wang, Q., Cheng, L.: AwareWare: an adaptation middleware for heterogeneous environments. In: IEEE International Conference on Communications, Paris (2004)Google Scholar
  13. 13.
    Russell, S., Norvig, J.: Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River (2003)Google Scholar
  14. 14.
    Hassan, M., Mougharbel, I., Meskawi, N., Tigli, J.-Y., Riveill, M.: Design considerations for assistive platforms in ambient computing for disabled people - wheelchair in an ambient environment. In: Abdulrazak, B., Giroux, S., Bouchard, B., Pigot, H., Mokhtari, M. (eds.) ICOST 2011. LNCS, vol. 6719, pp. 246–250. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Snoeck, N., Kranenburg, H., Eertink, H.: Plan recognition in smart environments. In: IEEE ICDIM 07, Lyon, France (2007)Google Scholar
  16. 16.
    Boger, J., Hoey, J., Poupart, P., Boutilier, C., Fernie, G., Mihailidis, A.: A planning system based on Markov decision processes to guide people with dementia through activities of daily living. IEEE Inf. Technol. Biomed. 10(2), 323–333 (2006)CrossRefGoogle Scholar
  17. 17.
    Szepesvari, C.: Reinforcement Learning Algorithms for MDPs. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool, San Rafael (2009)Google Scholar
  18. 18.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction Adaptive Computation and Machine Learning. MIT Press, Cambridge (1998)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Faculty of Economic Sciences and Business AdministrationLebanese UniversityHadatLebanon

Personalised recommendations