Context Aware Life Pattern Prediction Using Fuzzy-State Q-Learning

  • Mohamed Ali Feki
  • Sang Wan Lee
  • Zeungnam Bien
  • Mounir Mokhtari
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4541)


In an Assistive Eenvironment (AE), explicit/obtrusive interfaces for human/computer interaction can demand exclusive user attention and, often, replacement of them with implicit ones embedded into real-world artifacts for intuitive and unobtrusive use is desirable. As a part of solution, Context Aware can be utilized to recognize current context situation from a combination of low-level sensed contexts. Assuming the current context recognized, this paper tackles the next logical step of "the prediction of future contexts". This information allows the system to know patterns and their interrelations in user behaviour, which are not apparent at the lower levels of raw sensor data. The present paper analyzes prerequisites for user-centred prediction of future context and presents an algorithm for autonomous context recognition and prediction, based on our proposed Fuzzy-State Q- Learning technique as well as on some established methods for data-based prediction.


Context aware assistive environment prediction Fuzzy-State Q-learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Orr, R.J., Abowd, G.D.: The Smart Floor: A Mechanism for Natural User Identification and Tracking. In: Proceedings of 2000 Conference on Human Factors in Computing Systems (CHI 2000), ACM Press, NY (2000)Google Scholar
  2. 2.
    Mozer, M.C.: The Neural Network House: An Environment that Adapts to its Inhavitants. In: Proc. of the American Association for Artificial Intelligence Spring Symposium on Intelligent Environments, pp. 110–114 (1998)Google Scholar
  3. 3.
    Lesser, et al.: The Intelligent Home Testbed. In: Proc. of Autonomy Control Software Workshop (January 1999)Google Scholar
  4. 4.
    House_n Living Laboratory Introduction
  5. 5.
    Das, S.K., Cook, D.J., Bhattacharya, A., Hierman, E., Lin, T.Y.: The Role of Prediction Algorithms in the MAVHome Smart Home Architecture. IEEE Wireless Communications, Special Issue Smart Homes 9(6), 77–84 (2002)Google Scholar
  6. 6.
    Nurmi, P., Martin, M., Flanagan, J.A.: Enabling Proactiveness through Context Prediction. In: Proceedings of the Workshop on Context Awareness for Proactive Systems, Helsinki (2005)Google Scholar
  7. 7.
    Petzold, J., Bagci, F., et al.: Global and Local State Context Prediction. Artificial Intelligence in Mobile Systems 2003 (AIMS 2003) in Conjunction with the Fifth International Conference on Ubiquitous Computing 2003, Seattle, USA (2003)Google Scholar
  8. 8.
    Adams, L., Hunt, L., Moore, M.: The “Aware- System” - Prototyping an Augmentative Communication Interface, presented at RESNA 2003 (2003)Google Scholar
  9. 9.
    Alm, N., Arnott, J.L., Newell, A.F.: Prediction and Conversational Momentum in an Augmentative Communication System. Communications of the ACM 35, 46–57 (1992)CrossRefGoogle Scholar
  10. 10.
    Dey, A.K.: Understanding and Using Context. Personal and Ubiquitous Computing, Special issue on Situated Interaction and Ubiquitous Computing (2001)Google Scholar
  11. 11.
    J. E. Bardram, UbiHealth 2003: The 2nd International Workshop on Ubiquitous Computing for Pervasive Healthcare Applications, Seattle, Washington, October 12, part of the UbiComp 2003 Conference
  12. 12.
    Schmidt, A.: Ubiquitous Computing - Computing in Context. Ph.D. Dissertation, Department of Computer Science, Lancaster University (November 2002)Google Scholar
  13. 13.
    Gu, T., Wang, X.H., Pung, H.K., Zhang, D.Q.: An ontology-based context model in intelligent environments. In: Proceedings of the Communication Networks and Distributed Systems Modeling and Simulation Conference (CNDS’04) (January 2004)Google Scholar
  14. 14.
    Chen,H., et al.: Intelligent Agents Meet the Semantic Web in Smart Spaces. Article, IEEE Internet Computing (November 2004) Google Scholar
  15. 15.
    Mayrhofer, R.: An Architecture for Context Prediction. PhD thesis, Johannes Kepler University of Linz, Austria (October 2004)Google Scholar
  16. 16.
    Yoichiro, M.: Modified Q-Learning Method with Fuzzy State Division and Adaptive Rewards. In: Proc. of IEEE International Conference on Fuzzy Systems, pp. 1556–1561. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  17. 17.
    Christopher, J.C.H.W., Peter, D.: Q-Learning. Machine Learning 8, 279–292 (1992)zbMATHGoogle Scholar
  18. 18.
    Bezdek, J.C.: Fuzziness vs. Probability - Again. IEEE Trans. Fuzzy Systems 2(1), 1–3 (1994)zbMATHCrossRefGoogle Scholar
  19. 19.
    B. Kosko.: The probability Mopnopoly. IEEE Transactions on Fuzzy Systems. vol. 2(1) (1994) Google Scholar
  20. 20.
    Hamid, R.B.: Fuzzy Q-learning: A New Approach for Fuzzy Dynamic Programming. IEEE World Congress on Computational Intelligence, pp. 486–491 (1994)Google Scholar
  21. 21.
    Suh, I.H., Kim, J.-H., Frank Rhee, C.-H.: Fuzzy Q-learning for Autonomous Robot Systems. In: Proc. of IEEE International Conference on Neural Network, pp. 1738–1743. IEEE Computer Society Press, Los Alamitos (1997)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Mohamed Ali Feki
    • 1
  • Sang Wan Lee
    • 2
  • Zeungnam Bien
    • 2
  • Mounir Mokhtari
    • 1
  1. 1.Institut Nationale des Télécommunications, 9 rue Charles Fourier 91011 EvryFrance
  2. 2.Korea Advanced Institute of Science and Technology, Daejeon 305-701Republic of Korea

Personalised recommendations