Sensor-Based Human Activity Recognition in a Multi-user Scenario

  • Liang Wang
  • Tao Gu
  • Xianping Tao
  • Jian Lu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5859)


Existing work on sensor-based activity recognition focuses mainly on single-user activities. However, in real life, activities are often performed by multiple users involving interactions between them. In this paper, we propose Coupled Hidden Markov Models (CHMMs) to recognize multi-user activities from sensor readings in a smart home environment. We develop a multimodal sensing platform and present a theoretical framework to recognize both single-user and multi-user activities. We conduct our trace collection done in a smart home, and evaluate our framework through experimental studies. Our experimental result shows that we achieve an average accuracy of 85.46% with CHMMs.


Multi-user activity recognition probabilistic model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brand, M.: Coupled hidden markov models for modeling interacting processes. Technical Report (November 1997)Google Scholar
  2. 2.
    Du, Y., Chen, F., Xu, W., Li, Y.: Recognizing interaction activities using dynamic bayesian network. In: Proc. of ICPR 2006, Hong Kong, China (August 2006)Google Scholar
  3. 3.
    Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proc. Int’l. Joint Conf. on Artificial Intelligence, San Francisco (1993)Google Scholar
  4. 4.
    Gong, S., Xiang, T.: Recognition of group activities using dynamic probabilistic networks. In: Proc. of ICCV 2003, Nice, France, October 2003, pp. 742–749 (2003)Google Scholar
  5. 5.
    Gu, T., Wu, Z., Wang, L., Tao, X., Lu, J.: Mining Emerging Patterns for Recognizing Activities of Multiple Users in Pervasive Computing. In: Proc. of the 6th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous 2009), Toronto, Canada (July 2009)Google Scholar
  6. 6.
    Huynh, T., Blanke, U., Schiele, B.: Scalable recognition of daily activities from wearable sensors. In: Hightower, J., Schiele, B., Strang, T. (eds.) LoCA 2007. LNCS, vol. 4718, pp. 50–67. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Ivanov, Y.A., Bobick, A.F.: Recognition of visual activities and interactions by stochastic parsing. IEEE Trans. Pattern Recognition and Machine Intelligence 22(8), 852–872 (2000)CrossRefGoogle Scholar
  8. 8.
    Katz, S., Ford, A.B., Moskowitz, R.W., Jackson, B.A., Jaffe, M.W.: Studies of illness in the aged. the index of adl: A standardized measure of biological and psychological function. Journal of the American Medical Association 185, 914–919 (1963)Google Scholar
  9. 9.
    Lester, J., Choudhury, T., Borriello, G.: A practical approach to recognizing physical activities. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 1–16. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Logan, B., Healey, J., Philipose, M., Munguia-Tapia, E., Intille, S.: A long-term evaluation of sensing modalities for activity recognition. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 483–500. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Oliver, N., Rosario, B., Pentland, A.: A Bayesian computer vision system for modeling human interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 831–843 (2000)CrossRefGoogle Scholar
  12. 12.
    Park, S., Trivedi, M.M.: Multi-person interaction and activity analysis: A synergistic track- and body- level analysis framework. Machine Vision and Applications: Special Issue on Novel Concepts and Challenges for the Generation of Video Surveillance Systems 18(3), 151–166 (2007)zbMATHGoogle Scholar
  13. 13.
    Philipose, M., Fishkin, K.P., Perkowitz, M., Patterson, D.J., Fox, D., Kautz, H., Hähnel, D.: Inferring activities from interactions with objects. IEEE Pervasive Computing 3, 50–57 (2004)CrossRefGoogle Scholar
  14. 14.
    Smith, E.R., Mackie, D.M.: Social Psychology. Routledge, London (1999)Google Scholar
  15. 15.
    van Kasteren, T.L.M., Noulas, A.K., Englebienne, G., Kröse, B.: Accurate activity recognition in a home setting. In: Proc. of International Conference on Ubiquitous Computing (Ubicomp 2008), Korea (September 2008)Google Scholar
  16. 16.
    Wang, S., Pentney, W., Popescu, A.M., Choudhury, T., Philipose, M.: Common sense based joint training of human activity recognizers. In: Proc. Int’l. Joint Conf. on Artificial Intelligence, Hyderabad (January 2007)Google Scholar
  17. 17.
    Wyatt, D., Choudhury, T., Bilmes, J., Kautz, H.: A privacy sensitive approach to modeling multi-person conversations. In: Proc. IJCAI, India (January 2007)Google Scholar
  18. 18.
    Yacoob, Y., Black, M.J.: Parameterized modeling and recognition of activities. In: Proc. of International Conference on Computer Vision, ICCV 1998 (1998)Google Scholar
  19. 19.
    Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I., Lathoud, G.: Modeling individual and group actions in meetings: A two-layer hmm framework. In: Proc. of CVPRW 2004, Washington, DC, USA (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Liang Wang
    • 1
  • Tao Gu
    • 2
  • Xianping Tao
    • 1
  • Jian Lu
    • 1
  1. 1.State Key Laboratory for Novel Software TechnologyNanjing University 
  2. 2.Department of Mathematics and Computer ScienceUniversity of Southern Denmark 

Personalised recommendations