Sensor-Based Activity Recognition Review

  • Liming ChenEmail author
  • Chris D. Nugent


This chapter presents a comprehensive survey on the state of the art of various aspects of sensor-based activity recognition. It first examines the general rationale and distinctions of different sensor technologies for activity monitoring. Then we review the major approaches and methods associated with sensor-based activity modeling and recognition from which strengths and weaknesses of those approaches are analysed and highlighted. The survey makes a primary distinction between data-driven and knowledge-driven approaches, and uses this distinction to structure our survey.


  1. 1.
    Mozer MC (1998) The neural network house: an environment that adapts to its inhabitants. In: Proceedings of AAAI spring symposium on intelligent environmentsGoogle Scholar
  2. 2.
    Leonhardt U, Magee J (1998) Multi-sensor location tracking. In: Proceedings of the 4th annual ACM/IEEE international conference on mobile computing and networking. ACM, New York, NY, USA, pp 203–214Google Scholar
  3. 3.
    Golding AR, Lesh N (1999) Indoor navigation using a diverse set of cheap, wearable sensors. In: Third international symposium on wearable computers digest of papers, pp 29–36Google Scholar
  4. 4.
    Schmidt A, Beigl M, Gellersen HW (1999) There is more to context than location. Comput GraphGoogle Scholar
  5. 5.
    Randell C, Muller H (2000) Context awareness by analysing accelerometer data. In: Fourth international symposium on wearable computers digest of papers, pp 175–176Google Scholar
  6. 6.
    Gellersen HW, Schmidt A, Beigl M (2002) Multi-sensor context-awareness in mobile devices and smart artifacts. Mob Netw ApplGoogle Scholar
  7. 7.
    Van Laerhoven K, Aidoo Ka, Lowette S (2001) Real-time analysis of data from many sensors with neural networks. In: Proceedings of 5th international symposium on wearable computerGoogle Scholar
  8. 8.
    Foerster F, Fahrenberg J (2000) Motion pattern and posture: correctly assessed by calibrated accelerometers. Behav Res Methods Instruments ComputGoogle Scholar
  9. 9.
    Laerhoven K, Van Cakmakci O (2000) What shall we teach our pants? In: Fourth international symposium on wearable computers, digest of papers, pp 77–83Google Scholar
  10. 10.
    Lee SW, Mase K (2002) Activity and location recognition using wearable sensors. IEEE Pervasive Comput 1(3):24–32Google Scholar
  11. 11.
    Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Ferscha A, Mattern F (eds) Pervasive computing. Springer, Berlin, pp 1–17Google Scholar
  12. 12.
    Patterson DJ, Liao L, Fox D, Kautz H (2003) Inferring high-level behavior from low-level sensors. Presented at the 12 October 2003Google Scholar
  13. 13.
    Nugent CD, Mulvenna MD, Hong X, Devlin S (2009) Experiences in the development of a Smart Lab. Int J Biomed Eng Technol 2:319–331CrossRefGoogle Scholar
  14. 14.
    Chan M, Estève D, Escriba C, Campo E (2008) A review of smart homes—present state and future challenges. Comput Methods Programs Biomed 91:55–81CrossRefGoogle Scholar
  15. 15.
    Programme A, AAL programme - active assisted living programme - ageing well.
  16. 16.
    Kern N, Schiele B, Junker H, Lukowicz P, Troster G (2002) Wearable sensing to annotate meeting recordings. In: Proceedings - international symposium on wearable computers, ISWCGoogle Scholar
  17. 17.
    Lukowicz P, Ward JA, Junker H, Stäger M, Tröster G, Atrash A, Starner T (2004) Recognizing workshop activity using body worn microphones and accelerometers. Presented at the 2004Google Scholar
  18. 18.
    Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv. 43(3):16CrossRefGoogle Scholar
  19. 19.
    Ashbrook D, Starner T (2003) Using GPS to learn significant locations and predict movement across multiple users. Pers Ubiquitous Comput. 7(5):275–286CrossRefGoogle Scholar
  20. 20.
    Liao L, Patterson DJ, Fox D, Kautz H (2007) Learning and inferring transportation routines. Artif IntellGoogle Scholar
  21. 21.
    Sung M, DeVaul R, Jimenez S, Gips J, Pentland A (2004) Shiver motion and core body temperature classification for wearable soldier health monitoring systems. In: Eighth international symposium on wearable computers, 2004. ISWC 2004Google Scholar
  22. 22.
    Harm H, Amft O, Roggen D, Tröster G (2008) SMASH: a distributed sensing and processing garment for the classification of upper body postures. In: Proceedings of the 3rd international ICST conference on body area networksGoogle Scholar
  23. 23.
    Pantelopoulos A, Bourbakis NG (2010) A survey on wearable sensor-based systems for health monitoring and prognosisGoogle Scholar
  24. 24.
    Dakopoulos D, Bourbakis NG (2010) Wearable obstacle avoidance electronic travel aids for blind: a surveyGoogle Scholar
  25. 25.
    Yoo J, Cho N, Yoo H-J (2008) Analysis of body sensor network using human body as the channel. In: Proceedings of the ICST 3rd international conference on body area networks. ICST (Institute for computer sciences, social-informatics and telecommunications engineering), ICST, Brussels, Belgium, pp 13:1–13:4Google Scholar
  26. 26.
    Cooper RA, Ding D, Simpson R, Fitzgerald SG, Spaeth DM, Guo S, Koontz AM, Cooper R, Kim J, Boninger ML (2005) Virtual reality and computer-enhanced training applied to wheeled mobility: an overview of work in pittsburgh. Assist Technol 17(2):159–170CrossRefGoogle Scholar
  27. 27.
    Au LK, Wu WH, Batalin MA, Stathopoulos T, Kaiser WJ (2008) Demonstration of active guidance with SmartCane. In: 2008 international conference on information processing in sensor networks (ipsn 2008), pp 537–538Google Scholar
  28. 28.
    Kim J, He J, Lyons K, Starner T (2007) The gesture watch: a wireless contact-free gesture based wrist interface. In: Proceedings - international symposium on wearable computers, ISWCGoogle Scholar
  29. 29.
    Madan A, Caneel R (2004) Towards socially-intelligent wearable networksGoogle Scholar
  30. 30.
    Wang Q, Timmermans A, Chen W, Jia J, Ding L, Xiong L, Rong J, Markopoulos P (2018) Stroke patients’ acceptance of a smart garment for supporting upper extremity rehabilitation. IEEE J Transl Eng Heal Med 6:1–9Google Scholar
  31. 31.
    Wilson D, Atkeson C (2005) Simultaneous tracking and activity recognition (STAR) using many anonymous, binary sensors. In: Proceedings of the third international conference on pervasive computing, (PERVASIVE2005)Google Scholar
  32. 32.
    Wren CR, Tapia EM (2006) Toward scalable activity recognition for sensor networks. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  33. 33.
    Srivastava MB, Muntz R, Potkonjak M (2001) Smart kindergarten: sensor-based wireless networks for smart developmental problem-solving enviroments. In: Proceedings of the 7th annual international conference on mobile computing and networking - MobiCom ’01 (2001)Google Scholar
  34. 34.
    Hollosi D, Schröder J, Goetze S, Appell JE (2010) Voice activity detection driven acoustic event classification for monitoring in smart homes. In: 2010 3rd international symposium on applied sciences in biomedical and communication technologies, ISABEL 2010Google Scholar
  35. 35.
    Aipperspach R, Cohen E, Canny J (2006) Modeling human behavior from simple sensors in the home. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  36. 36.
    Philipose M, Fishkin KP, Perkowitz M, Patterson DJ, Fox D, Kautz H, Hähnel D (2004) Inferring activities from interactions with objectsGoogle Scholar
  37. 37.
    Fishkin KP, Philipose M, Rea A (2005) Hands-on RFID: wireless wearables for detecting use of objects. In: Proceedings - international symposium on wearable computers, ISWCGoogle Scholar
  38. 38.
    Patterson DJ, Fox D, Kautz H, Philipose M (2005) Fine-grained activity recognition by aggregating abstract object usage. In: Proceedings - international symposium on wearable computers, ISWCGoogle Scholar
  39. 39.
    Hodges MR, Pollack ME (2007) An ‘object-use fingerprint’: the use of electronic sensors for human identification. In: Proceedings of international conference on ubiquitous computing (UbiComp ’07)Google Scholar
  40. 40.
    Buettner M, Prasad R, Philipose M, Wetherall D (2009) Recognizing daily activities with RFID-based sensors. In: Proceedings of the 11th international conference on ubiquitous computing - Ubicomp ’09Google Scholar
  41. 41.
    Gu T, Wu Z, Tao X, Pung HK, Lu J (2009) epSICAR: an emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In: 7th annual IEEE international conference on pervasive computing and communications, PerCom 2009Google Scholar
  42. 42.
    Quinn JA, Williams CKI, McIntosh N (2009) Factorial switching linear dynamical systems applied to physiological condition monitoring. IEEE Trans Pattern Anal Mach IntellGoogle Scholar
  43. 43.
    Horvitz EJ, Breese JS, Heckerman D, Hovel D, Rommelse K (2013) The Lumiere project: Bayesian user modeling for inferring the goals and needs of software usersGoogle Scholar
  44. 44.
    Kautz H, Fox D, Etzioni O, Borriello G, Arnstein L (2002) An overview of the assisted cognition project. In: Proceedings of AAAIGoogle Scholar
  45. 45.
    Kan P, Huq R, Hoey J, Goetschalckx R, Mihailidis A (2011) The development of an adaptive upper-limb stroke rehabilitation robotic system. J Neuroeng Rehabil 8:33CrossRefGoogle Scholar
  46. 46.
    Stikic M, Schiele B (2009) Activity recognition from sparsely labeled data using multi-instance learning. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  47. 47.
    Maurer U, Rowe A, Smailagic A, Siewiorek D (2006) Location and activity recognition using eWatch: a wearable sensor platform. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)CrossRefGoogle Scholar
  48. 48.
    Brdiczka O, Crowley JL, Reignier P (2009) Learning situation models in a smart home. IEEE Trans Syst Man Cybern Part B Cybern 39(1):56–63CrossRefGoogle Scholar
  49. 49.
    Chen DT, Yang J, Wactlar H (2005) A study of detecting social interaction with sensors in a nursing home environmentGoogle Scholar
  50. 50.
    Ravi N, Mysore P, Littman ML, Dandekar N (2005) Activity recognition from accelerometer dataGoogle Scholar
  51. 51.
    Vail DL, Veloso MM, Lafferty JD (2007) Conditional random fields for activity recognition. In: Proceedings of the 6th international joint conference on autonomous agents and multiagent systems - AAMAS ’07Google Scholar
  52. 52.
    Liao L, Fox D, Kautz H (2007) Hierarchical conditional random fields for GPS-based activity recognition. In: Thrun S, Brooks R, Durrant-Whyte H (eds) Robotics research. Springer, Berlin, pp 487–506CrossRefGoogle Scholar
  53. 53.
    Hu DH, Yang Q (2008) CIGAR: concurrent & interleaving goal & activity recognition. In: AAAI conference on artificial intelligenceGoogle Scholar
  54. 54.
    Mahdaviani M, Choudhury T (2007) Fast and scalable training of semi-supervised crfs with application to activity recognition. Adv Neural InfGoogle Scholar
  55. 55.
    Guralnik V, Haigh K (2002) Learning models of human behaviour with sequential patterns. In: AAAI workshop on automation as caregiverGoogle Scholar
  56. 56.
    Modayil J, Levinson R, Harman C (2008) Integrating sensing and cueing for more effective activity reminders. In: AAAI fall symposium AI eldercare new solutions to old problemsGoogle Scholar
  57. 57.
    Oliver N, Garg A, Horvitz E (2004) Layered representations for learning and inferring office activity from multiple sensory channels. Comput Vis Image UnderstGoogle Scholar
  58. 58.
    Pentney W, Philipose M, Bilmes J (2008) Structure learning on large scale common sense statistical models of human state. In: Proceedings of the 23rd national conference on artificial intelligence, vol 3, pp 1389–1395. AAAI PressGoogle Scholar
  59. 59.
    Wu J, Osuntogun A, Choudhury T, Philipose M, Rehg JM (2007) A scalable approach to activity recognition based on object use. In: Proceedings of the IEEE international conference on computer visionGoogle Scholar
  60. 60.
    Omar F, Sinn M, Truszkowski J (2010) Comparative analysis of probabilistic models for activity recognition with an instrumented walker. In: Proceedings of the 26th conference on uncertainty in artificial intelligenceGoogle Scholar
  61. 61.
    Sánchez D, Tentori M, Favela J (2008) Activity recognition for the smart hospital. IEEE Intell SystGoogle Scholar
  62. 62.
    Wyatt D, Philipose M, Choudhury T (2005) Unsupervised activity recognition using automatically mined common sense. In: Proceedings of 20th national conference on artificial intelligenceGoogle Scholar
  63. 63.
    Perkowitz M, Philipose M, Fishkin K, Patterson DJ (2004) Mining models of human activities from the web. In: Proceedings of the 13th conference on world wide web - WWW ’04Google Scholar
  64. 64.
    Tapia EM, Choudhury T, Philipose M (2006) Building reliable activity models using hierarchical shrinkage and mined ontology. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  65. 65.
    Palmes P, Pung HK, Gu T, Xue W, Chen S (2010) Object relevance weight pattern mining for activity recognition and segmentation. Pervasive Mob Comput 6(1):43–57CrossRefGoogle Scholar
  66. 66.
    Albrecht D, Zukerman I, Nicholson A (1998) Bayesian models for keyhole plan recognition in an adventure game. User Model User-adapt InteractGoogle Scholar
  67. 67.
    Kautz Ha (1991) A formal theory of plan recognition and its implementation. Presented at the 1991Google Scholar
  68. 68.
    Wobcke W (2002) Two logical theories of plan recognition. J Log Comput 12(3):371–412MathSciNetCrossRefGoogle Scholar
  69. 69.
    Bouchard B, Giroux S, Bouzouane A (2006) A smart home agent for plan recognition of cognitively-impaired patients. J Comput 1(5):53–62Google Scholar
  70. 70.
    Shanahan M (1997) Solving the frame problem: a mathematical investigation of the common sense law of inertia. MIT PressGoogle Scholar
  71. 71.
    Chen L, Nugent C, Mulvenna M, Finlay D, Hong X, Poland M (2008) A logical framework for behaviour reasoning and assistance in a smart home. Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  72. 72.
    Chen D, Yang J, Wactlar HD (2004) Towards automatic analysis of social interaction patterns in a nursing home environment from video. In: Proceedings of the 6th ACM SIGMM international workshop on multimedia information retrieval - MIR ’04Google Scholar
  73. 73.
    Hakeem A, Shah M (2004) Ontology and taxonomy collaborated framework for meeting classification. In: Proceedings - international conference on pattern recognitionGoogle Scholar
  74. 74.
    Georis B (2004) A video interpretation platform applied to bank agency monitoring. In: Intelligent distributed surveillance systems (IDSS-04) (2004)Google Scholar
  75. 75.
    Nevatia R, Hobbs J, Bolles B (2004) An ontology for video event representation. In: IEEE computer society conference on computer vision and pattern recognition workshopsGoogle Scholar
  76. 76.
    François ARJ, Nevatia R, Hobbs J, Bolles RC (2005) VERL: an ontology framework for representing and annotating video events. IEEE Multimed 12(4):76–86CrossRefGoogle Scholar
  77. 77.
    Akdemir U, Turaga P, Chellappa R (2008) An ontology based approach for activity recognition from video. In: Proceeding of the 16th ACM international conference on multimedia - MM ’08Google Scholar
  78. 78.
    Yamada N, Sakamoto K, Kunito G, Isoda Y, Yamazaki K, Tanaka S (2007) Applying ontology and probabilistic model to human activity recognition from surrounding things. IPSJ Digit CourGoogle Scholar
  79. 79.
    Latfi F, Lefebvre B, Descheneaux C (2007) Ontology-based management of the telehealth smart home, dedicated to elderly in loss of cognitive autonomy. In: CEUR workshop proceedingsGoogle Scholar
  80. 80.
    Klein M, Schmidt A, Lauer R (2007) Ontology-centred design of an ambient middleware for assisted living: the case of SOPRANO. ContextGoogle Scholar
  81. 81.
    Chen L, Nugent C, Mulvenna M, Finlay D, Hong X (2009) Semantic smart homes: towards knowledge rich assisted living environments. Stud Comput IntellGoogle Scholar
  82. 82.
    Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24(6):961–974CrossRefGoogle Scholar
  83. 83.
    Ye J, Stevenson G, Dobson S (2011) A top-level ontology for smart environments. Pervasive Mob Comput 7(3):359–378CrossRefGoogle Scholar
  84. 84.
    Riboni D, Bettini C (2011) OWL 2 modeling and reasoning with complex human activities. Pervasive Mob Comput 7(3):379–395CrossRefGoogle Scholar
  85. 85.
    Preuveneers D, den Bergh J, Wagelaar D, Georges A, Rigole P, Clerckx T, Berbers Y, Coninx K, Jonckers V, De Bosschere K (2004) Towards an extensible context ontology for ambient intelligence. In: Markopoulos P, Eggen B, Aarts E, Crowley JL (eds) Ambient intelligence. Springer, Berlin, pp 148–159CrossRefGoogle Scholar
  86. 86.
    Cook DJ (2012) Learning setting-generalized activity models for smart spaces. IEEE Intell Syst 2010(99):1Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and InformaticsDe Montfort UniversityLeicesterUK
  2. 2.School of ComputingUlster UniversityBelfastUK

Personalised recommendations