• Liming ChenEmail author
  • Chris D. Nugent


This chapter provides an overview on the background, basic concepts, existing approaches and methodologies, potential applications, opportunities and research trends and directions for computational behaviour analysis. It first introduces the background and context of this book, and the basic concepts and terms used in the discussion of activity recognition in the book. It then provides a high-level review on dominant approaches and methods that have been used for activity recognition in related research communities. Following this, it discusses potential application domains and particularly highlights the role and opportunities of activity recognition in ambient assisted living, which has recently been under vigorous investigation, and also serves as the main application scenario in our discussions throughout the book. Finally the chapter presents research trends and directions of this research field. This chapter is intended to provide necessary technical background and context for readers to help them best prepared for reading and understanding the book.


  1. 1.
    Weiser M (1991) The computer for the 21st century. Sci Am (1991)Google Scholar
  2. 2.
    Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video TechnolGoogle Scholar
  3. 3.
    Wren CR, Tapia EM (2006) Toward scalable activity recognition for sensor networks. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  4. 4.
    Wang Y, Cang S, Yu H (2018) A data fusion-based hybrid sensory system for older people’s daily activity and daily routine recognition. IEEE Sens J 18:6874–6888CrossRefGoogle Scholar
  5. 5.
    Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image UnderstGoogle Scholar
  6. 6.
    Cédras C, Shah M (1995) Motion-based recognition a survey. Image Vis ComputGoogle Scholar
  7. 7.
    Gavrila DM (1999) The visual analysis of human movement: a survey. Comput Vis Image UnderstGoogle Scholar
  8. 8.
    Poppe R (2010) A survey on vision-based human action recognition. Image Vis ComputGoogle Scholar
  9. 9.
    Moeslund TB, Hilton A, Krüger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104(2–3):90–126CrossRefGoogle Scholar
  10. 10.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput SurvGoogle Scholar
  11. 11.
    Weinland D, Ronfard R, Boyer E (2011) A survey of vision-based methods for action representation, segmentation and recognition. Comput Vis Image UnderstGoogle Scholar
  12. 12.
    Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput SurvGoogle Scholar
  13. 13.
  14. 14.
    Helal S, Mann W, El-Zabadani H, King J, Kaddoura Y, Jansen E (2005) The gator tech smart house: a programmable pervasive space. Computer (Long Beach Calif) 38:50–60CrossRefGoogle Scholar
  15. 15.
    Design I Inhaus design.
  16. 16.
    Georgia Institute of Technology: Aware Home Research Initiative.
  17. 17.
  18. 18.
  19. 19.
    Lin W, Xing S, Nan J, Wenyuan L, Binbin L (2018) Concurrent recognition of cross-scale activities via sensorless sensing. IEEE Sens J 19(2):658–669CrossRefGoogle Scholar
  20. 20.
    De-La-Hoz-Franco E, Ariza-Colpas P, Quero JM, Espinilla M (2018) Sensor-based datasets for human activity recognition – a systematic review of literature. IEEE Access 6:59192–59210CrossRefGoogle Scholar
  21. 21.
    Nef T, Urwyler P, Büchler M, Tarnanas I, Stucki R, Cazzoli D, Müri R, Mosimann U (2015) Evaluation of three state-of-the-art classifiers for recognition of activities of daily living from smart home ambient data. Sensors (Basel) 15:11725–11740CrossRefGoogle Scholar
  22. 22.
    Liu J, Shahroudy A, Xu D, Kot Chichung A, Wang G (2017) Skeleton-based action recognition using spatio-temporal LSTM network with trust gates. IEEE Trans Pattern Anal Mach Intell XX:1–14Google Scholar
  23. 23.
    Liao L, Fox D, Kautz H (2007) Extracting places and activities from GPS traces using hierarchical conditional random fields. Int J Rob ResGoogle Scholar
  24. 24.
    Huỳnh T, Schiele B (2006) Unsupervised discovery of structure in activity data using multiple eigenspaces. In: Hazas M, Krumm J, Strang T (eds) Location- and context-awareness. Springer, Berlin, pp 151–167CrossRefGoogle Scholar
  25. 25.
    Kautz HA (2014) A formal theory of plan recognition and its implementation. In: Reasoning about plansGoogle Scholar
  26. 26.
    Wobcke W (2002) Two logical theories of plan recognition. J Log ComputGoogle Scholar
  27. 27.
    Bouchard B, Giroux S, Bouzouane A (2006) A smart home agent for plan recognition of cognitively-impaired patients. J ComputGoogle Scholar
  28. 28.
    Chen L, Nugent C, Mulvenna M, Finlay D, Hong X, Poland M (2008) A logical framework for behaviour reasoning and assistance in a smart home. Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  29. 29.
    Triboan D, Chen L, Chen F, Wang Z (2017) Semantic segmentation of real-time sensor data stream for complex activity recognition. Pers Ubiquitous ComputGoogle Scholar
  30. 30.
    Wu T, Lian C, Hsu JYY (2007) Joint recognition of multiple concurrent activities using factorial conditional random fields. In: Proceedings 22nd conferences on artificial intelligenceGoogle Scholar
  31. 31.
    Modayil J, Bai T, Kautz H (2008) Improving the recognition of interleaved activities. In: Proceedings of the 10th international conference on Ubiquitous computing - UbiComp ’08Google Scholar
  32. 32.
    Gu T, Wu Z, Tao X, Pung HK, Lu J (2009) epSICAR: an emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In: 7th annual IEEE international conference on pervasive computing and communications, PerCom 2009Google Scholar
  33. 33.
    Gong S, Xiang T (2003) Recognition of group activities using dynamic probabilistic networks. In: Proceedings ninth IEEE international conference on computer visionGoogle Scholar
  34. 34.
    Nguyen N, Venkatesh S, Bui H (2006) Recognising behaviours of multiple people with hierarchial probabilistic model and statistical data association. In: British machine vision conferenceGoogle Scholar
  35. 35.
    Oliver N, Rosario B, Pentland A (1999) A Bayesian computer vision system for modeling human interactions. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  36. 36.
    Choudhury T, Basu S (2005) Modeling conversational dynamics as a mixed-memory markov process. Adv Neural Inf Process Syst 17Google Scholar
  37. 37.
    Oliver N, Garg A, Horvitz E (2004) Layered representations for learning and inferring office activity from multiple sensory channels. In: Computer vision and image understandingGoogle Scholar
  38. 38.
    Wyatt D, Choudhury T, Bilmes J, Kautz H (2007) A privacy-sensitive approach to modeling multi-person conversations. In: IJCAI international joint conference on artificial intelligenceGoogle Scholar
  39. 39.
    Youtian D, Feng C, Wenli X, Yongbin L (2006) Recognizing interaction activities using dynamic Bayesian network. In: Proceedings - international conference on pattern recognitionGoogle Scholar
  40. 40.
    Lian C-C, Hsu JY-J (2008) Chatting activity recognition in social occasions using factorial conditional random fields with iterative classification. In: Proceedings of the 23rd national conference on artificial intelligence, vol 3. AAAI Press, pp 1814–1815Google Scholar
  41. 41.
    Lin ZH, Fu LC (2007) Multi-user preference model and service provision in a smart home environment. In: Proceedings of the 3rd IEEE international conference on automation science and engineering, IEEE CASE 2007Google Scholar
  42. 42.
    Wang L, Gu T, Tao X, Lu J (2008) Sensor-based human activity recognition in a multi-user scenario. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  43. 43.
    Singla G, Cook DJ, Schmitter-Edgecombe M (2010) Recognizing independent and joint activities among multiple residents in smart environments. J Ambient Intell Humaniz ComputGoogle Scholar
  44. 44.
    Hoey J, Grze M (2011) Distributed control of situated assistance in large domains with many tasks. In: Twenty-first international conference on automated planning and scheduling (2011)Google Scholar
  45. 45.
    Nugent C, Finlay D, Davies R, Wang H, Zheng H, Hallberg J, Synnes K, Mulvenna M (2007) homeML ? An open standard for the exchange of data within smart environments. In: Pervasive computing for quality of life enhancementGoogle Scholar
  46. 46.
    Chen L, Nugent C, Al-Bashrawi A (2009) Semantic data management for situation-aware assistance in ambient assisted living. In: Proceedings of the 11th international conference on information integration and web-based applications & services - iiWAS ’09Google Scholar
  47. 47.
    Biswas J, Baumgarten M, Tolstikov A, Wai AAP, Nugent C, Chen L, Donnelly M (2010) Requirements for the deployment of sensor based recognition systems for ambient assistive living. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  48. 48.
    Chen L, Nugent C (2009) Ontology-based activity recognition in intelligent pervasive environments. Int J Web Inf SystGoogle Scholar
  49. 49.
    Okeyo G, Chen L, Wang H, Sterritt R (2011) Ontology-based learning framework for activity assistance in an adaptive smart home. In: Chen L, Nugent CD, Biswas J, Hoey J (eds) Activity recognition in pervasive intelligent environments. Atlantis Press, Paris, pp 237–263CrossRefGoogle Scholar
  50. 50.
    Modayil J, Levinson R, Harman C (2008) Integrating sensing and cueing for more effective activity reminders. In: AAAI fall symposium: AI in eldercare: new solutions to old problemsGoogle Scholar
  51. 51.
    Pollack ME, Brown L, Colbry D, McCarthy CE, Orosz C, Peintner B, Ramakrishnan S, Tsamardinos I (2003) Autominder: an intelligent cognitive orthotic system for people with memory impairment. In: Robotics and autonomous systemsCrossRefGoogle Scholar
  52. 52.
    Kan P, Huq R, Hoey J, Goetschalckx R, Mihailidis A (2011) The development of an adaptive upper-limb stroke rehabilitation robotic system. J Neuroeng Rehabil 8:33CrossRefGoogle Scholar
  53. 53.
    Hoey J, Poupart P, von Bertoldi A, Craig T, Boutilier C, Mihailidis A (2010) Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process. Comput Vis Image UnderstGoogle Scholar
  54. 54.
    Hoey J, Pltz T, Jackson D, Monk A, Pham C, Olivier P (2011) Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive Mob ComputGoogle Scholar
  55. 55.
    Latfi F, Lefebvre B, Descheneaux C (2007) Ontology-based management of the telehealth smart home, dedicated to elderly in loss of cognitive autonomy. In: CEUR workshop proceedings (2007)Google Scholar
  56. 56.
    Klein M, Schmidt A, Lauer R (2007) Ontology-centred design of an ambient middleware for assisted living: the case of SOPRANO. Context (2007)Google Scholar
  57. 57.
    Chen L, Nugent C, Mulvenna M, Finlay D, Hong X (2009) Semantic smart homes: towards knowledge rich assisted living environments. Stud Comput IntellGoogle Scholar
  58. 58.
    Patel S, Lorincz K, Hughes R, Huggins N, Growdon J, Standaert D, Akay M, Dy J, Welsh M, Bonato P (2009) Monitoring motor fluctuations in patients with parkinsons disease using wearable sensors. IEEE Trans Inf Technol Biomed (2009)Google Scholar
  59. 59.
    Fogarty J, Au C, Hudson SE (2006) Sensing from the basement: a feasibility study of unobtrusive and low-cost home activity recognition. In: Proceedings of the annual ACM symposium on user interface software and technologyGoogle Scholar
  60. 60.
    Patel SN, Reynolds MS, Abowd GD (2008) Detecting human movement by differential air pressure sensing in HVAC system ductwork: an exploration in infrastructure mediated sensing. In: Lecture notes in computer science (including subseries Lecture notes in artificial intelligence and lecture notes in bioinformatics)Google Scholar
  61. 61.
    Mei L, Easterbrook S (2007) Evaluating user-centric adaptation with goal models. In: Proceedings - ICSE 2007 workshops: first international workshop on software engineering for pervasive computing applications, systems, and environments, SEPCASE’07Google Scholar
  62. 62.
    Chen L, Rashidi P (2012) Situation, activity and goal awareness in ubiquitous computing. Int J Pervasive Comput Commun 8:216–224CrossRefGoogle Scholar
  63. 63.
    Yin J, Yang Q, Pan JJ (2008) Sensor-based abnormal human-activity detection. IEEE Trans Knowl Data EngGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and InformaticsDe Montfort UniversityLeicesterUK
  2. 2.School of ComputingUlster UniversityBelfastUK

Personalised recommendations