Composite Activity Recognition

  • Liming ChenEmail author
  • Chris D. Nugent


Activity recognition is essential in providing activity assistance for users in smart homes. While significant progress has been made for single-user single-activity recognition, it still remains a challenge to carry out real-time progressive composite activity recognition. This Chapter introduces a hybrid approach to composite activity modelling and recognition by extending existing ontology-based knowledge-driven approach with temporal modelling and reasoning methods. It combines and describes in details ontological activity modelling which establishes relationships between activities and their involved entities, and temporal activity modelling which defines relationships between constituent activities of a composite activity, thus providing powerful representation capabilities for composite activity modelling. The Chapter describes an integrated architecture for composite activity recognition, and elaborates a unified activity recognition algorithm for the recognition of simple and composite activities. As an essential part of the model, the Chapter also presents methods for developing temporal entailment rules to support the interpretation and inference of composite activities. An example case study has been undertaken using a number of experiments to evaluate and demonstrate the proposed approach in a feature-rich multi-agent prototype system.


  1. 1.
    Modayil J, Bai T, Kautz H (2008) Improving the recognition of interleaved activities. In: Proceedings of the 10th international conference on Ubiquitous computing – UbiComp 2008Google Scholar
  2. 2.
    Patterson DJ, Fox D, Kautz H, Philipose M (2005) Fine-grained activity recognition by aggregating abstract object usage. In: Proceedings of international symposium on wearable computers, ISWCGoogle Scholar
  3. 3.
    van Kasteren T, Noulas A, Englebienne G, Kröse B (2008) Accurate activity recognition in a home setting. In: Proceedings of the 10th international conference on Ubiquitous computing - UbiComp 2008Google Scholar
  4. 4.
    Wu T, Lian C, Hsu JYY (2007) Joint recognition of multiple concurrent activities using factorial conditional random fields. In: Proceedings of 22nd conference artificial intelligenceGoogle Scholar
  5. 5.
    Hao DH, Pan SJ, Zheng VW, Liu NN, Yang Q (2008) Real world activity recognition with multiple goals. In: Proceedings of the 10th international conference on Ubiquitous computing – UbiComp 2008Google Scholar
  6. 6.
    Hu DH, Yang Q (2008) CIGAR: Concurrent and interleaving goal and activity recognition. In: AAAI conference on artificial intelligenceGoogle Scholar
  7. 7.
    Helaoui R, Niepert M, Stuckenschmidt H (2011) Recognizing interleaved and concurrent activities: A statistical-relational approach. In: 2011 IEEE international conference on pervasive computing and communications. PerCom 2011Google Scholar
  8. 8.
    Helaoui R, Niepert M, Stuckenschmidt H (2011) Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships. In: Pervasive and mobile computingGoogle Scholar
  9. 9.
    Steinhauer H, Chua S (2010) Utilising temporal information in behaviour recognition.In: AAAI Spring SymposiumGoogle Scholar
  10. 10.
    Okeyo G, Chen L, Wang H, Sterritt R (2012) A hybrid ontological and temporal approach for composite activity modelling.In: Proceedings of 11th IEEE international conference on trust, security and privacy in computing and communications trust. - 11th IEEE international conference ubiquitous computing and communication. IUCC-2012, pp. 1763–1770Google Scholar
  11. 11.
    Chen L, Nugent CD, Wang, H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data EngGoogle Scholar
  12. 12.
    Riboni D, Bettini C (2011) OWL 2 modeling and reasoning with complex human activities. Pervasive Mob, ComputCrossRefGoogle Scholar
  13. 13.
    Chen L, Nugent C (2009) Ontology-based activity recognition in intelligent pervasive environments. Int J Web Inf SystGoogle Scholar
  14. 14.
    Allen JF (2013) Maintaining knowledge about temporal intervals. In: Readings in qualitative reasoning about physical systemsGoogle Scholar
  15. 15.
    Gu T, Wang L, Wu Z, Tao X, Lu J (2011) A pattern mining approach to sensor-based human activity recognition. IEEE Trans Knowl Data EngGoogle Scholar
  16. 16.
    Saguna S, Zaslavsky A, Chakraborty D (2011) Recognizing concurrent and interleaved activities in social interactions. In: Proceedings IEEE 9th international conference on dependable, autonomic and secure computing, DASC 2011Google Scholar
  17. 17.
    Padovitz A, Loke SW, Zaslavsky A (2004) Towards a theory of context spaces. In: Proceedings second IEEE annual conference on pervasive computing and communications. Workshops, PerComGoogle Scholar
  18. 18.
    Welty C, Fikes R, Makarios S (2006) A reusable ontology for fluents in OWL. In: Formal ontology in information systems. IOS PressGoogle Scholar
  19. 19.
    Bucks RS, Ashworth DL, Wilcock GK, Siegfried K (1996) Assessment of activities of daily living in dementia: development of the bristol activities of daily living scale. age ageingGoogle Scholar
  20. 20.
    Lawton MP, Brody EM (1969) Assessment of older people: Self-maintaining and instrumental activities of daily living. GerontologistGoogle Scholar
  21. 21.
    Katz S, Downs TD, Cash HR, Grotz RC (1970) Progress in development of the index of ADL. GerontologistGoogle Scholar
  22. 22.
    Bartos A, Martínek P, Řípová D (2010) The bristol activities of daily living scale BADLS-CZ for the evaluation of patients with dementiaGoogle Scholar
  23. 23.
    Bucks RS, Haworth J (2002) Bristol activities of daily living scale: a critical evaluation. Expert Rev NeurotherGoogle Scholar
  24. 24.
    Philipose M, Fishkin KP, Perkowitz M, Patterson DJ, Fox D, Kautz H, Hähnel D (2004) Inferring activities from interactions with objectsGoogle Scholar
  25. 25.
    Riboni D, Pareschi L, Radaelli L, Bettini C (2011) Is ontology-based activity recognition really effective? In: 2011 IEEE international conference on pervasive computing and communications workshops, PERCOM Workshops 2011Google Scholar
  26. 26.
    Horrocks I (2005) OWL: A description logic based ontology language. In: Lecture notes in computer science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)zbMATHGoogle Scholar
  27. 27.
    Mann CJH (2003) The description logic handbook – theory, implementation and applications. KybernetesGoogle Scholar
  28. 28.
    Baader F, Calvanese D, McGuinness DL, Nardi D, Patel-Schneider PF (2010) The description logic handbook: theory implementation and applications. Cambridge University Press, New YorkzbMATHGoogle Scholar
  29. 29.
    Horrocks I, Patel-Schneider PF, Bechhofer S, Tsarkov D (2005) OWL rules: A proposal and prototype implementation. Web SemantGoogle Scholar
  30. 30.
    Bellifemine F, Poggi A, Rimassa G (2001) JADE: a FIPA2000 compliant agent development environment. In: international conference on autonomous agents and multiagent systemsGoogle Scholar
  31. 31.
    Stanford University, University, S.: ProtégéGoogle Scholar
  32. 32.
    Stardog-union: Pellet: OWL 2 Reasoner for Java,
  33. 33.
    Friedman-Hill E (2008) Jess The rule engine for Java PlatformGoogle Scholar
  34. 34.
    Jing Mei, EP Bontas: Technical Reports: Reasoning Paradigms for Owl Ontologies.
  35. 35.
    Chan M, Campo E, Estève D, Fourniols JY (2009) Smart homes---Current features and future perspectivesGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and InformaticsDe Montfort UniversityLeicesterUK
  2. 2.School of ComputingUlster UniversityBelfastUK

Personalised recommendations