Skip to main content
SpringerLink
Log in

Indoor Localization and Human Activity Tracking with Multiple Kinect Sensors

  • Chapter
  • First Online:
Smart Assisted Living

Part of the book series: Computer Communications and Networks ((CCN))

Abstract

In this chapter, we describe a system that provides continuous localization and behavioral analysis of a person’s motion pattern over an indoor living space using multiple Kinect sensors. The skeleton data from all sensors is transferred to the host computer via TCP sockets into a program where the data is integrated into a single world coordinate system using a calibration technique. Multiple cameras are placed with some overlap in the field of view for the successful calibration of the cameras and continuous tracking of the patients. Localization and behavioral data is stored in a CSV file for further analysis. The experiments show that the system can reliably detect sitting and standing poses, as well as basic gait parameters of a user who is walking within the field of view. This system may be used in an assistive living environment to track the activities of daily living of seniors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Atrsaei A, Salarieh H, Alasty A, Abediny M (2018) Human arm motion tracking by inertial/magnetic sensors using unscented Kalman filter and relative motion constraint. J Intell Rob Syst 90(1–2):161–170

    Article  Google Scholar 

  2. Azis NA, Choi HJ, Iraqi Y (2015) Substitutive skeleton fusion for human action recognition. In: 2015 International conference on big data and smart computing (BigComp). IEEE, pp 170–177

    Google Scholar 

  3. Beymer D, Konolige K (1999) Real-time tracking of multiple people using continuous detection. In: IEEE Frame rate workshop, pp 1–8

    Google Scholar 

  4. Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Van Gool L (2009) Robust tracking-by-detection using a detector confidence particle filter. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 1515–1522

    Google Scholar 

  5. Caon M, Yue Y, Tscherrig J, Mugellini E, Khaled OA (2011) Context-aware 3d gesture interaction based on multiple kinects. In: Proceedings of the first international conference on ambient computing, applications, services and technologies, AMBIENT. Citeseer, pp 7–12

    Google Scholar 

  6. Chen D, Bharucha AJ, Wactlar HD (2007) Intelligent video monitoring to improve safety of older persons. In: 2007 29th Annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 3814–3817

    Google Scholar 

  7. Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 5:564–575

    Article  Google Scholar 

  8. Ercan AO, Gamal AE, Guibas LJ (2013) Object tracking in the presence of occlusions using multiple cameras: a sensor network approach. ACM Trans Sensor Netw (TOSN) 9(2):16

    Google Scholar 

  9. Fuentes LM, Velastin SA (2001) People tracking in surveillance applications. In: Proceedings of 2nd IEEE international workshop on PETS, Kauai, Hawaii, USA

    Google Scholar 

  10. Ikemura S, Fujiyoshi H (2010) Real-time human detection using relational depth similarity features. In: Asian conference on computer vision. Springer, Berlin, pp 25–38

    Chapter  Google Scholar 

  11. Jones B, Sodhi R, Murdock M, Mehra R, Benko H, Wilson A, Ofek E, MacIntyre B, Raghuvanshi N, Shapira L (2014) Roomalive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the 27th annual ACM symposium on user interface software and technology. ACM, pp 637–644

    Google Scholar 

  12. Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45

    Article  Google Scholar 

  13. Klingbeil L, Wark T (2008) A wireless sensor network for real-time indoor localisation and motion monitoring. In: 2008 International conference on information processing in sensor networks (IPSN 2008). IEEE, pp 39–50

    Google Scholar 

  14. Lin Q, Zhang D, Chen L, Ni HB, Zhou S (2014) Managing elders’ wandering behavior using sensors-based solutions: a survey. Int J Gerontol 8(2):49–56

    Article  Google Scholar 

  15. Lun R, Zhao W (2015) A survey of applications and human motion recognition with microsoft kinect. Int J Pattern Recognit Artif Intell 29(05):1555008

    Article  Google Scholar 

  16. Lun R, Gordon C, Zhao W (2016) The design and implementation of a kinect-based framework for selective human activity tracking. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 002890–002895

    Google Scholar 

  17. Lun R, Gordon C, Zhao W (2016) Tracking the activities of daily lives: an integrated approach. In: 2016 Future technologies conference (FTC). IEEE, pp 466–475

    Google Scholar 

  18. Masuyama G, Kawashita T, Umeda K (2017) Complementary human detection and multiple feature based tracking using a stereo camera. ROBOMECH J 4(1):24

    Article  Google Scholar 

  19. Munaro M, Basso F, Menegatti E (2016) Openptrack: open source multi-camera calibration and people tracking for RGB-D camera networks. Robot Auton Syst 75:525–538

    Article  Google Scholar 

  20. Papoulis A, Pillai SU (2002) Probability, random variables, and stochastic processes. Tata McGraw-Hill Education

    Google Scholar 

  21. Poland MP, Nugent CD, Wang H, Chen L (2012) Genetic algorithm and pure random search for exosensor distribution optimisation. Int J Bio-Inspired Comput 4(6):359–372

    Article  Google Scholar 

  22. Ponraj G, Ren H (2018) Sensor fusion of leap motion controller and flex sensors using Kalman filter for human finger tracking. IEEE Sens J 18(5):2042–2049

    Article  Google Scholar 

  23. Spinello L, Arras KO (2011) People detection in RGB-D data. In: 2011 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 3838–3843

    Google Scholar 

  24. Torres-Solis J, Chau T (2010) Wearable indoor pedestrian dead reckoning system. Pervasive Mob Comput 6(3):351–361

    Article  Google Scholar 

  25. Williamson B, LaViola J, Roberts T, Garrity P (2012) Multi-kinect tracking for dismounted soldier training. In: Proceedings of the interservice/industry training, simulation, and education conference (I/ITSEC), pp 1727–1735

    Google Scholar 

  26. Wren CR, Pentland AP (1998) Dynamic models of human motion. In: Proceedings third IEEE international conference on automatic face and gesture recognition. IEEE, pp 22–27

    Google Scholar 

  27. Zhao W, Lun R, Espy DD, Reinthal MA (2014) Realtime motion assessment for rehabilitation exercises: integration of kinematic modeling with fuzzy inference. J Artif Intell Soft Comput Res 4(4):267–285

    Article  Google Scholar 

  28. Zhao W, Lun R, Gordon C, Fofana ABM, Espy DD, Reinthal A, Ekelman B, Goodman GD, Niederriter JE, Luo C et al (2016) Liftingdoneright: a privacy-aware human motion tracking system for healthcare professionals. Int J Handheld Comput Res (IJHCR) 7(3):1–15

    Article  Google Scholar 

  29. Zhao W, Lun R, Gordon C, Fofana ABM, Espy DD, Reinthal MA, Ekelman B, Goodman GD, Niederriter JE, Luo X (2017) A human-centered activity tracking system: toward a healthier workplace. IEEE Trans Human-Mach Syst 47(3):343–355

    Article  Google Scholar 

  30. Zhao W, Reinthal MA, Espy DD, Luo X (2017) Rule-based human motion tracking for rehabilitation exercises: realtime assessment, feedback, and guidance. IEEE Access 5:21382–21394

    Article  Google Scholar 

  31. Zhao W, Wu Q, Reinthal A, Zhang N (2018) Design, implementation, and field testing of a privacy-aware compliance tracking system for bedside care in nursing homes. Appl Syst Innov 1(1):3

    Article  Google Scholar 

  32. Zhao W (2016) A concise tutorial on human motion tracking and recognition with Microsoft kinect. Sci China Inf Sci 59(9):93101

    Google Scholar 

  33. Zhao W (2016) On automatic assessment of rehabilitation exercises with realtime feedback. In: 2016 IEEE international conference on electro information technology (EIT). IEEE, pp 0376–0381

    Google Scholar 

  34. Zhao W, Lun R (2016) A kinect-based system for promoting healthier living at home. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 000258–000263

    Google Scholar 

  35. Zhao W, Espy DD, Reinthal MA, Feng H (2014) A feasibility study of using a single kinect sensor for rehabilitation exercises monitoring: a rule based approach. In: 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE). IEEE, pp 1–8

    Google Scholar 

  36. Zhao W, Feng H, Lun R, Espy DD, Reinthal MA (2014) A kinect-based rehabilitation exercise monitoring and guidance system. In: 2014 IEEE 5th international conference on software engineering and service science. IEEE, pp 762–765

    Google Scholar 

  37. Zhao W, Lun R, Espy DD, Reinthal MA (2014) Rule based realtime motion assessment for rehabilitation exercises. In: 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE). IEEE, pp 133–140

    Google Scholar 

  38. Zhao W, Espy DD, Reinthal MA, Ekelman B, Goodman G, Niederriter J (2015) Privacy-aware human motion tracking with realtime haptic feedback. In: 2015 IEEE international conference on mobile services. IEEE, pp 446–453

    Google Scholar 

  39. Zhao W, Lun R, Gordon C, Fofana AB, Espy DD, Reinthal MA, Ekelman B, Goodman G, Niederriter J, Luo C et al (2016) A privacy-aware kinect-based system for healthcare professionals. In: 2016 IEEE international conference on electro information technology (EIT). IEEE, pp 0205–0210

    Google Scholar 

  40. Zhao W, Wu Q, Espy DD, Reinthal MA, Luo X, Peng Y (2017) A feasibility study on using a kinect-based human motion tracking system to promote safe patient handling. In: 2017 IEEE international conference on electro information technology (EIT). IEEE, pp 462–466

    Google Scholar 

  41. Zhao W, Wu Q, Padaraju V, Bbela M, Reinthal A, Espy D, Luo X, Qiu T (2017) A privacy-aware compliance tracking system for skilled nursing facilities. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 3568–3573

    Google Scholar 

  42. Zhao W, Pillai JA, Leverenz JB, Luo X (2018) Technology-facilitated detection of mild cognitive impairment: a review. In: 2018 IEEE international conference on electro/information technology (EIT). IEEE, pp 0284–0289

    Google Scholar 

  43. Zhu L, Wong KH (2013) Human tracking and counting using the kinect range sensor based on Adaboost and Kalman filter. In: International symposium on visual computing. Springer, Berlin, pp 582–591

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beihang University, Beijing, China

    Shunkun Yang

  2. Cleveland State University, Cleveland, USA

    Akshat Hans & Wenbing Zhao

  3. University of Science and Technology Beijing, Beijing, China

    Xiong Luo

Authors
  1. Shunkun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akshat Hans
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenbing Zhao .

Editor information

Editors and Affiliations

  1. School of Computer Science & Informatics, De Montfort University, Leicester, UK

    Feng Chen

  2. Life Supporting Technologies, Universidad Politecnica de Madrid, Madrid, Spain

    Rebeca I. GarcĂ­a-Betances

  3. School of Computing, Ulster University, Belfast, UK

    Liming Chen

  4. Life Supporting Technologies, Universidad Politecnica de Madrid, Madrid, Spain

    María Fernanda Cabrera-Umpiérrez

  5. School of Computing, Ulster University, Newtownabbey, UK

    Chris Nugent

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yang, S., Hans, A., Zhao, W., Luo, X. (2020). Indoor Localization and Human Activity Tracking with Multiple Kinect Sensors. In: Chen, F., García-Betances, R., Chen, L., Cabrera-Umpiérrez, M., Nugent, C. (eds) Smart Assisted Living. Computer Communications and Networks. Springer, Cham. https://doi.org/10.1007/978-3-030-25590-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-25590-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-25589-3

  • Online ISBN: 978-3-030-25590-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation