Skip to main content
SpringerLink
Log in

A Neurocognitive Robot Assistant for Robust Event Detection

  • Chapter
  • First Online:
Trends in Ambient Intelligent Systems

Part of the book series: Studies in Computational Intelligence ((SCI,volume 633))

Abstract

Falls represent a major problem in the public health care domain, especially among the elderly population. Therefore, there is a motivation to provide technological solutions for assisted living in home environments. We introduce a neurocognitive robot assistant that monitors a person in a household environment. In contrast to the use of a static-view sensor, a mobile humanoid robot will keep the moving person in view and track his/her position and body motion characteristics. A learning neural system is responsible for processing the visual information from a depth sensor and denoising the live video stream to reliably detect fall events in real time. Whenever a fall event occurs, the humanoid will approach the person and ask whether assistance is required. The robot will then take an image of the fallen person that can be sent to the person’s caregiver for further human evaluation and agile intervention. In this paper, we present a number of experiments with a mobile robot in a home-like environment along with an evaluation of our fall detection framework. The experimental results show the promising contribution of our system to assistive robotics for fall detection of the elderly at home.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    OpenNI/NITE: http://www.openni.org/software.

  2. 2.

    Aldebaran Robotics: http://www.aldebaran-robotics.com/.

  3. 3.

    RoboCup Project: http://www.robocup.org/.

  4. 4.

    Processing IDE: http://processing.org/.

  5. 5.

    ROSbridge_suite: http://wiki.ros.org/rosbridge_suite.

  6. 6.

    ROSProcessing: https://github.com/pronobis/ROSProcessing.

  7. 7.

    JSON API: http://jsonapi.org/.

  8. 8.

    NAOqi framework: https://community.aldebaran-robotics.com/doc/1-14/dev/naoqi/index.html.

  9. 9.

    Ubuntu Desktop: http://www.ubuntu.com/desktop.

  10. 10.

    ROS Groovy Galapagos: http://wiki.ros.org/groovy.

References

  1. World Health Organization: Global Report on Falls Prevention in Older Age. http://www.who.int/ageing/publications/Falls_prevention7March.pdf

  2. Tinetti, M.E., Liu, W.L., Claus, E.B.: Predictors and prognosis of inability to get up after falls among elderly persons. J. Am. Med. Assoc. 269(1), 65–70 (1993)

    Article  Google Scholar 

  3. Scheffer, A.C., Schuurmans, M.J., van Dijk, N., van der Hooft, T., de Rooij, S.E.: Fear of falling: measurement strategy, prevalence, risk factors and consequences among older persons. Age Ageing 37(1), 19–24 (2008)

    Article  Google Scholar 

  4. Kaluza, B., Cvetkovic, B., Dovgan, E., Gjoreski, H., Gams, M., Lustrek, M.: A multi-agent care system to support independent living. Int. J. Artif. Intell. Tools 23(1), 1–20 (2013)

    Google Scholar 

  5. Vettier, B., Garbay, C.: Abductive agents for human activity monitoring. Int. J. Artif. Intell. Tools 23 (2014)

    Google Scholar 

  6. Microsoft Kinect for Windows: http://www.microsoft.com/en-us/kinectforwindows/. Cited 10 Sept 2014

  7. ASUS Xtion PRO LIVE sensor: http://www.asus.com/Commercial_3D_Sensor/Xtion_PRO_LIVE. Cited 10 Sept 2014

  8. Jiang, Z., Lin, Z., Davis, L.S.: Recognizing human actions by learning and matching shape-motion prototype trees. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 533–547 (2012)

    Article  Google Scholar 

  9. Parisi, G. I., Wermter, S.: Hierarchical som-based detection of novel behavior for 3D human tracking. In: Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), pp. 1380–1387, Dallas, Texas, USA (2013)

    Google Scholar 

  10. Kachouie, R., Sedighadeli, S., Khosla, R., Chu, M.-T.: Socially assistive robots in elderly care: a mixed-method systematic literature review. Int. J. Hum. Comput. Interact. 30(5), 369–393 (2014)

    Article  Google Scholar 

  11. Igual, R., Medrano, C., Plaza, I.: Challenges, issues and trends in fall detection systems. BioMed. Eng. OnLine 12–66 (2013)

    Google Scholar 

  12. Mubashir, M., Shao, L., Seed, L.: A survey on fall detection: principles and approaches. Neurocomputing 100, 144–152 (2013)

    Article  Google Scholar 

  13. Bourke, A.K., de Ven, V., Gamble, M., O’Connor, R., Murphy, K., Bogan, E., McQuade, E., Finucane, P., OLaighin, G., Nelson, J.: Assessment of waist-worn tri-axial accelerometer based fall-detection algorithms using continuous unsupervised activities. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2782–2785. Institute of Electrical and Electronics Engineers, Buenos Aires (2010)

    Google Scholar 

  14. Lai, C.F., Chang, S.Y., Chao, H.C., Huang, Y.M.: Detection of cognitive injured body region using multiple triaxial accelerometers for elderly falling. IEEE Sens. J. 11, 763–770 (2011)

    Article  Google Scholar 

  15. Kerdegari, H., Samsudin, K., Ramli, A.R., Mokaram, S.: Evaluation of fall detection classification approaches. In: Proceedings of the 4th International Conference on Intelligent and Advanced Systems, pp. 131–136. Institute of Electrical and Electronics Engineers, Kuala Lumpur (2012)

    Google Scholar 

  16. Cheng, J., Chen, X., Shen, M.: A framework for daily activity monitoring and fall detection based on surface electromyography and accelerometer signals. IEEE J. Biomed. Health Inf. 17(1), 38–45 (2013)

    Article  Google Scholar 

  17. Albert, M.V., Kording, K., Herrmann, M., Jayaraman, A.: Fall classification by machine learning using mobile phones. PLoS ONE 7, e36556 (2012)

    Article  Google Scholar 

  18. Lee, R.Y.W., Carlisle, A.J.: Detection of falls using accelerometers and mobile phone technology. Age Ageing 0, 1–7 (2011)

    Google Scholar 

  19. Fang, S.H., Liang, Y.C., Chiu, K.M.: Developing a mobile phone-based fall detection system on android platform. In: Proceedings of the Conference on Computing, Communications and Applications, pp. 143–146, Hong Kong (2012)

    Google Scholar 

  20. Abbate, S., Avvenuti, M., Bonatesta, F., Cola, G., Corsini, P., Vecchio, A.: A smartphone-based fall detection system. Pervasive Mob. Comput. 8, 883–899 (2012)

    Article  Google Scholar 

  21. Patsadu, O., Nukoolkit, C., Watanapa, B.: Survey of smart technologies for fall motion detection: techniques, algorithms and tools. In: Papasratorn, B., et al. (eds.) IAIT 2012, CCIS 344, pp. 137–147. Springer, Heidelberg (2012)

    Google Scholar 

  22. Botia, J.A., Villa, A., Palma, J.: Ambient assisted living system for in-home monitoring of healthy independent elders. Expert Syst. Appl. 39, 8136–8148 (2012)

    Article  Google Scholar 

  23. Lee, T., Mihailidis, A.: An intelligent emergency response system: preliminary development and testing of automated fall detection. J. Telemed. Telecare 11, 194–198 (2005)

    Article  Google Scholar 

  24. Miaou, S.G., Sung, P.H., Huang, C.Y.: A customized human fall detection system using omni-camera images and personal information. In: Proceedings of the 1st Distributed Diagnosis and Home Healthcare Conference, pp. 39–42. Institute of Electrical and Electronics Engineers, Arlington (2006)

    Google Scholar 

  25. Rougier, C., Meunier, J., St-Arnaud, A., Rousseau, J.: Robust video surveillance for fall detection based on human shape deformation. IEEE Trans. Circuits Syst. Video Technol. 21, 611–622 (2011)

    Article  Google Scholar 

  26. Liu, C.L., Lee, C.H., Lin, P.M.: A fall detection system using k-nearest neighbor classifier. Expert Syst. Appl. 37, 7174–7181 (2010)

    Article  Google Scholar 

  27. Cucchiara, R., Prati, A., Vezzani, R.: A multi-camera vision system for fall detection and alarm generation. Expert Syst. 24, 334–345 (2007)

    Article  Google Scholar 

  28. Hazelhoff, L., Han, J., de With, P.H.N.: Video-based fall detection in the home using principal component analysis. In: Bland-Talon, J., Bourennane, S., Philips, W., Popescu, D., Scheunders, P. (eds.) Proceedings of the 10th International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 298–309. Springer, Juan-les-Pins (2008)

    Google Scholar 

  29. Diraco, G., Leone, A., Siciliano, P.: An active vision system for fall detection and posture recognition in elderly healthcare. In: Conference and Exhibition: Design, Automation and Test in Europe, pp. 1536–1541. European Design and Automation Association, Dresden (2010)

    Google Scholar 

  30. Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans. cybern. 43(5), 1318–1334 (2013)

    Article  Google Scholar 

  31. Rougier, C., Auvinet, E., Rousseau, J., Mignotte, M., Meunier, J.: Fall detection from depth map video sequences. In: Abdulrazak, B., et al. (eds.) ICOST 2011. LNCS 6719, pp. 121–128 (2011)

    Google Scholar 

  32. Planinc, R., Kampel, M.: Introducing the use of depth data for fall detection. In: Personal Ubiquitous Computing, vol. 17, pp. 1063–1072. Springer, Heidelberg (2012)

    Google Scholar 

  33. Mastorakis, G., Makris, D.: Fall detection system using Kinects infrared sensor. J. Real-Time Image Process (2012) (Springer)

    Google Scholar 

  34. Zhang, C., Tian, Y., Capezuti, E.: Privacy preserving automatic fall detection for elderly using RGBD cameras. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) Proceedings of the 13th International Conference on Computers Helping People with Special Needs, pp. 625–633. Springer, Linz (2012)

    Google Scholar 

  35. Gasparrini, S., Cippitelli, E., Spinsante, S., Gambi, E.: A depth-based fall detection system using a kinect sensor. Sensors 14, 2756–2775 (2014)

    Article  Google Scholar 

  36. Cogniron: Cognitive Robot Companion. http://www.cogniron.org. Cited 15 Jan 2015

  37. LIREC: Living with Robots and Interactive Companions. http://lirec.eu/. Cited 15 Jan 2015

  38. Hermes: Cognitive Care and Guidance for Active Ageing. http://www.fp7-hermes.eu. Cited 15 Jan 2015

  39. KSERA: Knowledgable SErvice Robots for Aging. http://ksera.ieis.tue.nl. Cited 15 Jan 2015

  40. GiraffPlus: http://www.giraffplus.eu. Cited 15 Jan 2015

  41. ROBOT-ERA: Implementation and integration of advanced robotic systems and intelligent Environments in real scenarios for the ageing population. http://www.robot-era.eu. Cited 15 Jan 2015

  42. Amirabdollahian, F., Bedaf, S., Bormann, R., Draper, H., Evers, V., Gallego Perez, J., Gelderblom, G.J., et al.: Assistive technology design and development for acceptable robotics companions for ageing years. Paladyn J. Behav. Robot. 4(2), 1–9 (2013)

    Article  Google Scholar 

  43. Mundher, Z.A., Zhong, J.: A real-time fall detection system in elderly care using mobile robot and kinect sensor. Int. J. Mater. Mech. Manuf. 2(2), 133–138 (2014)

    Google Scholar 

  44. Volkhardt, M., Schneemann, F., Gross, H.-M.: Fallen person detection for mobile robots using 3D depth data. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (IEEE-SMC), pp. 3573–3578, Manchester, GB (2013)

    Google Scholar 

  45. Martinson, E.: Detecting occluded people for robotic guidance. In: IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 744–749, Edinburgh, Scotland, UK (2014)

    Google Scholar 

  46. Volkhardt, M., Gross, H.-M.: Finding people in home environments with a mobile robot. In: European Conference on Mobile Robots (ECMR), pp. 282–287, Barcelona, Spain (2013)

    Google Scholar 

  47. Parisi, G.I., Weber, C., Wermter, S.: Human action recognition with hierarchical growing neural gas. In: Wermter, S., et al. (eds.) Proceedings of the International Conference on Artificial Neural Networks (ICANN), pp. 89–96, Hamburg, Germany (2014)

    Google Scholar 

  48. Papadopoulos, G.Th., Axenopoulos, A., Daras, P.: Real-time skeleton-tracking-based human action recognition using kinect data. In: Gurrin, C., et al. (eds.) MultiMedia Modeling. LNCS, vol. 8325, pp. 473–483, Springer International Publishing (2014)

    Google Scholar 

  49. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. 34(3), 334–352 (2004)

    Article  Google Scholar 

  50. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surveill. 15 (2009)

    Google Scholar 

  51. Kohonen, T.: Self-organizing map, 2nd edn. Springer, Heidelberg (1995)

    Google Scholar 

  52. Hu, W., Xie, D., Tan, T.: A hierarchical self-organizing approach for learning the patterns of motion trajectories. IEEE Trans. Neural Netw. 15(1), 135–144 (2004)

    Article  Google Scholar 

  53. Nag, A.K., Mitra, A., Mitra, S.: Multiple outlier detection in multivariate data using self-organizing maps title. Comput. Stat. 2(2), 245–264 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  54. Parisi, G.I., Barros, P., Wermter, S.: FINGeR: framework for interactive neural-based gesture recognition. In: Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), pp. 443–447, Bruges, Belgium (2014)

    Google Scholar 

  55. Parisi, G.I., Jirak, D., Wermter, S.: HandSOM: neural clustering of hand motion for gesture recognition in real time. In: Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 981–986, Edinburgh, Scotland, UK (2014)

    Google Scholar 

  56. Hoglund, A.J., Hatonen, K., Sorvari, A.S.: A computer host-based user anomaly detection system using self-organizing maps. In: IEEE-INNS-ENNS International Joint Conference on Neural Networks, vol. 5, pp. 411–416 (2000)

    Google Scholar 

  57. Van Rijsbergen, C.J.: Information retrieval, 2nd edn. Information Retrieval, Butterworth-Heinemann, London (1979)

    Google Scholar 

  58. simple-openni—OpenNI library for Processing: https://code.google.com/p/simple-openni/. Cited 15 Jan 2015

  59. The Robot Operating System (ROS): http://www.ros.org/. Cited 15 Jan 2015

  60. Wang, S., Zabir, S., Leibe, B.: Lying pose recognition for elderly fall detection. In: Conference on Robotics: Science and Systems VII, pp. 44–50, Los Angeles, CA, USA (2011)

    Google Scholar 

  61. Aerts, M.B., Esselink, R.A.J., Post, B., van de Warrenburg, B.P.C., Bloem, B.R.: Improving the diagnostic accuracy in parkinsonism: a three-pronged approach. Pract. Neurol. 12(2), 77–87 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Erik Strahl for his invaluable technical contribution and help. The authors gratefully acknowledge funding by the DAAD German Academic Exchange Service (Kz:A/13/94748)—Cognitive Assistive Systems Project, by the DFG German Research Foundation (grant #1247)—International Research Training Group CINACS (Cross-modal Interaction in Natural and Artificial Cognitive Systems), and the DFG under project CML (TRR169).

Author information

Authors and Affiliations

  1. Department of Informatics, Knowledge Technology, University of Hamburg, Vogt-Koelln-Strasse 30, 22527, Hamburg, Germany

    German I. Parisi & Stefan Wermter

Authors
  1. German I. Parisi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Wermter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to German I. Parisi .

Editor information

Editors and Affiliations

  1. School of Engineering & Technology, Sharda University, Greater Noida, India

    Kiran Kumar Ravulakollu

  2. Computing (Ministry of Comm. and IT, Centre for Development of Advanced, Noida, India

    Mohammad Ayoub Khan

  3. (MIR Labs), Machine Intelligence Research Labs, Auburn, Washington, USA

    Ajith Abraham

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Parisi, G.I., Wermter, S. (2016). A Neurocognitive Robot Assistant for Robust Event Detection. In: Ravulakollu, K., Khan, M., Abraham, A. (eds) Trends in Ambient Intelligent Systems. Studies in Computational Intelligence, vol 633. Springer, Cham. https://doi.org/10.1007/978-3-319-30184-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30184-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-30182-2

  • Online ISBN: 978-3-319-30184-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation