Skip to main content
SpringerLink
Log in

Multi-view fall detection based on spatio-temporal interest points

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Many countries are experiencing a rapid increase in their elderly populations, increasing the demand for appropriate healthcare systems including fall-detection systems. In recent years, many fall-detection systems have been developed, although most require the use of wearable devices. Such systems function only when the subject is wearing the device. A vision-based system presents a more convenient option. However, visual features typically depend on camera view; a single, fixed camera may not properly identify falls occurring in various directions. Thus, this study presents a solution that involves using multiple cameras. The study offers two main contributions. First, in contrast to most vision-based systems that analyze silhouettes to detect falls, the present system proposes a novel feature for measuring the degree of impact shock that is easily detectable with a wearable device but more difficult with a computer vision system. In addition, the degree of impact shock is less sensitive to camera views and can be extracted more robustly than a silhouette. Second, the proposed method uses a majority-voting strategy based on multiple views to avoid performing the tedious camera calibration required by most multiple-camera approaches. Specifically, the proposed method is based on spatio-temporal interest points (STIPs). The number of local STIP clusters is designed to indicate the degree of impact shock and body vibration. Sequences of these features are concatenated into feature vectors that are then fed into a support vector machine to classify the fall event. A majority-voting strategy based on multiple views is then used for the final determination. The proposed method has been applied to a publicly available dataset to offer evidence that the proposed method outperforms existing methods based on the same data input.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Alpaydin E (2010) Introduction to machine learning. MIT Press

  2. Alwan M, Rajendran PJ, Kell S, Mack D, Daldl S, Wolfe M, Felder R (2006) A smart and passive floor-vibration based fall detector for elderly. In: Proc. IEEE Int. Conf. Information and Communication Technologies, pp. 1003–1007

  3. Anderson D, Luke RH, Keller JM, Skubic M, Rantz M, Aud M (2009) Linguistic summarization of video for fall detection using voxel person and fuzzy logic. Comput Vis Image Underst 113(1):80–89

    Article  Google Scholar 

  4. Auvinet E, Multon F, St-Arnaud A, Rousseau J, Meunier J (2011) Fall detection with multiple cameras: an occlusion-resistant method based on 3-D Silhouette vertical distribution. IEEE Trans Inf Technol Biomed 15(2):290–300

    Article  Google Scholar 

  5. Auvinet E, Rougier C, Meunier J, St-Arnaud A, Rousseau J (2010) Multiple cameras fall dataset. DIRO - Universit’e de Montr’eal, Tech. Rep. 1350

  6. Banerjee T, Keller JM, Skubic M, Stone E (2014) Day or night activity recognition from video using fuzzy clustering techniques. IEEE Trans Fuzzy Syst 22(3):483–493

    Article  Google Scholar 

  7. Belbachir AN, Litzenberger M, Schram S, Hofstätter M, Bauer D, Schön P, Humenberger M, Sulzbachner C, Lunden T, Merne M (2012) CARE: a dynamic stereo vision sensor system for fall detection. In: Proc. IEEE Int. Symp. Circuits and Systems, pp. 731–734

  8. Bianchi F, Redmond S, Narayanan M, Cerutti S, Lovell N (2010) Barometric pressure and triaxial accelerometry-based falls event detection. IEEE Trans Neural Syst Rehabil Eng 18(6):619–627

    Article  Google Scholar 

  9. Boissy P, Choquette S, Hamel M, Noury N (2007) User-based motion sensing and fuzzy logic for automated fall detection in older adults. Telemed e-Health 13(6):683–693

    Article  Google Scholar 

  10. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MathSciNet  MATH  Google Scholar 

  11. Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  12. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  13. Dollar P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal features. In: Proc. Int. Conf. Visual Surveillance and Performance Evaluation of Tracking and Surveillance

  14. Doukas CN, Maglogiannis I (2011) Emergency fall incidents detection in assisted living environments utilizing motion, sound, and visual perceptual components. IEEE Trans Inf Technol Biomed 15(2):277–289

    Article  Google Scholar 

  15. Fleck S, Strasser W (2008) Smart camera based monitoring system and its application to assisted living. Proc IEEE 96(10):1698–1714

    Article  Google Scholar 

  16. Fleury A, Vacher M, Noury N (2010) SVM-based multimodal classification of activities of daily living in health smart homes: sensors, algorithms, and first experimental results. IEEE Trans Inf Technol Biomed 14(2):274–283

    Article  Google Scholar 

  17. Goutte C, Gaussier E (2005) A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Proc. Europe Conf. Information Retrieval Research, pp. 345–359

  18. Harris C, Stephens M (1988) A combined corner and edge detector. In: Proc. Alvey Vision Conference

  19. Hawley-Hague H, Boulton E, Hall A, Rfeiffer K, Todd C (2014) Older adults’ perceptions of technologies aimed at falls prevention, detection or monitoring: a systematic review. Int J Medical Inform 83(6):416–426

    Article  Google Scholar 

  20. Huang S-H, Pan Y-C (2013) Learning-based fall detection using RGB-D cameras. In: Proc. IEEE Int. Conf. Machine Vision Applications, pp. 439–442

  21. Karantonis DM, Narayanan MR, Mathie M, Lovell NH, Celler BG (2006) Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. IEEE Trans Inf Technol Biomed 10(1):156–167

    Article  Google Scholar 

  22. Laptev I (2005) On space-time interest points. Int J Comput Vis 64(2–3):107–123

    Article  Google Scholar 

  23. Lee T, Mihailidis A (2005) An intelligent emergency response system: preliminary development and testing of automated fall detection. J Telemed Telecare 11(4):194–198

    Article  Google Scholar 

  24. Lin C-S, Hsu HC, Lay Y-L, Chiu C-C, Chao C-S (2007) Wearable device for real-time monitoring of human falls. Measurement 40(9–10):831–840

    Article  Google Scholar 

  25. Liu H, Chen S, Kubota N (2013) Intelligent video systems and analytics: a survey. IEEE Trans Ind Inf 9(3):1222–1233

    Article  Google Scholar 

  26. Miaou SG, Sung PH, Huang CY (2006) A customized human fall detection system using omni-camera images and personal information. In: Proc. Int. Conf. Distributed Diagnosis Home Healthcare

  27. Mirmahboub B, Samavi S, Karimi N, Shirani S (2013) Automatic monocular system for human fall detection based on variations in silhouette area. IEEE Trans Biomed Eng 60(2):427–436

    Article  Google Scholar 

  28. Mubashir M, Shaon L, Seed L (2013) A survey on fall detection: principles and approaches. Neurocomputing 100:144–152

    Article  Google Scholar 

  29. Nait-Charif H, McKenna SJ (2004) Activity summarisation and fall detection in a supportive home environment. In: Proc. Int. Conf. Pattern Recognition, pp. 323–326

  30. Noury N, Fleury A, Rumeau P, Bourke AK, Laighin GO, Rialle V, Lundy JE (2007) Fall detection – principles and methods. In: Proc. Int. Conf. Engineering in Medicine & Biology Society, pp. 1663–1666

  31. Ozcan K, Mahabalagiri AK, Casares M, Velipasalar S (2013) Automatic fall detection and activity classification by a wearable embedded smart camera. IEEE J Emerging Sel Top Circuits Syst 3(2):125–136

    Article  Google Scholar 

  32. Qian H, Mao Y, Xiang W, Wang Z (2008) Home environment fall detection system based on a cascaded multi-SVM classifier. In: Proc. Int. Conf. Control, Automation, and Robot Vision

  33. Rimminen H, Lindstrom J, Linnavuo M, Sepponen R (2010) Detection of falls among the elderly by a floor sensor using the electric near field. IEEE Trans Inf Technol Biomed 14(6):1475–1476

    Article  Google Scholar 

  34. Rougier C, Meunier J, St-Arnaud A, Rousseau J (2011) Robust video surveillance for fall detection based on human shape deformation. IEEE Trans Circuits Syst Video Technol 21(5):611–622

    Article  Google Scholar 

  35. Scovanner P, Ali S, Shah M (2007) A 3-dimensional sift descriptor and its application to action recognition. In: Proc. Int. Conf. Multimedia

  36. Sixsmth A, Johnson N (2004) A smart sensor to detect falls of the elderly. IEEE Trans Pervasive Comput 3(2):42–47

    Article  Google Scholar 

  37. Sung KK, Poggio T (1998) Example-based learning for view-based human face detection. IEEE Trans Pattern Anal Mach Intell 20(1):39–51

    Article  Google Scholar 

  38. Thome N, Miguet S, Ambellouis S (2008) A real-time, multiview fall detection system: a LHMM-based approach. IEEE Trans Circuits Syst Video Technol 18(11):1522–1532

    Article  Google Scholar 

  39. Toreyin BU, Dedeoglu Y, Cetin AE (2006) HMM based falling person detection using both audio and video. In: Proc. IEEE Signal Processing and Communications Applications

  40. Willems G, Tuytelaars T, Van Gool L (2008) An efficient dense and scale-invariant spatio-temporal interest point detector. In: Proc. European Conf. Computer Vision

  41. Yu X (2008) Approaches and principles of fall detection for elderly and patient. In: Proc. Int. Conf. E-health Network. Application Service

  42. Yu M, Naqvi SM, Rhuma A, Chambers J (2012) One class boundary method classifiers for application in a video-based fall detection system. IET Comput Vis 6(2):90–100

    Article  MathSciNet  Google Scholar 

  43. Zhang T, Wang J, Liu P, Hou J (2006) Fall detection by embedding an accelerometer in cellphone and using KFD algorithm. Int J Comput Sci Netw Secur 6(10):277–284

    Google Scholar 

  44. Zigel Y, Litvak D, Gannot I (2009) A method for automatic fall detection of elderly people using floor vibrations and sound—proof of concept on human mimicking doll falls. IEEE Trans Biomed Eng 56(12):2858–2867

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank E. Auvinet, C. Rougier, J. Meunier, A. St-Arnaud, and J. Rousseau for providing “Multiple cameras fall dataset,” and anonymous reviewers for the valuable and insightful comments on the earlier version of this manuscript. This work was supported by the National Science Council of Taiwan, Republic of China (NSC-103-2221-E-155-033-), the Nature Science Foundation of China (No. 61202143), and the Natural Science Foundation of Fujian Province (No. 2013J05100).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Xiamen University, Xiamen, Fujian, China

    Songzhi Su & Shaozi Li

  2. Fujian Key Laboratory of the Brain-like Intelligent Systems, Xiamen University, Xiamen, Fujian, China

    Songzhi Su & Shaozi Li

  3. Department of Computer Science and Engineering, Yuan Ze University, Taoyuan, Taiwan

    Sin-Sian Wu & Shu-Yuan Chen

  4. Department of Computer Science and Information Engineering, Chien Hsin University of Science and Technology, Taoyuan, Taiwan

    Der-Jyh Duh

Authors
  1. Songzhi Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sin-Sian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shu-Yuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Der-Jyh Duh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaozi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Shu-Yuan Chen or Shaozi Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, S., Wu, SS., Chen, SY. et al. Multi-view fall detection based on spatio-temporal interest points. Multimed Tools Appl 75, 8469–8492 (2016). https://doi.org/10.1007/s11042-015-2766-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2766-3

Keywords

Search

Navigation