Advertisement

Hand-Tremor Frequency Estimation in Videos

  • Silvia L. PinteaEmail author
  • Jian Zheng
  • Xilin Li
  • Paulina J. M. Bank
  • Jacobus J. van Hilten
  • Jan C. van Gemert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11134)

Abstract

We focus on the problem of estimating human hand-tremor frequency from input RGB video data. Estimating tremors from video is important for non-invasive monitoring, analyzing and diagnosing patients suffering from motor-disorders such as Parkinson’s disease. We consider two approaches for hand-tremor frequency estimation: (a) a Lagrangian approach where we detect the hand at every frame in the video, and estimate the tremor frequency along the trajectory; and (b) an Eulerian approach where we first localize the hand, we subsequently remove the large motion along the movement trajectory of the hand, and we use the video information over time encoded as intensity values or phase information to estimate the tremor frequency. We estimate hand tremors on a new human tremor dataset, TIM-Tremor, containing static tasks as well as a multitude of more dynamic tasks, involving larger motion of the hands. The dataset has 55 tremor patient recordings together with: associated ground truth accelerometer data from the most affected hand, RGB video data, and aligned depth data.

Keywords

Video hand-tremor analysis Phase-based tremor frequency detection Human tremor dataset Eulerian hand tremors 

Notes

Acknowledgements

This work is part of the research program Technology in Motion (TIM) (628.004.001), which is financed by the Netherlands Organisation for Scientific Research (NWO). Many thanks for the help with data collection to Elma Ouwehand, MSc.

References

  1. 1.
    Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: CVPR, pp. 3686–3693 (2014)Google Scholar
  2. 2.
    di Biase, L., et al.: Tremor stability index: a new tool for differential diagnosis in tremor syndromes. Brain (2017). awx104Google Scholar
  3. 3.
    Bulat, A., Tzimiropoulos, G.: Human pose estimation via convolutional part heatmap regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 717–732. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46478-7_44CrossRefGoogle Scholar
  4. 4.
    Carreira, J., Agrawal, P., Fragkiadaki, K., Malik, J.: Human pose estimation with iterative error feedback. In: CVPR, pp. 4733–4742 (2016)Google Scholar
  5. 5.
    Cutler, R., Davis, L.S.: Robust real-time periodic motion detection, analysis, and applications. PAMI 22(8), 781–796 (2000)CrossRefGoogle Scholar
  6. 6.
    Dai, H., Zhang, P., Lueth, T.C.: Quantitative assessment of parkinsonian tremor based on an inertial measurement unit. Sensors 15(10), 25055–25071 (2015)CrossRefGoogle Scholar
  7. 7.
    Davis, A., Bouman, K.L., Chen, J.G., Rubinstein, M., Durand, F., Freeman, W.T.: Visual vibrometry: estimating material properties from small motion in video. In: CVPR, pp. 5335–5343 (2015)Google Scholar
  8. 8.
    Elble, R.J., McNames, J.: Using portable transducers to measure tremor severity. Tremor Other Hyperkinetic Movements 6 (2016)Google Scholar
  9. 9.
    Elgharib, M., Hefeeda, M., Durand, F., Freeman, W.T.: Video magnification in presence of large motions. In: CVPR, pp. 4119–4127 (2015)Google Scholar
  10. 10.
    Fleet, D.J., Jepson, A.D.: Computation of component image velocity from local phase information. IJCV 5(1), 77–104 (1990)CrossRefGoogle Scholar
  11. 11.
    Freeman, W.T., Adelson, E.H., et al.: The design and use of steerable filters. TPAMI 13(9), 891–906 (1991)CrossRefGoogle Scholar
  12. 12.
    Goldenberg, R., Kimmel, R., Rivlin, E., Rudzsky, M.: Behavior classification by eigendecomposition of periodic motions. Pattern Recogn. 38(7), 1033–1043 (2005)CrossRefGoogle Scholar
  13. 13.
    Hssayeni, M.D., Burack, M.A., Ghoraani, B.: Automatic assessment of medication states of patients with Parkinson’s disease using wearable sensors. In: EMBC, pp. 6082–6085. IEEE (2016)Google Scholar
  14. 14.
    Jeon, H., et al.: Automatic classification of tremor severity in Parkinson’s disease using a wearable device. Sensors 17(9), 2067 (2017)CrossRefGoogle Scholar
  15. 15.
    Kayaba, H., Kokumai, Y.: Non-contact full field vibration measurement based on phase-shifting. In: CVPR, pp. 3655–3663 (2017)Google Scholar
  16. 16.
    Kooij, J.: SenseCap: synchronized data collection with Microsoft Kinect2 and LeapMotion. In: ACMMM, pp. 1218–1221. ACM (2016)Google Scholar
  17. 17.
    Kooij, J.F.P., van Gemert, J.C.: Depth-aware motion magnification. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 467–482. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46484-8_28CrossRefGoogle Scholar
  18. 18.
    Levy, O., Wolf, L.: Live repetition counting. In: CVPR, pp. 3020–3028 (2015)Google Scholar
  19. 19.
    Lu, C., Ferrier, N.J.: Repetitive motion analysis: segmentation and event classification. PAMI 26(2), 258–263 (2004)CrossRefGoogle Scholar
  20. 20.
    Pintea, S.L., van Gemert, J.C.: Making a case for learning motion representations with phase. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 55–64. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-49409-8_8CrossRefGoogle Scholar
  21. 21.
    Pishchulin, L., et al.: DeepCut: joint subset partition and labeling for multi person pose estimation. In: CVPR, pp. 4929–4937 (2016)Google Scholar
  22. 22.
    Polana, R., Nelson, R.C.: Detection and recognition of periodic, nonrigid motion. IJCV 23(3), 261–282 (1997)CrossRefGoogle Scholar
  23. 23.
    Ramakrishna, V., Munoz, D., Hebert, M., Andrew Bagnell, J., Sheikh, Y.: Pose machines: articulated pose estimation via inference machines. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 33–47. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10605-2_3CrossRefGoogle Scholar
  24. 24.
    Ran, Y., Weiss, I., Zheng, Q., Davis, L.S.: Pedestrian detection via periodic motion analysis. IJCV 71(2), 143–160 (2007)CrossRefGoogle Scholar
  25. 25.
    Ripin, Z.M., Chan, P.Y.: Pathological hand tremor measurement—challenges and advances. In: Ibrahim, F., Usman, J., Ahmad, M.Y., Hamzah, N., Teh, S.J. (eds.) ICIBEL 2017. IP, vol. 67, pp. 3–8. Springer, Singapore (2018).  https://doi.org/10.1007/978-981-10-7554-4_1CrossRefGoogle Scholar
  26. 26.
    Roy, K., Rao, G.S., Anouncia, S.M.: A learning based approach for tremor detection from videos. In: ICOS, pp. 71–76. IEEE (2013)Google Scholar
  27. 27.
    Runia, T.F., Snoek, C.G., Smeulders, A.W.: Real-world repetition estimation by Div, Grad and Curl. In: CVPR, pp. 9009–9017 (2018)Google Scholar
  28. 28.
    Soran, B., Hwang, J., Lee, S., Shapiro, L.: Tremor detection using motion filtering and SVM. In: ICPR, pp. 178–181 (2012)Google Scholar
  29. 29.
    Taewoong Um, T., et al.: Parkinson’s Disease Assessment from a Wrist-Worn Wearable Sensor in Free-Living Conditions: Deep Ensemble Learning and Visualization. CoRR (2018)Google Scholar
  30. 30.
    Tralie, C.J., Perea, J.A.: (Quasi) periodicity quantification in video data, using topology. CoRR (2017)Google Scholar
  31. 31.
    Uhríková, Z., Rŭžička, E., Hlaváč, V., Nugent, C.D.: TremAn: a tool for measuring tremor frequency from video sequences. Mov. Disord. 25(4), 504–506 (2010)CrossRefGoogle Scholar
  32. 32.
    Uhríková, Z., et al.: Validation of a new tool for automatic assessment of tremor frequency from video recordings. J. Neurosci. Methods 198(1), 110–113 (2011)CrossRefGoogle Scholar
  33. 33.
    Victor, B., He, Z., Morgan, S., Miniutti, D.: Continuous video to simple signals for swimming stroke detection with convolutional neural networks. In: CVPR (2017)Google Scholar
  34. 34.
    Vidailhet, M., Roze, E., Jinnah, H.A.: A simple way to distinguish essential tremor from tremulous Parkinson’s disease. Brain 140(7), 1820–1822 (2017)CrossRefGoogle Scholar
  35. 35.
    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T.: Phase-based video motion processing. SIGRAPH 32(4), 80 (2013)zbMATHGoogle Scholar
  36. 36.
    Wei, S., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: CVPR, pp. 4724–4732 (2016)Google Scholar
  37. 37.
    Xia, L., Zou, B., Liu, H., Su, H., Qianghui, H.: A new method for evaluating postural hand tremor based on cmos camera. Optik-Int. J. Light Electron Optics 126(5), 507–512 (2015)CrossRefGoogle Scholar
  38. 38.
    Zarchan, P., Musoff, H.: Fundamentals of Kalman Filtering: A Practical Approach (2013)Google Scholar
  39. 39.
    Zhang, Y., Pintea, S.L., van Gemert, J.C.: Video acceleration magnification. In: CVPR (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Silvia L. Pintea
    • 1
    Email author
  • Jian Zheng
    • 1
  • Xilin Li
    • 1
  • Paulina J. M. Bank
    • 2
  • Jacobus J. van Hilten
    • 2
  • Jan C. van Gemert
    • 1
  1. 1.Vision LabDelft University of TechnologyDelftNetherlands
  2. 2.Department of NeurologyLeiden University Medical CenterLeidenNetherlands

Personalised recommendations