Advertisement

Motion-tolerant heart rate estimation from face videos using derivative filter

  • Zhao Yang
  • Xuezhi YangEmail author
  • Xiu Wu
Article
  • 35 Downloads

Abstract

Imaging photoplethysmography (IPPG) technique allows us to extract blood volume pulse (BVP) signals from face videos for measuring heart rate (HR), which is useful in applications such as neonatal monitoring, telemedicine and affective computing. Because the BVP signal is small, the HR estimation results are sensitive to face motion disturbance caused by spontaneous head movements and facial expressions of subjects. In this paper, we design a novel filtering method for refining the RGB signals with motion artifacts. Based on the observation that subtle color changes of face skin are smoother than large face motions at temporal scale, we use the three-order derivative of Gaussian filter to select subtle color changes under large motions. Our method is validated on both our self-collected dataset and public dataset MAHNOB-HCI containing face videos with head movements and facial expressions. By employing the proposed filtering method to pre-process the RGB signals before BVP signal extraction, a range of IPPG methods are improved to generate robust HR estimation results under realistic situations.

Keywords

Photoplethysmography Imaging Derivative filter Motion interference Heart rate estimation 

Notes

Acknowledgements

We acknowledge funding support from: Training Programme Foundation for Application of Scientific and Technological Achievements of Hefei University of Technology (JZ2018YYPY0289) and the Specialized Research Fund for the Doctoral Program of Higher Education of China (JZ2018HGBZ0186).

References

  1. 1.
    Asthana A, Zafeiriou S, Cheng S, Pantic M (2013) Robust Discriminative Response Map Fitting with Constrained Local Models. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, pp 3444–3451.  https://doi.org/10.1109/CVPR.2013.442 Google Scholar
  2. 2.
    de Haan G, Jeanne V (2013) Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed Eng 60(10):2878–2886.  https://doi.org/10.1109/TBME.2013.2266196 CrossRefGoogle Scholar
  3. 3.
    de Haan G, van Leest A (2014) Improved motion robustness of remote-PPG by using the blood volume pulse signature. Physiol Meas 35(9):1913.  https://doi.org/10.1088/0967-3334/35/9/1913 CrossRefGoogle Scholar
  4. 4.
    Flash T, Hogan N (1985) The coordination of arm movements: An experimentally confirmed mathematical model. J Neurosci 5(7):1688–1703.  https://doi.org/10.1523/JNEUROSCI.05-07-01688.1985 CrossRefGoogle Scholar
  5. 5.
    Guven G, Gurkan H, Guz U (2018) Biometric identification using fingertip electrocardiogram signals. SIViP 12:933.  https://doi.org/10.1007/s11760-018-1238-4 CrossRefGoogle Scholar
  6. 6.
    Koenderink JJ (1984) The structure of images. Biol Cybern 50(5):363–370.  https://doi.org/10.1007/BF00336961 MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Koenderink JJ, van Doorn AJ (1992) Generic neighborhood operators. IEEE Trans Pattern Anal Mach Intell 14(6):597–605.  https://doi.org/10.1109/34.141551 CrossRefGoogle Scholar
  8. 8.
    Kumar M, Veeraraghavan A, Sabharwal A (2015) DistancePPG: Robust non-contact vital signs monitoring using a camera. Biomedical Optics Express 6(5):1565–1588.  https://doi.org/10.1364/BOE.6.001565 CrossRefGoogle Scholar
  9. 9.
    Lindeberg T (2013) Scale-space theory in computer vision. Springer Science & Business Media, Kluwer Academic Publishers, NorwellzbMATHGoogle Scholar
  10. 10.
    Liu XN, Yang XZ, Jin J, Li JS (2018) Self-adaptive signals separation for non-contact heart rate estimation from facial video in realistic environments. Physiol Meas 39(6):06NT01CrossRefGoogle Scholar
  11. 11.
    Mikolajczyk K, Schmid C (2001) Indexing based on scale invariant interest points, vol 1. Proceedings Eighth IEEE International Conference on Computer Vision (ICCV), Vancouver, pp 525–531.  https://doi.org/10.1109/ICCV.2001.937561 Google Scholar
  12. 12.
    Monkaresi H, Bosch N, Calvo R et al (2017) Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans Affect Comput 8(1):15–28.  https://doi.org/10.1109/TAFFC.2016.2515084 CrossRefGoogle Scholar
  13. 13.
    Piazzi A, Visioli A (2000) Global minimum-jerk trajectory planning of robot manipulators. IEEE Trans Ind Electron 47(1):140–149.  https://doi.org/10.1109/41.824136 CrossRefGoogle Scholar
  14. 14.
    Poh MZ, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774.  https://doi.org/10.1364/OE.18.010762 CrossRefGoogle Scholar
  15. 15.
    Rohrer B, Fasoli S, Krebs HI, Hughes R, Volpe B, Frontera WR, Stein J, Hogan N (2002) Movement smoothness changes during stroke recovery. J Neurosci 22(18):8297–8304.  https://doi.org/10.1523/JNEUROSCI.22-18-08297.2002 CrossRefGoogle Scholar
  16. 16.
    Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55.  https://doi.org/10.1109/TAFFC.2011.25 CrossRefGoogle Scholar
  17. 17.
    Sun Y, Thakor N (2016) Photoplethysmography revisited: from contact to noncontact from point to imaging. IEEE Trans Biomed Eng 63(3):463–477CrossRefGoogle Scholar
  18. 18.
    Temko A (2017) Accurate wearable heart rate monitoring during physical exercises using PPG. IEEE Trans Biomed Eng 64(9):2016–2024.  https://doi.org/10.1109/TBME.2017.2676243 CrossRefGoogle Scholar
  19. 19.
    Tomasi C, Kanade T (1991) Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon UniversityGoogle Scholar
  20. 20.
    Tulyakov S, Alameda-Pineda X, Ricci E, Yin L, Cohn JF, Sebe N (2016) Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, pp 2396–2404.  https://doi.org/10.1109/CVPR.2016.263 Google Scholar
  21. 21.
    Verkruysse W, Svaasand LO, Nelson JS (2008) Remote plethysmographic imaging using ambient light. Opt Express 16(26):21434–21445.  https://doi.org/10.1364/OE.16.021434 CrossRefGoogle Scholar
  22. 22.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Kauai, pp 511–518.  https://doi.org/10.1109/CVPR.2001.990517 Google Scholar
  23. 23.
    Wang W, den Brinker AC, Stuijk S, de Haan G (2017) Algorithmic Principles of Remote-PPG. IEEE Trans Biomed Eng 64(7):1479–1491.  https://doi.org/10.1109/TBME.2016.2609282 CrossRefGoogle Scholar
  24. 24.
    Wang W, den Brinker AC, Stuijk S, de Haan G (2017) Amplitude-selective filtering for remote-PPG. Biomedical Optics Express 8(3):1965–1980.  https://doi.org/10.1364/BOE.8.001965 CrossRefGoogle Scholar
  25. 25.
    Xu S, Sun L, Rohde GK (2014) Robust efficient estimation of heart rate pulse from video. Biomedical Optics Express 5(4):1124–1135.  https://doi.org/10.1364/BOE.5.001124 CrossRefGoogle Scholar
  26. 26.
    Zhao F, Li M, Qian Y, Tsien JZ (2013) Remote measurements of heart and respiration rates for telemedicine. PLoS One 8(10):e71384.  https://doi.org/10.1371/journal.pone.0071384 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer and InformationHefei University of TechnologyHefeiChina
  2. 2.Anhui Province Key Laboratory of Industry Safety and Emergency TechnologyHefeiChina

Personalised recommendations