Signal, Image and Video Processing

, Volume 13, Issue 3, pp 423–430 | Cite as

Motion-resistant heart rate measurement from face videos using patch-based fusion

  • Zhao Yang
  • Xuezhi YangEmail author
  • Jing Jin
  • Xiu Wu
Original Paper


The ability to measure heart rate (HR) from face videos is useful in applications such as neonatal monitoring, telemedicine and affective computing. In the realistic environments, subjects often have spontaneous head movements and facial expressions which severely degrade the performances of the current methods. We propose a novel patch-based fusion framework for estimating accurate HR from face videos in the presence of subjects’ motions. The wavelet time–frequency analysis is applied on the raw blood volume pulse (BVP) signals for selecting less contaminated patches. Furthermore, a weighted fusion formula is constructed to obtain the final precise BVP signal, which is based on frequency and gradient information. Our method is validated on both our self-collected dataset and public dataset MAHNOB-HCI. Compared with the state of the art, experimental results show that the proposed method has an obvious superiority in the accuracy and robustness.


Photoplethysmography Imaging Wavelet time–frequency analysis Fusion Heart rate estimation 



We acknowledge funding support from: Training Programme Foundation for Application of Scientific and Technological Achievements of Hefei University of Technology (JZ2018YYPY0289) and the Specialized Research Fund for the Doctoral Program of Higher Education of China (JZ2018HGBZ0186).


  1. 1.
    Kannel, W.B., Kannel, C., Paffenbarger, R.S., Cupples, L.A.: Heart rate and cardiovascular mortality: the Framingham study. Am. Heart J. 113(6), 1489–1494 (1953). CrossRefGoogle Scholar
  2. 2.
    Temko, A.: Accurate heart rate monitoring during physical exercises using PPG. IEEE Trans. Biomed. Eng. 64(9), 2016–2024 (2017). CrossRefGoogle Scholar
  3. 3.
    Guven, G., Gurkan, H., Guz, U.: Biometric identification using fingertip electrocardiogram signals. Signal Image Video Process. 12, 1–8 (2018). CrossRefGoogle Scholar
  4. 4.
    Allen, J.: Photoplethysmography and its application in clinical physiological measurement. Physiol. Meas. 28(3), R1-39 (2007). CrossRefGoogle Scholar
  5. 5.
    Wu, T., Blazek, V., Schmitt, H.: Photoplethysmography imaging: a new noninvasive and non-contact method for mapping of the dermal perfusion changes. Proc. SPIE 4163, 62–70 (2000). CrossRefGoogle Scholar
  6. 6.
    Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008). CrossRefGoogle Scholar
  7. 7.
    Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010). CrossRefGoogle Scholar
  8. 8.
    Kwon, S., Kim, H., Park K.S.: Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: IEEE Engineering in Medicine and Biology Society (EMBC). San Diego, CA, USA, pp. 2174–2177 (2012).
  9. 9.
    Poh, M.Z., McDuff, D.J., Picard, R.W.: Advancements in non-contact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011). CrossRefGoogle Scholar
  10. 10.
    Li, X.B., Chen, J., Zhao, G., Pietikainen, M.: Remote heart rate measurement from face videos under realistic situation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA, pp. 4264–4271 (2014).
  11. 11.
    De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013). CrossRefGoogle Scholar
  12. 12.
    Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Algorithmic principles of remote-PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2017). CrossRefGoogle Scholar
  13. 13.
    Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Amplitude-selective filtering for remote-PPG. Biomed. Opt. Express 8(3), 1965–1980 (2017). CrossRefGoogle Scholar
  14. 14.
    Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: IEEE International Conference on Computer Vision (ICCV). Washington, DC, USA, pp. 3640–3648 (2015).
  15. 15.
    Kumar, M., Veeraraghavan, A., Sabharwal, A.: DistancePPG: robust non-contact vital signs monitoring using a camera. Biomed. Opt. Express 6(5), 1565–1588 (2015). CrossRefGoogle Scholar
  16. 16.
    Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J.F., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, pp. 2396–2404 (2016).
  17. 17.
    Ha, R.Y., Nojima, K., Adams, W.J., Brown, S.A.: Analysis of facial skin thickness: defining the relative thickness index. Plast. Reconstr. Surg. 115(6), 1769–1773 (2005). CrossRefGoogle Scholar
  18. 18.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Kauai, HI, USA, pp. 511–518 (2001).
  19. 19.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Portland, OR, USA, pp. 3444–3451 (2013).
  20. 20.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University (1991)Google Scholar
  21. 21.
    Daubechies, I., Heil, C.: Ten Lectures on Wavelets. Capital City Press, Vermont (1992)CrossRefzbMATHGoogle Scholar
  22. 22.
    Tarvainen, M.P., Ranta-aho, P.O., Karjalainen, P.A.: An advanced detrending method with application to HRV analysis. IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002). CrossRefGoogle Scholar
  23. 23.
    Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2012). CrossRefGoogle Scholar
  24. 24.
    Huang, S.J., Hsieh, C.T., Huang, C.L.: Application of Morlet wavelets to supervise power system disturbances. IEEE Trans. Power Deliv. 14(1), 235–243 (1999). CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer and InformationHefei University of TechnologyHefeiChina
  2. 2.Anhui Province Key Laboratory of Industry Safety and Emergency TechnologyHefeiChina

Personalised recommendations