Skip to main content
SpringerLink
Log in

On the Diffusion Process for Heart Rate Estimation from Face Videos Under Realistic Conditions

  • Conference paper
  • First Online:
Pattern Recognition (GCPR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10496))

Included in the following conference series:

Abstract

This work addresses the problem of estimating heart rate from face videos under real conditions using a model based on the recursive inference problem that leverages the local invariance of the heart rate. The proposed solution is based on the canonical state space representation of an Itō process and a Wiener velocity model. Empirical results yield to excellent real-time and estimation performance of heart rate in presence of disturbing factors, like rigid head motion, talking and facial expressions under natural illumination conditions making the process of heart rate estimation from face videos applicable in a much broader sense. To facilitate comparisons and to support research we made the code and data for reproducing the results public available.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We also reimplemented other methods [14, 25, 41], since their code is not available. Unfortunately, we obtained worse results.

References

  1. Arnold, I.: Ordinary Differential Equations. MIT Press, Cambridge (1973)

    MATH  Google Scholar 

  2. Bland, J., Altman, D.: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327(8476), 307–310 (1986)

    Article  Google Scholar 

  3. Blanik, N., Blazek, C., Pereira, C., Blazek, V., Leonhardt, S.: Wearable photoplethysmographic sensors: past and present. In: Proceedings of the SPIE 9034, Medical Imaging: Image Processing (2014)

    Google Scholar 

  4. Blazek, C., Hülsbusch, M.: Assessment of allergic skin reactions and their hemodynamical quantification using photoplethysmography imaging. In: Proceedings of 11th International Symposium CNVD, Computer-Aided Noninvasive Vascular Diagnostics, vol. 3, 85–90 (2005)

    Google Scholar 

  5. Blazek, V.: Optoelektronische Erfassung und rechnerunterstützte Analyse der Mikrozirkulations-Rhythmik. Biomed. Techn. 30(1), 121–122 (1985)

    Article  Google Scholar 

  6. Blazek, V., Blanik, N., Blazek, C., Paul, M., Pereira, C., Koeny, M., Venema, B., Leonhardt, S.: Active and passive optical imaging modality for unobtrusive cardiorespiratory monitoring and facial expressions assessment. Assessment. Anesth Analg. 124, 104–119 (2017)

    Article  Google Scholar 

  7. Bloom, H., Bar-Shalom, Y.: The interacting multiple model algorithm for systems with markovian switching coefficients. IEEE Trans. Autom. Control 33(8), 780–783 (1988)

    Article  MATH  Google Scholar 

  8. Cardoso, J.: High-order contrasts for independent component analysis. Neural Comput. 11(1), 157–192 (1999)

    Article  MathSciNet  Google Scholar 

  9. Cox, D.: Some statistical methods connected with series of events. J. Roy. Stat. Soc. 17(2), 129–164 (1950)

    MathSciNet  MATH  Google Scholar 

  10. Durbin, J., Koopman, S.: Time Series Analysis by State Space Methods. Oxford University Press, Oxford (2001)

    MATH  Google Scholar 

  11. Feynman, R., Leighton, R., Sands, M.: The Feynman Lectures on Physics, vol. 1. Addison-Wesley, Boston (1963). Chap. 21

    MATH  Google Scholar 

  12. Gray, R., Neuhoff, D.: Quantization. IEEE Trans. Inf. Theory 44(6), 2325–2383 (1998)

    Article  MATH  Google Scholar 

  13. Grewal, M., Andrews, A.: Kalman Filtering Theory and Practice Using Matlab. Wiley Interscience, Hoboken (2001)

    MATH  Google Scholar 

  14. de Haan, G., Jeanne, V.: Robust pulse-rate from chrominance-based rppg. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2014)

    Article  Google Scholar 

  15. de Haan, G., van Leest, A.: Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiol. Meas. 3(9), 1913–1926 (2014)

    Article  Google Scholar 

  16. Hertzman, A.: Photoelectric plethysmography of the fingers and toes in man. Exp. Biol. Med. 37(3), 529–534 (1937)

    Article  Google Scholar 

  17. Hülsbusch, M.: A functional imaging technique for opto-electronic assessment of skin perfusion. Ph.D. thesis, RWTH Aachen University (2008)

    Google Scholar 

  18. Itô, K.: On Stochastic Differential Equations, vol. 4. Memoris of The American Mathematical Society (1951)

    Google Scholar 

  19. Jazwinski, A.: Stochastic Processes and Filtering Theory. Academic Press, New York (1970)

    MATH  Google Scholar 

  20. Jones, R.H.: Fitting multivariate models to unequally spaced data. In: Parzen, E. (ed.) Time Series Analysis of Irregularly Observed Data. LNS, vol. 25, pp. 158–188. Springer, New York (1984). doi:10.1007/978-1-4684-9403-7_8

    Chapter  Google Scholar 

  21. Kalman, R., Bucy, R.: New results in linear filtering and prediction theory. Trans. ASME-J. Basic Eng. 83, 95–108 (1961)

    Article  MathSciNet  Google Scholar 

  22. Khintchine, A.: Korrelationstheorie der stationären stochastischen Prozesse. Springer-Mathematische Annalen 109, 604–615 (1934)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: IEEE International Conference on Computer Vision, pp. 3640–3648 (2015)

    Google Scholar 

  24. Lewandowska, M., Ruminski, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity. In: Proceedings of the FedCSIS, Szczecin, Poland, pp. 405–410 (2011)

    Google Scholar 

  25. Li, X., Chen, J., Zhao, G., Pietikinen, M.: Remote heart rate measurement from face videos under realistic situations. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH (2014)

    Google Scholar 

  26. Lomb, N.: Least-squares frequency analysis of unequally spaced data. Astrophys. Space Sci. 39(2), 447–462 (1976)

    Article  Google Scholar 

  27. Makhnin, O.: Filtering for some stochastic processes with discrete observations. Ph.D. thesis, Department of Statistics and Probability, Michigan State University (2002)

    Google Scholar 

  28. McDuff, D., Gontarek, S., Picard, R.: Remote measurement of cognitive stress via heart rate variability. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2957–2960 (2014)

    Google Scholar 

  29. Moço, A., Stuijk, S., de Haan, G.: Ballistocardiographic artifacts in PPG imaging. IEEE Trans. Biomed. Eng. 63(9), 1804–1811 (2015)

    Article  Google Scholar 

  30. Moço, A., Stuijk, S., de Haan, G.: Motion robust PPG-imaging through color channel mapping. Biomed. Opt. Express 7, 1737–1754 (2016)

    Article  Google Scholar 

  31. Molitor, H., Knaizuk, M.: A new bloodless method for continuous recording of peripheral change. J. Pharmacol. Exp. Theret. 27, 5–16 (1936)

    Google Scholar 

  32. Øksendal, B.: Stochastic Differential Equations. Springer, Heidelberg (2003)

    Book  MATH  Google Scholar 

  33. Oliver, B., Pierce, J., Shannon, C.: The philosophy of PCM. Proc. IRE 36, 1324–1331 (1948)

    Article  Google Scholar 

  34. Osman, A., Turcot, J., Kaliouby, R.E.: Supervised learning approach to remote heart rate estimation from facial videos. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2015)

    Google Scholar 

  35. Poh, M., McDuff, J., Picard, R.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)

    Article  Google Scholar 

  36. Ramirez, G., Fuentes, O., Crites, S., Jimenez, M., Ordonez, J.: Color analysis of facial skin: detection of emotional state. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 474–479 (2014)

    Google Scholar 

  37. Särkkä, S.: Recursive Bayesian inference on stochastic differential equations. Ph.D. thesis, Helsinki University of Technology (2006)

    Google Scholar 

  38. Särkkä, S., Solin, A., Nummenmaa, A., Vehtari, T., Vanni, F.L.: Dynamic retrospective filtering of physiological noise in BOLD fMRI. NeuroImage 60(2), 1517–1527 (2012)

    Article  Google Scholar 

  39. Scargle, J.: Studies in astronomical time series analysis. II - statistical aspects of spectral analysis of unevenly spaced data. Astrophys. J. 263(1), 835–853 (1982)

    Article  Google Scholar 

  40. Teplov, V.: Blood pulsation imaging. Ph.D. thesis, Department of Applied Physics, University of Eastern Finland (2014)

    Google Scholar 

  41. Tulyakov, S., Pineda, X.A., Ricci, E., Yin, L., Cohn, J., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  42. Verkruysse, W., Svaasand, L., Nelson, J.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)

    Article  Google Scholar 

  43. Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001)

    Article  Google Scholar 

  44. Wang, W., Stuijk, S., de Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2015)

    Article  Google Scholar 

  45. Wiener, N.: The average of an analytical functional and the brownian movement. Proc. Nat. Acad. Sci. USA 7(1), 294–298 (1921)

    Article  MathSciNet  Google Scholar 

  46. Wiener, N.: Generalized harmonic analysis. Acta Mathematica 55, 117–258 (1930)

    Article  MathSciNet  MATH  Google Scholar 

  47. Zakai, M.: On the optimal filtering of diffusion processes. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 11(3), 230–243 (1969)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CanControls GmbH, Aachen, Germany

    Christian S. Pilz

  2. Institute of Safety Technology, University of Wuppertal, Wuppertal, Germany

    Jarek Krajewski

  3. Philips Chair for Medical Information Technology, Helmholtz-Institute for Biomedical Engineering, RWTH Aachen University, Aachen, Germany

    Vladimir Blazek

Authors
  1. Christian S. Pilz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jarek Krajewski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Blazek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian S. Pilz .

Editor information

Editors and Affiliations

  1. University of Basel, Basel, Switzerland

    Volker Roth

  2. University of Basel, Basel, Switzerland

    Thomas Vetter

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Pilz, C.S., Krajewski, J., Blazek, V. (2017). On the Diffusion Process for Heart Rate Estimation from Face Videos Under Realistic Conditions. In: Roth, V., Vetter, T. (eds) Pattern Recognition. GCPR 2017. Lecture Notes in Computer Science(), vol 10496. Springer, Cham. https://doi.org/10.1007/978-3-319-66709-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66709-6_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66708-9

  • Online ISBN: 978-3-319-66709-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation