Skip to main content
SpringerLink
Log in

Block-based adaptive ROI for remote photoplethysmography

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Remote photoplethysmography (rPPG) can achieve contactless human vital signs monitoring, but its signal quality is limited by the remote operation nature. In practical applications, improving the rPPG signal quality becomes an essential task. As a remote imaging technique, rPPG utilizes a camera to capture a video of a skin area, especially the facial area, then focuses on a particular sub-area as the region of interest (ROI). In this paper, we investigated a novel adaptive ROI (AROI) approach for improving the rPPG signal quality. In this approach, block-based spatial-temporal division is performed on a captured face video. Based on these segmented video pipelines, the spatial-temporal quality distribution of the rPPG signals is estimated using a signal-to-noise ratio (SNR) feature. Afterwards, AROIs are calculated through mean-shift clustering and adaptive thresholding in SNR maps. As the AROI can be dynamically adjusted according to the spatial-temporal quality distribution of rPPG signals on the face, the quality of the final recovered rPPG signal is improved. The performance of the proposed AROI approach was evaluated with both still and moving subjects. Compared to conventional ROI methods for rPPG, the proposed AROI obtained a higher accuracy in heart rate measurement. And the state-of-the-art motion-resistant rPPG techniques can be effectively enhanced through being integrated with the AROI.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Allen J (2007) Photoplethysmography and its application in clinical physiological measurement. Physiol Meas 28:R1–R39. doi:10.1088/0967-3334/28/3/R01

    Article  Google Scholar 

  2. Bay H, Ess A, Tuytelaars T, Gool LV (2008) SURF: speeded up robust features. Comput Vis Image Und 110:346–359. doi:10.1007/11744023_32

    Article  Google Scholar 

  3. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310. doi:10.1016/S0140-6736(86)90837-8

    Article  Google Scholar 

  4. Bland JM, Altman DG (2007) Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat 17:571–582. doi:10.1080/10543400701329422

    Article  MathSciNet  Google Scholar 

  5. Bousefsaf F, Maaoui C, Pruski A (2014) Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate. Biomed Signal Process Control 8:568–574. doi:10.1016/j.bspc.2013.05.010

    Article  Google Scholar 

  6. Comaniciu D, Meer P (2002) Mean shift: A robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24:603–619. doi:10.1109/34.1000236

    Article  Google Scholar 

  7. Crowe JA, Damianou D (1992) The wavelength dependence of the photoplethysmogram and its application to pulse oximetry. In: Proc IEEE Annu Int Conf Eng Med Biol Soc. p 2423–2424. doi:10.1109/IEMBS.1992.5761522

  8. Feng L (2015a) Block-based adaptive ROI for remote photoplethysmography (still subject) https://youtu.be/7bR8pEM5bnc. Accessed 24 July 2015

  9. Feng L (2015b) Block-based adaptive ROI for remote photoplethysmography (moving subject) https://youtu.be/TR_fkny2tt0. Accessed 24 July 2015

  10. Feng L, Po LM, Xu X, Li Y, Ma R (2014) Motion resistant remote imaging photoplethysmography based on optical properties of skin. IEEE Trans Circuits Syst Video Technol 25:879–891. doi:10.1109/TCSVT.2014.2364415

    Article  Google Scholar 

  11. Fitzpatrick TB (1975) Sun and skin (Soleil et peau). J Med Esthétique 2:33–34. doi:10.1159/000251345

    Google Scholar 

  12. Haan G, Jeanne V (2013) Robust pulse rate from chrominance-based rPPG. IEEE Trans Biomed Eng 60:2878–2886. doi:10.1109/TBME.2013.2266196

    Article  Google Scholar 

  13. Han J, Zhang D, Hu X, Guo L, Ren J, Wu F (2015) Background prior-based salient object detection via deep reconstruction residual. IEEE Trans Circuits Syst Video Technol 25:1309–1321. doi:10.1109/TCSVT.2014.2381471

    Article  Google Scholar 

  14. Hayes MJ, Smith PR (1998) Artifact reduction in photoplethysmography. Appl Opt 37:7437–7446. doi:10.1364/AO.37.007437

    Article  Google Scholar 

  15. He Z, Yi S, Cheung YM, You X, Tang YY (2016) Robust object tracking via key patch sparse representation. IEEE Trans Cybern 47:354–364. doi:10.1109/TCYB.2016.2514714

    Google Scholar 

  16. Holton BD, Mannapperum K, Lesniewski PJ, Thomas JC (2013) Signal recovery in imaging photoplethysmography. Physiol Meas 34:1499–1511

    Article  Google Scholar 

  17. Huang J, You X, Yuan Y, Yang F, Lin L (2010) Rotation invariant iris feature extraction using Gaussian Markov random fields with non-separable wavelet. Neurocomputing 73:883–894. doi:10.1016/j.neucom.2009.09.016

    Article  Google Scholar 

  18. Jacob B, Chen J, Huang Y, Cohen I (2009) Pearson correlation coefficient. In: Cohen I (ed) Noise reduction in speech processing. Springer, Berlin, pp 1–4

    Google Scholar 

  19. Jones MJ, Rehg JM (2002) Statistical color models with application to skin detection. Int J Comput Vis 46:81–96. doi:10.1023/A:1013200319198

    Article  MATH  Google Scholar 

  20. Kawasaki N (2011) Improving motion robustness of contact-less monitoring of heart rate using video analysis. MS thesis, Department of Mathematics and Computer Science, Eindhoven University of Technology

  21. Lempe G, Zaunseder S, Wirthgen T, Zipser S, Malberg H (2013) ROI selection for remote photoplethysmography. Bildverarbeitung für die Medizin p:99–103. doi:10.1007/978-3-642-36480-8_19

  22. Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proc IJCAI. p 674–679 https://cecas.clemson.edu/~stb/klt/lucas_bruce_d_1981_1.pdf

  23. Ma X, Liu Q, He Z, Zhang X, Chen WS (2016) Visual tracking via exemplar regression model. Knowl-Based Syst 106:26–37. doi:10.1016/j.knosys.2016.05.028

    Article  Google Scholar 

  24. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst, Man, Cybern, Syst SMC-9(1):62–66. doi:10.1109/TSMC.1979.4310076

    Article  MathSciNet  Google Scholar 

  25. Poh M, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18:10762–10774. doi:10.1364/OE.18.010762

    Article  Google Scholar 

  26. Sun Y, Hu S, Azorin-Peris V, Greenwald S, Chambers J (2011) Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise. J Biomed Opt 16:077010–1–077010‑9. doi:10.1117/1.3602852

    Google Scholar 

  27. Sun Y, Hu S, Azorin-Peris V, Kalawsky R, Greenwald S (2013) Noncontact imaging photoplethysmography to effectively access pulse rate variability. J Biomed Opt 18:061205–061205. doi:10.1117/1.JBO.18.6.061205

    Article  Google Scholar 

  28. Tarassenko L, Villarroel M, Guazzi A, Jorge J, Clifton DA, Pugh C (2014) Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol Meas 35:807–831. doi:10.1088/0967-3334/35/5/807

    Article  Google Scholar 

  29. Tomasi C, Kanade T (1991) Detection and tracking of point features. Carnegie Mellon University Technical Report CMU-CS-91-132 https://cecas.clemson.edu/~stb/klt/tomasi-kanade-techreport-1991.pdf

  30. Tsouri GR, Kyal S, Dianat S, Mestha LK (2012) Constrained independent component analysis approach to nonobtrusive pulse rate measurements. J Biomed Opt 17:077011–1–077011‑4. doi:10.1117/1.JBO.17.7.077011

    Article  Google Scholar 

  31. Verkruysse W, Svaasand LO, Nelson JS (2008) Remote plethysmographic imaging using ambient light. Opt Express 16:21434–21445. doi:10.1364/OE.16.021434

    Article  Google Scholar 

  32. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proc CVPR. p I-511–I-518. doi: 10.1109/CVPR.2001.990517

  33. Wu HY, Rubinstein M, Shih E, Guttag JV, Durand F, Freeman WT (2012) Eulerian video magnification for revealing subtle changes in the world. ACM Trans Graph 31:65–1–65–8. doi:10.1145/2185520.2185561

    Article  Google Scholar 

  34. Zarzoso V, Comon P (2010) Robust independent component analysis by iterative maximization of the kurtosis contrast with algebraic optimal step size. IEEE Trans Neural Netw 21:248–261. doi:10.1109/TNN.2009.2035920

    Article  Google Scholar 

  35. Zhang D, You X, Wang P, Yanushkevich SN, Tang YY (2009) Facial biometrics using nontensor product wavelet and 2D discriminant techniques. Int J Pattern Recogn 23:521–543. doi:10.1142/S0218001409007260

    Article  Google Scholar 

  36. Zhang D, Han J, Han J, Shao L (2016) Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining. IEEE Trans Neural Netw Learn Syst 27:1163–1176. doi:10.1109/TNNLS.2015.2495161

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, City University of Hong Kong, Hong Kong, SAR, China

    Lai-Man Po, Litong Feng, Yuming Li & Xuyuan Xu

  2. Department of Information Systems, City University of Hong Kong, Hong Kong, SAR, China

    Terence Chun-Ho Cheung

  3. Department of Computer Science, Chu Hai College of Higher Education, Hong Kong, SAR, China

    Kwok-Wai Cheung

Authors
  1. Lai-Man Po
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Litong Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuyuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Terence Chun-Ho Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kwok-Wai Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Litong Feng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Po, LM., Feng, L., Li, Y. et al. Block-based adaptive ROI for remote photoplethysmography. Multimed Tools Appl 77, 6503–6529 (2018). https://doi.org/10.1007/s11042-017-4563-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4563-7

Keywords

Search

Navigation