Blood pulsation measurement using cameras operating in visible light: limitations
- 1.7k Downloads
The paper presents an automatic method for analysis and processing of images from a camera operating in visible light. This analysis applies to images containing the human facial area (body) and enables to measure the blood pulse rate. Special attention was paid to the limitations of this measurement method taking into account the possibility of using consumer cameras in real conditions (different types of lighting, different camera resolution, camera movement).
The proposed new method of image analysis and processing was associated with three stages: (1) image pre-processing—allowing for the image filtration and stabilization (object location tracking); (2) main image processing—allowing for segmentation of human skin areas, acquisition of brightness changes; (3) signal analysis—filtration, FFT (Fast Fourier Transformation) analysis, pulse calculation.
Results and conclusions
The presented algorithm and method for measuring the pulse rate has the following advantages: (1) it allows for non-contact and non-invasive measurement; (2) it can be carried out using almost any camera, including webcams; (3) it enables to track the object on the stage, which allows for the measurement of the heart rate when the patient is moving; (4) for a minimum of 40,000 pixels, it provides a measurement error of less than ±2 beats per minute for p < 0.01 and sunlight, or a slightly larger error (±3 beats per minute) for artificial lighting; (5) analysis of a single image takes about 40 ms in Matlab Version 188.8.131.524 (R2010b) with Image Processing Toolbox Version 7.1 (R2010b).
KeywordsImage processing Blood pulsation Dynamic analysis Pulse
Fast Fourier Transformation
surface mount device
continuous monitoring of the patient, infant for fear of cot death,
monitoring the heart rate (beat frequency changes) of the patient staying at home (hospital) while sleeping (using infrared light),
monitoring the sleep in the elderly and notifying medical services automatically in case of cardiac arrhythmias or pulse absence.
tracking the object (the patient’s face)—automatic stabilization;
measurement for small image resolutions (an older camera or distant object, the patient’s body);
fully automatic measurement;
division of the image into areas with simultaneous measurement of the pulse in eight people at once.
In particular, this article, compared to the existing known work in this field [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23] and [24, 25], includes quantitative assessment of the impact of lighting, frequency of the camera operation and the effect of spatial resolution of images on the results of pulse measurement. Thus, it presents limitations in the practical application of methods for the contactless pulse rate measurement. These limitations are extremely interesting in practical terms as they account for quantitative guidance to the appropriate selection of cameras, type of lighting in the room, and their impact on the results obtained. The results obtained in this study will enable to answer how a change in the type of lighting or camera will affect the measurement accuracy. Thus, the results will provide valuable information for the evaluation of the sensitivity of the proposed new algorithm to changes in the parameters of image acquisition. And this in turn has a direct impact on the location of the camera relative to the patient (patients), the number of light sources and the camera type. It will be also possible to optimize the algorithm (reduce computational complexity) by reducing the resolution of the analysed image—thereby agreeing to the measurement error (calculated in this article). Therefore, limitations of blood pulsation measurement using cameras operating in visible light are important in the evaluation of the afore-mentioned possibilities of practical application, optimization of the algorithm for image analysis and the range of allowable (in terms of the measurement error) changes in light sources, cameras and their resolution. The measurements (discussed below) enable to assess the accuracy with which it will be possible to remotely measure the blood pulsation rate in the existing acquisition conditions (at what minimum camera resolution and at what lighting).
This solution that uses new methods of image analysis and processing is presented in the following sections. Particular attention is also paid to the restrictions on its use.
The measurements were made using two types of cameras imaging the human face. These were some of the cheapest cameras working with USB data transfer 1.0 and 2.0 type Logitech C170 and Gembird CAM69U. The colour image spatial resolution did not exceed M × N = 480 × 640 pixels (M—row, N—column), and the resolution of brightness levels 256 for each RGB component (3·28) at a frequency of 30 frames per second. The face was illuminated with different light sources with illuminance from 200 to 500 lx: daylight, artificial light of a LED (Corn—T35, 6.5 W, 550 lm, colour temperature 3000–6000 K, 230 V AC), artificial light of a bulb with filament (Osram Classic, 60 W), artificial fluorescent light (linear, Leuci, TL5 80 W). Brightness was selected based on the standards and conditions for health and safety related to the brightness level of workplace lighting. 10 measurements lasting longer than 1 min were performed for each of the 4 types of lighting. Then, the image resolution was changed proportionally from 640 × 480 pixels to 60 × 40 pixels every 10 pixels. This gave a total of 30 frames per second, 30·60·10·4 = 72,000 analysed images for one of the two cameras. The reference measurement of the pulse rate was carried out using the PO 80 pulse oximeter by Beurer. As part of this study, no tests were performed on patients. All the videos were recorded during a typical work on the computer that required the use of a camera. The people for whom the images were acquired retrospectively from the camera (a total of 10 people) were aged 30–55 years. They all agreed to participate in the study (make the previously recorded data available). The recorded images were analysed on a PC with Intel® Xeon® X5680 3.33 GHz CPU in Matlab® Version 184.108.40.2064 (R2010b) with Image Processing Toolbox Version 7.1 (R2010b).
image pre-processing—allowing for the image filtration and stabilization (object location tracking);
main image processing—allowing for segmentation of human skin areas, acquisition of brightness changes;
signal analysis—filtration, FFT (Fast Fourier Transformation) analysis, pulse calculation.
These steps are explained in detail in the following subsections.
Main image processing
Limitations of using the described measurement method
The presented methodology of proceedings, image analysis and signal analysis was applied to all the collected measurements for different types of lighting. The results were related to the reference pulse oximeter (Beurer PO 80). The results are shown in the following subsections.
Correlation of the results obtained with the pulse oximeter
Effect of light
Effect of the type of lighting of the face on the pulse measurement
Type of lighting
Measurement error δ for the adopted level of significance
±2 [beats per minute] for p < 0.01
±3 [beats per minute] for p < 0.3
Dependent on control systems (one-half power results in a greater measurement error)
Bulb with filament
±2 [beats per minute] for p < 0.02
Comparable with visible light
Fluorescent light bulb
±4 [beats per minute] for p < 0.1
A greater error for a higher heart rate
Effect of camera operation frequency
Effect of the image spatial resolution
Impact of the number of pixels on the accuracy of results
Number of pixels
Error δ [beats/minute]
No possibility of measurement
For the number of pixels above 10,000 the results of heart rate measurement have an error of ±5 beats per minute. This is the area containing the face, only 100 × 100 pixels. Depending on the type of camera and the distance of the patient’s face from the camera this is a relatively small requirement for the practical use of the described measurement method.
In the literature, there are different ways of applying the methods of analysis and processing of the human face images. These are methods for identifying the position of the eyes, forehead, cheeks in infrared images [35, 36] focused on dermatology diagnosis. There are also face recognition methods [37, 38] used for identifying people—spread in the last few decades [39, 40, 41, 42]. Only a few publications relate to attempts to perform non-contact blood pulsation measurement. In work , the authors showed the possibility of using infrared laser light and a phone camera to measure blood pulsation. This solution, however, requires special lighting and synchronization with the phone . In work , the authors showed a pulse measurement method based on a sequence of images coming from the camera. This work, however, was devoted to the methods of image enhancement (components R, G or B) and was not closely related to the analysis of the noise influence on the measurement . In addition, the area of the face and the human body was not recognized automatically. Other works [38, 41] show similar ways of measuring the blood pulse rate which do not allow for tracking the object and fully automated measurement in imperfect lighting conditions. The measurement error obtained by the authors of these works is ±5 beats per minute. The algorithm and error analysis presented in this article is free of these disadvantages. Moreover, a much smaller measurement error (±2 beats per minute) was obtained.
it allows for non-contact and non-invasive measurement;
it can be carried out using almost any camera, including webcams;
it enables to track the object on the stage, which allows for the measurement of the heart rate when the patient is moving;
for a minimum of 40,000 pixels, it provides a measurement error of less than ±2 beats per minute for p < 0.01 and sunlight, or a slightly larger error (±3 beats per minute) for artificial lighting (see Table 1);
analysis of a single image takes about 40 ms in Matlab Version 220.127.116.114 (R2010b) with Image Processing Toolbox Version 7.1 (R2010b);
frequency of the camera operation must be a minimum of 10 frames per second to reduce the impact of the resulting additional measurement error of ±0.5 beats per minute (see Fig. 6)
- it can be used for:
monitoring the pulse of all patients in any hospital room if the conditions discussed in this article are preserved (and the error sources are minimized);
to monitor the pulse in the elderly at home—during sleep, relaxation, etc.;
to monitor the pulse of infants—the prevention of cot death;
to monitor those particularly excited (provided that this excitation is manifested by an increased frequency of heartbeat) for example: during meetings, conferences, air travel and so on.
a minimum size of the skin area visible by the camera—10,000 pixels. A smaller number of pixels results in a greater measurement error, while for 100 pixels the measurement is impossible;
type of lighting. The best results (the smallest measurement error) is for sunlight and artificial light—light bulb with filament;
minimum frequency of the camera operation of 15 frames per second. A lower camera frequency is possible but it increases measurement errors to ±3 beats per minute.
Currently, the presented algorithm and method are tested in different conditions and different temperature ranges [44, 45, 46] in the Department of Biomedical Computer Systems, University of Silesia in Sosnowiec and the Medical University of Silesia in Poland. These measurements confirm the impact of the type of premises (different arrangement of light sources which results in different measurement conditions) and the type of cameras (mainly their resolution, starting with the assessment of the usefulness of cameras monitoring the premises) on the results obtained. Thus, the algorithm sensitivity to changes in the acquisition parameters in different measurement conditions is determined.
I offer my very warm thanks to Professor Barbara Błońska-Fajfrowska and Dr. Slawomir Wilczyński from the Medical University of Silesia in Sosnowiec in Poland for providing data and measurement (the consent of the Ethics Committee No. KNW/0022/KB1/27/I/16 and others). I would also like to thank Professor Henryk Kasprzak from Wroclaw University of Technology for the inspiration to write this paper.
The author declares that he has no competing interests.
Availability of data and material
The data supporting the conclusions of this article are included within the article. Any queries regarding these data may be directed to the corresponding author.
Consent for publication
Ethics and consent
Not applicable. No tests, measurements or experiments were performed on humans as part of this work.
- 3.Liu C, Freeman WT, Szeliski R, Kang SB. Noise estimation from a single image. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006;1:901–908.Google Scholar
- 6.Huelsbusch M, Blazek V. Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI. In: Medical imaging: physiology and function from multidimensional images, conference proceedings. Bellingham, Wash: SPIE-The International Society for Optical Engineering, San Diego, CA, USA; 2002. p. 110–17.Google Scholar
- 9.Pursche T, Krajewski J, Moeller R. Video-based heart rate measurement from human faces. In: IEEE international conference on consumer electronics (ICCE), Las Vegas, NV: Institute of Electrical and Electronics Engineers Inc; 2012. p. 544–5.Google Scholar
- 14.Shi P, Peris VA, Echiadis A, Zheng J, Zhu Y, Cheang PYS, Hu S. Non-contact reflection photoplethysmographic towards effective human physiological monitoring. J Med Biol Eng. 2010;30:161–7.Google Scholar
- 18.Yu YP, Paramesran R, Lim CL. Video based heart rate estimation under different light illumination intensities. In: Intelligent signal processing and communication systems (ISPACS), 2014 International Symposium on Kuching, 216–221.Google Scholar
- 19.Blackford EB, Estepp JR. Effects of frame rate and image resolution on pulse rate measured using multiple camera imaging photoplethysmography. In: Medical imaging: biomedical applications in molecular, structural, and functional imaging, vol 9417, Orlando; 2015.Google Scholar
- 22.Andreotti F, Trumpp A, Malberg H, Zaunseder S. Improved heart rate detection for camera-based photoplethysmography by means of Kalman filtering. In: IEEE 35th International Conference on Electronics and Nanotechnology 2015. 428–433.Google Scholar
- 28.Kawulok M, Kawulok J, Nalepa J, Smołka B. Self-adaptive algorithm for segmenting skin regions. J Adv Signal Process. 2014;170:1–22.Google Scholar
- 36.Koprowski R. Some selected quantitative methods of thermal image analysis in Matlab. J Biophotonics. 2016;9(5):510–20.Google Scholar
- 45.Glowacz A, Glowacz Z. Recognition of monochrome thermal images of synchronous motor with the application of skeletonization and classifier based on words. Arch Metall Mater. 2015;60(1):27–32.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.