A Systematic Literature Review of the Factors that Influence the Accuracy of Consumer Wearable Health Device Data
- 1.4k Downloads
The use of consumer wearable health device (CWHD) for fitness tracing has seen an upward trend worldwide. CWHDs support individuals in taking ownership of their personal well-being and keeping track of their fitness goals. However, there are genuine concerns over the accuracy of the data collected by these devices. In this study, we investigated the factors that influence the accuracy of the data collected by CWHDs for heart rate measurement, physical activity (PA), and sleep monitoring using a systematic literature review. Forty-seven papers were analyzed from five electronic databases based on specific inclusion and exclusion criteria. All 47 papers that we analyzed were published by authors from developed countries. Using thematic analysis, we classified the factors that influence the accuracy of the data collected by CWHDs into three main groups, namely (i) the tracker and sensor type, (ii) the algorithm used in the device, and (iii) the limitation in the design, energy consumption, and processing capability of the device. The research results point to a dearth of studies that focus on the accuracy of the data collected by CWHDs by researchers from developing countries.
KeywordsConsumer wearable health device CWHD Wearable health device Wearable health device data accuracy Systematic literature review
1 Introduction and Background
Physical inactivity is listed as one of the major contributors to mortality, resulting in an estimated 3.2 million deaths worldwide . Adults are advised to engage in a minimum of 150 min of moderate-to-vigorous-intensity physical activity (PA) per week to reduce the risk of chronic diseases like hypertension, diabetes, and obesity [2, 3]. Physical inactivity can be reduced by walking, which is an inexpensive form of exercise for many adults and requires no special equipment [2, 4]. The adoption and use of consumer wearable health device (CWHD) for PA tracking is increasing. This increase is evident in wearable health technologies retaining the top-three global fitness trends since 2016, taking the first spot in 2016, 2017, and 2019 .
CWHDs support individuals to take ownership of their personal well-being and keep track of their fitness goals. To do this, CWHDs have features that support continuous monitoring and recording of physiological (e.g. heart rate, sleep pattern, blood sugar levels, and so on) and PA data (e.g. duration of PA, distance covered, energy expended, and so forth) [6, 7]. To promote healthy habits, CWHDs incorporate behavioural change techniques like goal setting, self-monitoring, feedback, social influence, and reward . In South Africa, the uptake of CWHDs are on the rise, especially among health-conscious individuals in the urban areas. This increase can partly be attributed to new practice by health insurers who use incentives to motivate their members to use wearable health device to track their PA .
In addition to the increase in the adoption of CWHDs by health-conscious individuals to track PA, wearable health devices are also used for remote monitoring of people with chronic disease conditions [6, 10, 11]. In both of these usage conditions, it is important that the data collected by the device is accurate. Inaccurate data from wearable health devices could lead to dire consequences, especially when the device is integrated with healthcare applications . Manufacturers of CWDHs often make strong claims about the accuracy and reliability of their devices . However, there are genuine concerns over the accuracy of the data collected by CWHDs . For example accelerometer and pedometer-based CWHDs are known to be inaccurate in their estimation of energy expended (EE), and are unable to accurately track the number of steps in PA like cycling [6, 7]. Users of CWHDs expect, and are increasingly demanding that manufacturers deliver on their promises. The two class action lawsuits filed by users against one of the major manufacturers of CWHD in 2016 underscore the importance of accurate and reliable data collected by CWHDs . Hence, it is no surprise that many researchers from developed countries [16, 17, 18] are focusing on the accuracy of the data collected by CWHDs. As discussed later in Sect. 3, all the papers analyzed in this systematic literature review (SLR) were published by authors from developed countries. This points to an apparent dearth of studies that focus on the accuracy of the data collected by CWHDs by researchers from developed countries, including Africa. To address this gap, this research investigates the factors that influence the accuracy of the data collected by CWHDs. More specifically the research focuses on the accuracy of the data generated from heart rate measurement, PA, and sleep monitoring. The research question that we address in the paper is: “What are the factors that influence the accuracy of the data collected by consumer health wearable devices?”.
The remaining sections of the paper are structured as follow: In Sect. 2 we present the process that was followed in the SLR. This is followed by detailed discussions of our analysis of the papers included in the SLR in Sect. 3. In Sect. 4, we discuss the study contribution, limitations, and the implications for the manufacturers of CWHDs.
2 Systematic Literature Review Process
In order to scope the SLR process, research articles were retrieved from the following scientific databases, based on their publication of quality and high impact research journals and conference papers: IEEE, PUBMED-NCBI, ScienceDirect, MDPI, and Springer. To ensure that we retrieve relevant papers, we used the following search phrases: “Consumer wearable health device” OR “wearable health device” OR “wearable health technology” OR “Personal health device” AND “Data Accuracy” OR “Reliability”.
Inclusion and Exclusion Criteria.
The search period for the SLR was between April and October 2019. An initial search on Google Scholar returned more than 22 000 results. To ensure a more realistic number of potential papers to screen for eligibility, we focused specifically on five databases, namely IEEE, PUBMED-NCBI, ScienceDirect, MDPI, and Springer. A total of 1393 papers were retrieved from the five databases. An additional 20 papers were retrieved from other sources (see Sect. 3 for the list of other sources), thus yielding a total of 1413 candidate papers for screening. Details of all 1413 papers were extracted and copied into an Excel worksheet with the following columns: Title, Author, Publication type, DOI, Abstract, Relevance, Included/Excluded 1st Screening, and Included/Excluded 2nd Screening.
In Excel, a Vertical Lookup (VLOOKUP) was performed on the papers’ Title and DOI to check for duplicates. This process resulted in 1214 unique sources. Thereafter, the 1214 sources were reviewed against the inclusion and exclusion criteria specified in Fig. 1, resulting in 465 papers. We then screened the 465 papers for relevance based on their title, keywords, and abstract. Of the 465 papers, 311 were excluded based on their title and 70 were excluded based on their abstracts. Thus, the remaining 84 papers were marked as relevant and eligible for further screening.
In this section, we present the results obtained from the analysis of the 47 papers included in the SLR.
Distribution of research papers per database.
No of papers
% of papers
Our analysis of the papers according to year of publication shows that the majority of papers were published between 2016 (13) and 2017 (12). The number of publications tapered down to eight in 2018, with only one of the papers analyzed being published in 2019. These statistics show increasing interest of researchers in the accuracy of the data collected by CWHDs. The limited number of papers in 2019 should not be construed as waning interest in the topic. Rather, it can be attributed to the search period for sources that were included in the SLR.
Our analysis of the papers included in the SLR shows that all authors are from developed countries. The majority of the papers (13) were published by authors from the United States of America (USA). There were six papers from Australia, five from China, and four from Korea. Three papers were published by authors from Italy and Japan respectively. There were two papers published by authors from Denmark, the United Kingdom, and Germany respectively. Authors from Argentina, Canada, India, Malaysia, Netherlands, Portugal, and Spain published one paper each. Based on our analysis, authors from African countries are conspicuously absent in the publication of papers that focus on the accuracy of the data collected by CWHDs.
Following the quantitative analysis, we identified common themes in the papers and grouped them into three categories. In the following sub-sections, we discuss the three main factors that influence the accuracy of the data collected by CWHDs.
3.1 The Tracker and Sensor Types
Photoplethysmography (PPG) sensors: PPG sensors are used in CWHD to monitor heart rate. Using optical sensors, changes in the blood volume of body tissues can be detected by shining light on the surface of the skin to detect discoloration when oxygen-rich blood is ‘flushed’ underneath the skin [12, 20].
Pedometer and accelerometer: Pedometer is a lightweight device with sensor that measures the number of steps taken or the distance covered. Accelerometer measures PA by detecting movements across three planes (side-to-side, up-and-down, or forward-and-backward) .
Actigraphy: This is a non-invasive, wrist-worn device that comes with accelerometer to measure sleep pattern by distinguishing between the states of wakefulness and sleep unobtrusively. This is based on the premise that limited movement is associated with sleep while increased movement is linked with wakefulness [22, 23].
PPG sensors can be attached to various body parts, including the upper arm, the earlobe, the forehead, the wrist, or the finger. The part of the body where a PPG sensor is attached can influence its level of accuracy. Signals from finger-based PPG sensors have higher wavelengths compared to other sites. This makes finger-based PPG sensors more accurate. However, wearing a finger-based PPG sensor can interfere with daily routines, which makes their use less practical compared to other PPG sensors .
Another factor that could influence the accuracy of PPG sensors is the colour of the light emitting diode (LED) light that comes with the sensor. The majority of CWHDs that utilize PPG sensors for monitoring heart rate come with green light PPG (gPPG) [12, 24, 25]. However, red light PPG (rPPG) sensors (i.e. pulse oximeters) are commonly used in clinical environments [12, 25, 26, 27]. rPPG sensors have a number of advantages over gPPG sensors. The green light in gPPG sensors emits shorter wavelengths and does not penetrate deeper into the innermost layer of the skin. In contrast, rPPG sensors can penetrate deeper into the skin because the human body does not absorb the red light [25, 26]. This property enables rPPG sensors to detect other biological signals like the arterial oxygen saturation, respiration, and blood pressure [24, 26, 27]. In addition, rPPG light is not absorbed by melanin (the pigmentation that is responsible for the colour of the human skin) but gPPG light absorbs melanin. Therefore, the skin colour does not influence the accuracy of heart rate measurements when using rPPG sensors. In contrast, darker skin colours influence the accuracy of gPPG sensors [19, 26, 28].
A drawback of rPPG sensors is that they are more susceptible to background noise generated from the body part that the device is attached (for example, hand waving or rubbing), often referred to as ‘motion artefact’. Motion artefacts are known to have negative influence on the accuracy of rPPG sensors. This is not the case for gPPG sensors, which are less vulnerable to the effects of motion artifacts [19, 25, 29].
Sleep, increased PA, and good nutrition are integral parts of maintaining personal well-being. Prior to the pervasive adoption and use of CWHDs, monitoring and tracking of sleep can only be carried out in specialized sleep laboratories using polysomnography (PSG). PSG measures sleep quality by collecting data on eye movements, heart rates, muscle tones, brain activities, and physical movements [13, 22]. The unnatural setting and the need for a sleep technologist to set up PSG equipment make its use impractical in a home setting. Consumer wearable sleep monitoring sensors, called actigraphy, is a non-invasive wrist-worn device that comes with accelerometer, heart rate, and respiratory monitor to detect and record the movements of the wearer at regular intervals in order to estimate sleep and wakefulness [13, 22, 23, 30, 31].
Actigraphy has been shown to be accurate in detecting the state of sleep, but less so in sensing wakefulness. For example, lying down could be misinterpreted as sleep due to the absence of movements, thereby leading to overestimation [13, 23, 32]. This deficiency is primarily due to the fact that actigraphy associates reduced movements with sleep. As such, actigraphy is not very effective in monitoring the different stages of sleep.
CWHDs are equipped with sensors to track PA in the form of pedometer or accelerometer. Previous studies show that while CWHDs with pedometer sensors are effective in estimating step counts, they typically underestimate energy expenditure (EE). Accelerometers on the other hand are deficient in their accurate measurement of steps taken in PA like cycling [3, 6, 7, 33]. The placement of a PA sensor and the speed of walking are some of the factors that could influence its accuracy. Pedometers are less accurate when the sensor is attached to the wrist or hip, compared to ankle-based pedometers. Similarly, slower walking speed, unsteady and uneven gaits influence accuracy [33, 34, 35, 36]. In the case of accelerometers, Nelson et al.  found that wrist-worn accelerometers are more accurate than hip-worn sensors. However, Simpson et al.  suggest that better accuracy could be achieved when an accelerator sensor is place around the ankle, especially for individuals that walk at slower speeds.
3.2 The Algorithm Used in Consumer Wearable Health Devices
The algorithm used to monitor health parameters by CWHDs is another factor that influence the accuracy of the data collected by the devices . The built-in algorithms in CWHDs support the measurement of bio-sensory and PA data, their processing, and the communication of the outcome of the measurements to the user. Manufacturers of CWHDs do not disclose the algorithms that are used to track and measure bio-sensory and PA for proprietary reasons [17, 33, 38, 39]. This makes it difficult for users to objectively make comparison between devices. In this section, we summarize the algorithms that could be used to monitor heart rate and PA. The discussion of algorithms is limited to the ones reported in the papers that were included in the SLR.
Algorithms for detecting and monitoring motion and PA, developed or proposed by researchers, include pedestrian dead reckoning (PDR) and zero velocity update (ZUPT) algorithms [38, 40, 41]. PDR algorithm estimates walking distance by sensing the number of steps taken and the length of each step. PDR algorithms are more accurate in their estimation of distance covered when the tracking device is attached to the foot . ZUPT algorithm is used to detect and bound static position errors that are accumulated when calculating distance covered using a PDR algorithm. The ZUPT algorithm then detects the periodic static states when the foot returns flat to the ground during walking [40, 41].
Researchers like [27, 41, 42, 43, 44] have proposed algorithms that could improve signals from PPG sensors, thereby improving their accuracy. As discussed in Sect. 3.1, the PPG sensors used to monitor heart rates are susceptible to background noise from ‘motion artefacts’, which could affect the accuracy of heart rate measurements. In their study, Yang et al.  develop an Adaptive Spectrum Noise Cancellation (ASNC) algorithm that significantly improve accuracy when ‘motion artefact’ increases. Yousefi et al.  also propose a motion-tolerant algorithm to improve signals from PPG sensors by removing ‘motion artefacts’. Similarly, Tang et al.  use the Empirical Mode Decomposition (EMD) and Discrete Wavelet Transform (DWT) algorithms to enhance and reduce noise from PPG signals. These authors provide evidence that demonstrate the ability of the algorithms to improve the accuracy of heart rates captured by PPG sensors.
Another element that is closely linked to the algorithms used to measure bio-sensory and PA data is the firmware installed on CWHDs. Firmware updates are necessary to ensure optimal performance and the security the data collected by the device. However, CWHDs can become vulnerable to privacy and security threats during firmware updates. The authors, Fereidooni et al.  and, Lin and Sun  provide evidence that it is possible for people with the technical wherewithal to inject arbitrary or malicious codes into CWHDs’ firmware during updates. The ability to modify firmware by unauthorized persons can affect the integrity of the data collected by CWHDs. Another concern about firmware updates is that the same CWHD could provide different measurements, depending on the firmware applied. Thus distorting the measurements even if other variables remain unchanged .
3.3 Limitations in the Design, Energy Consumption, and Processing Capabilities
Based on our analysis of the papers included in this SLR, the third main factor that could influence the accuracy of the data collected by CWHDs relates to inherent limitations in the design, energy consumption, and the processing capability of a device.
CWHDs are increasingly becoming part of the evolving Internet of Things (IoT) ecosystem. IoT-enabled wearable health devices provide opportunity for continuous monitoring of patients from the comfort of their homes and the transfer of health data to healthcare providers. However, the performance, energy consumption, and the form factor could influence the success of IoT-enabled CWHDs . The convenience and usefulness of a CWHD is dependent on the balance between the device’s size and its battery life. Smaller devices are easier to carry, but do not always have longer battery life. Conversely, longer battery life is commonly associated with bigger size devices [47, 48]. Additional strain is placed on the energy requirements of CWHDs due to continuous collection and exchange of physiological data between a CWHD, other connected IoT devices and applications (apps) .
The quality of the components (battery, storage capacity, Bluetooth module, etc.) fitted unto CWHDs can influence the accuracy of the data collected by the device. In a study by Haghi et al. , the authors confirm the influence of high-quality components on the performance of CWHDs. Components such as high storage capacity, long wearing battery, Bluetooth, and Wi-Fi compatibility performed better and were more accurate than devices with low quality components.
This paper presents a SLR of 47 papers that focus on the accuracy of the data collected by CWHDs. The results of our analysis showed that the highest number of papers were published in two high-quality databases, namely IEEE (36%) and PUBMED-NCBI (33%). 10% of the papers were published in ScienceDirect, while MDPI and Springer were at 2% respectively. The remaining 13% papers were published in journals such as BMC Public Health and PLOS Medicine. All 47 papers analyzed in the SLR were published by authors from developed countries, with the majority from the USA, followed by Australia and China. None of the authors are from developing countries, including Africa. This points to a gap in studies that focus on the accuracy of the data collected by CWHDs by authors are from developing countries. Based on our analysis, there are three main factors that influence the accuracy of the data collected by CWHDs. These are (i) the tracker and sensor type, (ii) the algorithm used in the CWHD, and (iii) the limitation in the design, energy consumption, and the processing capability of the device.
The study has a number of limitations. Firstly, the search and extraction of sources were based on specific key phrases that include consumer wearable health device and data accuracy. This meant that papers that could potentially have been relevant were excluded from the study because they did not use our search phrases in their keywords. Secondly, the study focused specifically on the accuracy of the data generated from heart rate measurement, PA, and sleep monitoring data. Research papers that focused on the accuracy of CWHDs in general were excluded from the study. The inclusion of such papers could have increased the number of factors beyond the three identified in this study. Finally, the proprietary nature of algorithms used to track and measure bio-sensory and PA data meant that the algorithms reported on in the study were those developed or proposed by researchers.
This study contributes to the number research that focus on the accuracy of the data collected by CWHDs. Given the increasing trend in the use of CWHDs across the globe, and the limited number of studies from developing countries that focus on the topic of accuracy of the data collected by CWHDs, it is imperative that more research is done to better understand the factors that influence their accuracy. The study also has implications for the manufacturers of CWHDs. It is important that the manufacturers of CWHDs take into account the factors that influence data accuracy in the design and development of their devices. This will enable users and healthcare professionals to make meaningful use of the data generated by these device, thus contributing to improved personal well-being and the quality of healthcare service delivery.
- 1.WHO: Fiscal policies for diet and the prevention of noncommunicable diseases. https://www.who.int/dietphysicalactivity/publications/fiscal-policies-diet-prevention/en/. Accessed 06 May 2019
- 3.Wise, J., Hongu, N.: Pedometer, accelerometer, and mobile technology for promoting physical activity. https://extension.arizona.edu/sites/extension.arizona.edu/files/pubs/az1491-2014.pdf. Accessed 17 Aug 2019
- 4.Dunton, G.F., Schneider, M.: Perceived barriers to walking for physical activity. Prevent. Chronic Dis. 3, A116 (2006)Google Scholar
- 9.Bond-Myatt, C.: Health wearables, apps & information protection. https://home.kpmg.com/za/en/home/insights/2016/06/health-wearables--apps---information-protection.html. Accessed 22 Oct 2019
- 11.Kalantarian, H., Lee, S.I., Mishra, A., Ghasemzadeh, H., Liu, J., Sarrafzadeh, M.: Multimodal energy expenditure calculation for pervasive health: a data fusion model using wearable sensors. In: 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), pp. 676–681. IEEE (2013)Google Scholar
- 15.Lang, M.: Beyond fitbit: a critical appraisal of optical heart rate monitoring wearables and apps, their current limitations and legal implications. Albany Law J. Sci. Technol. 28, 39–72 (2017)Google Scholar
- 21.Wise, J., Hongu, N.: Pedometer, accelerometer, and mobile technology for promoting physical activity. https://extension.arizona.edu/sites/extension.arizona.edu/files/pubs/az1491-2014.pdf. Accessed 8 Nov 2019
- 24.Lee, S., Shin, H., Hahm, C.: Effective PPG sensor placement for reflected red and green light, and infrared wristband-type photoplethysmography. In: 18th International Conference on Advanced Communication Technology (ICACT), pp. 556–558. IEEE (2016)Google Scholar
- 29.Matsumura, K., Rolfe, P., Lee, J., Yamakoshi, T.: iPhone 4 s photoplethysmography: which light color yields the most accurate heart rate and normalized pulse volume using the iPhysioMeter application in the presence of motion artifact? PLoS ONE 9, e91205 (2014). https://doi.org/10.1371/journal.pone.0091205CrossRefGoogle Scholar
- 35.Thorup, C.B., Andreasen, J.J., Sørensen, E.E., Grønkjær, M., Dinesen, B.I., Hansen, J.: Accuracy of a step counter during treadmill and daily life walking by healthy adults and patients with cardiac disease. BMJ Open 7, e011742 (2017). https://doi.org/10.1136/bmjopen-2016-011742CrossRefGoogle Scholar
- 43.Ban, D., Kwon, S.: Movement noise cancellation in PPG signals. In: IEEE International Conference on Consumer Electronics (ICCE), pp. 47–48. IEEE (2016)Google Scholar
- 44.Tang, S.D., Goh, Y.S., Wong, M.D., Lew, Y.E.: PPG signal reconstruction using a combination of discrete wavelet transform and empirical mode decomposition. In: 6th International Conference on Intelligent and Advanced Systems (ICIAS), pp. 1–4. IEEE (2016)Google Scholar
- 45.Fereidooni, H., Frassetto, T., Miettinen, M., Sadeghi, A.-R., Conti, M.: Fitness trackers: fit for health but unfit for security and privacy. In: IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 19–24. IEEE (2017)Google Scholar
- 47.Sharma, A., Pande, T., Aroul, P., Soundarapandian, K., Lee, W.: Circuits and systems for energy efficient smart wearables. In: IEEE International Electron Devices Meeting (IEDM), pp. 6.2. 1–6.2. 4. IEEE (2016)Google Scholar
- 48.Huang, J., Badam, A., Chandra, R., Nightingale, E.B.: Weardrive: fast and energy-efficient storage for wearables. In: Annual Technical Conference (USENIX ATC 2015), pp. 613–625 (2015)Google Scholar
- 49.Krachunov, S., et al.: Energy efficient heart rate sensing using a painted electrode ECG wearable. In: Global Internet of Things Summit (GIoTS), pp. 1–6. IEEE (2017)Google Scholar