Keywords

1 Introduction

Falls are a leading cause of fatal and non-fatal injuries for the aging population. Around a third of elderly people 65 years or older fall each year, and a half of those who do fall tend to fall more than once. As age increases, tendency to fall as well as the injuries one might sustain from falling likewise increases [11]. In the United States, fall-related emergency visits are estimated to be around 3 million per year [4].

Seniors’ safety, privacy, independence, economic and personal costs are few other factors that are affected since the fall victim requires continuous \(24\times 7\) assistance. Over 800,000 hospital admissions, 2.8 million injuries and 27,000 deaths have occurred in the past few years because of falls. Healthcare expenditures were approximately $48 million in Alaska out of which $22 million were due to falls of older people [9]. The risk of hospital admissions has been reduced up to 34% with the constant assistance provided to the elderly as per a study conducted by [27].

With improvements in science and technology in the past decade, the ability to provide more advanced \(24\times 7\) protection to elderly people is very important. This can be done by taking advantage of the Internet and its connecting ability to remote devices, which is known as the Internet of Things (IoT). The IoT is defined as the network of devices which can be identified with a unique IP address [17].

Fig. 1.
figure 1

Conceptual overview of Good-Eye

Full size image

In this work we propose Good-Eye, an IoT enabled Edge device that could detect and also predict fall related accidents. The motivation behind Good-Eye is to provide the following:

  1. 1.

    Constant care.

  2. 2.

    Easy to wear accessories convenient to any age.

  3. 3.

    Medical support as per the occurrence of emergencies irrespective of the location.

  4. 4.

    A methodology that promotes that precaution is better than cure.

The conceptual overview of Good-Eye wherein the sensor data are collected by the wearable along with the camera, and the user is notified about the fall condition through the LED, is represented in Fig. 1.

The paper is organized in the following fashion: Sect. 2 gives an overview of prior related work. Section 3 highlights the novel contributions of the Good-Eye System. Section 4 provides a discussion on the relationships between physiological parameters and falls, along with the possible consideration of signals. Section 5 provides the design and working principles of the Good-Eye System. Section 6 provides the implementation and validation of the system while, Sect. 7 provides the conclusions along with directions for future research.

2 Related Research Overview

Automatic fall detection has been a point of interest for decades. Multiple different implementations of an automatic fall detection sensor have been attempted, but these efforts are either restrictive in nature due to limited range, have low sample sizes, or unsatisfactory success rates [19]. Sole use of accelerometer sensors along with other physiological sensor data is proposed in [2]. The use of accelerometers with an RF signal to capture location is proposed in [5] and the angular velocity of 2D information is used to detect falls in [15]. These however, limit the scope of fall detection accuracy as no other physiological and vision parameters are considered. The scope of fall detection using barometric pressure sensors in floors is proposed in [23]. However, this is not an ideal solution as the location of the user is compromised.

The use of vision by using depth camera images with tangential position changes is used for fall detection in [13]. However, this may not be accurate enough as positions of the fall vary. A Raspberry Pi camera based solution for fall detection is proposed in [30]. However, this will affect the mobility of the user as it is location constrained. None of the solutions that are using sensors or camera, predict the fall before the actual event of fall.

Multiple industries have begun to make commercial products that involve automatic fall detection. However, according to commercial reviewers, these devices fail to accurately predict fall detection and often trigger false alarms [21]. These false alarms are so common that even running may trigger automatic fall detection. Also, none of the products provide prediction of fall before the actual event of fall. The top six marketable products along with their drawbacks are provided in Table 1.

Table 1. Wearable products and their drawbacks
Full size table

The Good-Eye System not only can ensure that it is detecting a user’s fall due to the wearable physiological sensors, but it also has fail-safes in the form of a heart rate sensor and camera in case the accelerometer registers a false positive. Additionally, the novel use of a camera provides unique data in that it can photograph the user’s surroundings should they fall, allowing first responders to more accurately find the location of the fallen person.

3 Novel Contributions

With a motivation to provide \(24\times 7\) care to elders with minimal human interaction, we propose the Good-Eye System. This is a combination of both physiological and computer vision systems which ensures warning the user before the event of fall. The Good-Eye System could be used not only to accurately detect falls but also to capture the environment in which a person has fallen, and their internal physiology for analyzing the reason behind the fall. This information could then be used to treat a patient that has fallen more quickly and effectively.

Fig. 2.
figure 2

Broad perspective of Good-Eye

Full size image

The flow of the Good-Eye System is represented schematically in Fig. 2. Here, the physiological data, along with the camera input data are taken from the user and the remote wall respectively, and are analyzed at the edge level processing unit. This processed data are sent to the family and doctor for help depending on the emergency.

The significance of the proposed system include the followings:

  • To provide a system that not only detects the fall but also predicts the fall.

  • To provide an improved method of fall detection that does not involve only accelerometers.

  • To provide vision information with a use of camera to the system.

  • To provide a system which has both wearable and an off-site devices to obtain much higher accuracy.

  • To incorporate the use of other physiological signal data as there is a definite change in the physiology when a person is about to face an accident.

  • To capture the environment before and during the fall to accurately analyze the nature of the fall.

4 Are Falls Related to Physiological Parameters?

Emotions and physiology are intimately connected, as explained in [26]. Falls can be considered as one of the stressors in the human body as they elicit a fight-or-flight response. Under stress conditions, active coping strategies and passive coping conditions occur. Hypertension and tachycardia (an increase in the heart rate) as well as hypotension and bradychardia (a decrease in the heart rate) occur under such conditions. A fall would likely trigger an active coping strategy. Thus, we can likely infer that a fall will cause a physiological response. Stress causes the release of epinephrine and has impact on various physiological parameters depending upon the stressors [22]. Some of the frequent physiological parameters that vary with age and that are affected by stress are discussed in the remaining of this Section.

4.1 Sweat

The sweat glands tend to become less effective due to aging skin [24]. This means that older individuals tend to sweat less, which means it might not be a useful factor to consider in fall detection. Sweat gland output per active gland was significantly lower for those aged 58–67 than it was for those aged 22–24 and 33–40 as stated in [12]. Sweat glands typically have decreased sweat output as one ages.

4.2 Heart Rates and Blood Pressures

Cardiac output decreases linearly at a rate of about 1% per year in normal subjects past the third decade [3]. According to [6] the resting supine diastolic blood pressure for younger men was \(66\pm 6\) and \(62\pm 8\) for older men.

4.3 Temperature

Significant change in the mean body temperature is not observed in the human body over time [29]. Under stress, temperature fluctuations can sometimes be observed depending on the area of the body. The temperature may not vary at the chest or stomach while it varies at the hands and wrist [22, 28].

4.4 Vision

Aging has a significant effect upon vision [18]. This is due to multiple factors, such as spatial contrast sensitivity loss, reduced eyesight in dark situations, and reduced processing potential in terms of visual information.

5 System Level Modeling of Good-Eye

Based on the above factors, the Good-Eye System is proposed in such a way that the behavioral changes in physiological signals are considered not just to detect falls but also to predict falls. The Good-Eye System consists of a wearable that could be placed near the chest portion of the user and an off-site on-wall camera that is connected to the wearable through Internet connectivity. The data collected from the system is processed at the edge device where the parameter analysis and the decision on prediction or detection is made as shown in Fig. 2. While this data is sent to the users as a feedback to notify them about the change, it is also sent to the helpline and cloud for storage.

Fig. 3.
figure 3

Architecture of Good-Eye

Full size image

5.1 Architecture of Good-Eye

The architecture of the Good-Eye System is shown in Fig. 3. Here, the input data are collected from the sensor input unit. These data are processed in the physiological sensor unit and image data unit which process the environmental and orientation change data observed in the on-site and off-site cameras. This data is then compared and analyzed to the set threshold ranges, as explained in Sect. 5.5. The decision is notified to the user by three LED lights.

5.2 The Parameters Considered for Good-Eye System

Modern fall-detection devices have no way to actually detect whether a human is wearing it or if it is being thrown. Additionally, accelerometers can have false alarms due to things like falling into a bed, moving down stairs, etc. So we added new parameters such that we would,

  1. 1.

    be able to detect whether a human is actually using the fall detection device.

  2. 2.

    be able to provide multiple instances of confirmation in order to make the system foolproof.

This would also make it possible to send important data to first responders so humans can verify whether a fall has occurred, such as average heart rate or images of the person’s surroundings in the moments before and after they fell. Factors considered in fall predicting and detecting approaches in Good-Eye System are:

  • Change in the axes of the accelerometer.

  • Sudden change in the heart rate variability of a person compared to the resting heart rate.

  • Having an on-site camera in the wearable to measure the change in orientation, to analyze the intensity of fall and provide certain care as per the emergency.

  • Having an off-site wall-mounted camera in the surrounding space of a person, enables continuous person detection and tracking to provide proper feedback.

5.3 Design Flow

As the Good-Eye System as a whole comprises of an additional off-site camera, the flow of the system is explained as follows.

On-site Design Flow of Good-Eye System. The physiological sensor data along with the environmental capture are obtained at the on-site portion of the Good-Eye System. The flow of the design is as shown in Fig. 4. Accelerometer sensor changes are considered as a prime source for the system to start running so as to respect the privacy of the user. Whenever an accelerometer reading change is detected, the sudden spike in heart rate variability is checked along with the change in the camera’s orientation. The moment there is an observed change in the accelerometer, the camera starts capturing the surroundings of the user.

Fig. 4.
figure 4

Design flow of Good-Eye

Full size image

Even if no change in the camera’s orientation is observed, the data obtained from the camera is sent to the parameter analysis unit where the range comparisons are done as explained in brief in Sect. 5.4. This is done in order to maintain a movement log of the user. When there is no sudden spike detected in heart rate variability, the sensor is again taken to an idle state. From the parameter analyses unit, the decision of fall prediction or detection is done as explained in Sect. 5.5. The decisions are sent to family or helpline based on the level of emergency.

Off-site Design Flow of Good-Eye System. The off-site or the on-wall camera plays a major role in fall prediction and detection here with a unique design flow as shown in Fig. 5. This is proposed considering facts which include the user forgetting to have an on-site unit. This on-wall camera starts working the moment a motion is detected in the room. This will then continuously start detecting and tracking the person so as to maintain the privacy of other people in the environment. When there is a sudden change in the movement of the user, instead of giving out false positive results, this will connect to the wearable to collect the physiological sensor data. Based on the physiological data and the on-site camera’s orientation, the parameter analyses along with the decision of prediction or detection are made.

Fig. 5.
figure 5

On-wall camera design flow of Good-Eye.

Full size image

In the unfortunate case of the user not wearing an on-site wearable or when the on-wall camera isn’t able to obtain the physiological sensor data from the wearable, the notification of “No Movement Detected” is sent to the guardian and to the doctors. This will also alert the user as a reminder to wear the wearable. With this notification, not only false positive cases will be reduced but also the incidents of stroke can be quickly addressed instead of waiting for the user to manually ask for help.

5.4 Parameter Analysis Unit

Parameter Data Acquisition. To incorporate the heart rate variability into the overall fall detection program, the Good-Eye System checks if there is a sudden spike in heart rate every few milliseconds, as the human body in such accidents experiences either a higher heart rate or a lower heart rate [26]. It is observed that the maximum heart rate in older men was lower (at around \(162 \pm 9\) beats/min) than the maximum heart rate in younger men (\(191 \pm 11\) beats/min) according to [6]. Therefore the heart rate variability to the resting heart rate of every individual is considered as the threshold.

A fall is dependent upon a period of weightlessness followed by a large impact that increases the acceleration of the y-axis of an accelerometer by around 3 g’s [10]. The accelerometer would constantly read the x, y, and z values of the g-force exerted upon a human being wearing the device. If the y value of the g-force exceeded ±3 g’s, the accelerometer would pass the threshold required to detect a fall.

The camera orientation sensor is implemented using a method that estimates orientation based on sequential Bayesian filtering [14]. The center of the frame is considered, with x and y axes. The respective R, G, and B values are calculated and the distances from each frame are stored for each pixel value. These pixel values are compared to the threshold in order to decide if the event is fall or not. A picture will be snapped when the accelerometer passes the threshold value, and another picture will be snapped when the accelerometer’s values return to a new resting position.

5.5 Fall Prediction and Detection Unit

The analysis for the decision if it is a fall or not a fall is based on considering the change in accelerometer data, change in heart rate variability and change in orientation of the camera. It is represented in Table 2.

Table 2. Analyses for fall prediction & detection
Full size table

5.6 Working Principle of the Proposed Good-Eye Model

The methodology that is involved in the camera of the Good-Eye System is explained as follows for two different Frames through Algorithm 1.

figure a

Even if a fall is not seen as occurring but the other two parameters (heart rate and accelerometer) are reached, the camera can send the last few seconds of data to a first responder, who can determine whether a fall has truly occurred. This makes the system more reliable, and prevents waste of resources. The proposed model is again represented with depth image data in Fig. 6.

Fig. 6.
figure 6

Proposed model of Good-Eye.

Full size image

6 Implementation and Validation

6.1 Implementation

Good-Eye System. The microcontroller where the processing is performed is connected to a tri-axial accelerometer, heartbeat sensor and a camera, as shown in Fig. 7.

Fig. 7.
figure 7

Implementation of Good-Eye

Full size image

The algorithm involved works by taking both heart rate and accelerometer data simultaneously. It stores the previous data as a means to compare between milliseconds of time. Once the accelerometer’s y-axis has a change of more than 2 g’s, the heart rates of the user is immediately compared along with the orientation check in camera. If the heart rate of the user has spiked by at least 10 bpm, an alarm triggers. The continued readings from the accelerometer, camera and heart rate are represented in Fig. 8.

Fig. 8.
figure 8

Results of Good-Eye.

Full size image

Good-Eye Connectivity. The continuous data collected from the Good-Eye system is stored in an open source cloud IoT analytics platform, as shown in Fig. 9. The data stored here can be accessed by the user depending on the requirement.

Fig. 9.
figure 9

Signal analysis Good-Eye.

Full size image
Fig. 10.
figure 10

Validation of Good-Eye.

Full size image
Table 3. Comparison with state of the art research
Full size table

6.2 Validation

Event of Fall Vs Event of Sitting. In order to verify the model’s algorithm, the validation of the model has been done with the already published Person Falling Dataset [31]. This has 144 instances of data where 6 subjects had performed sitting and falling separately. The process of sitting and falling has been captured by a depth camera in this dataset.

When these instances were fed to the Good-Eye model, the change in differentiating the sitting to falling has been found with an approximate accuracy of 95%. The differentiation is represented starting from taking the x and y axes, as stated in Algorithm 1 for making decision of fall as stated in Table 2. The event of Falling vs. event of Sitting is represented in Fig. 10.

The methodology implemented in the Good-Eye System is validated with state of the art research and wearables and is provided in Table 3.

7 Conclusions and Future Research

A way to reliably detect falls is of utmost importance for the health of elderly people. The device proposed here not only uses an accelerometer but also other physiological sensor data that enhance the usage of a fall detection device and make it more accurate. The camera inputs and physiological sensor data inputs are independently significant in relaying data and verifying that the device is accurately predicting and detecting the event of fall. This device may greatly increase the importance of fall detection devices as it manages to provide privacy, and convenience to prediction of fall to the user.

The relationship with the physiological data to falls has been explained above and additionally, incorporating all possible sensor data and increasing the model’s dataset to further improve the Good-Eye System is our future research.