Keywords

1 Introduction

The increase of the average lifespan across the world has been accompanied by an unprecedented upsurge in the occurrence of dementia, with high socio-economic costs, reaching 818 billion US dollars worldwide, in 2015. Nevertheless, its prevalence is increasing as the number of people aged 65 and older with Alzheimer’s disease may nearly triple by 2050, from 46.8 million to 131 million people around the world, the majority of which, living in an institution [1].

Assistive technology is called upon to contribute to cognitive and physical state improvement and to cost reduction, by prolonging independent living. However, not many existing solutions follow a holistic approach, in order to achieve greater impact. Pervasive technology solutions have already been employed in several ambient environments, either homes or clinics, but most of them focus on a single domain to monitor, using only a single or a few devices. Such applications include wandering behavior prevention with geolocation devices, monitoring physical activity, sleep, medication and performance in daily chores [2, 3].

In order to assess cognitive state, activity modelling and recognition appears to be a critical task, common amongst existing assistive technology. OWL has been widely used for modelling human activity semantics, reducing complex activity definitions to the intersection of their constituent parts. In most cases, activity recognition involves the segmentation of data into snapshots of atomic events, fed to the ontology reasoner for classification. Time windows [4] and slices [5] provide background knowledge about the order or duration [6] of activities are common approaches for segmentation. In this paradigm, ontologies are used to model domain information, whereas rules, which are widely embraced to compensate for OWL’s expressive limitations, aggregate activities, describing the conditions that drive the derivation of complex activities e.g. temporal relations.

The proposed system, Dem@Home, complements these developments by providing a holistic approach for context-aware monitoring and personalized care of dementia at homes, prolonging independent living. To begin with, the system integrates a wide range of sensor modalities and high-level analytics to support accurate monitoring of all aspects of daily life including physical activity, sleep and activities of daily living (ADLs), based on a service-oriented middleware [7]. After integrating them in a uniform knowledge representation format, Dem@Home employs semantic interpretation techniques to infer complex activity recognition from atomic events and highlight clinical problems. Specifically, it follows a hybrid reasoning scheme, using DL reasoning for activity detection and SPARQL to extract clinical problems. Utterly, Dem@Home presents information to applications tailored to clinicians, and patients, endorsing technology-aided clinical interventions to improve care. Dem@Home has been deployed and evaluated in four home pilots showing positive results.

2 The Dem@Home Framework

Dem@Home proposes a multidisciplinary approach that brings into effect the synergy of the latest advances in sensor technologies addressing a multitude of complementary modalities, large-scale fusion and mining, knowledge representation and intelligent decision-making support. In detail, the framework integrates several heterogeneous sensing modalities, such as physical activity and sleep sensor measurements, combined input from lifestyle sensors and higher-level image analytics, providing their unanimous semantic representation and interpretation.

The current selection of sensors, shown on Fig. 1, is comprised of proprietary, low-cost devices, originally intended for lifestyle monitoring, repurposed to a medical context. Each device is integrated by using dedicated modules that wrap their respective API, retrieve data and process them accordingly to generate atomic events from sensor observations e.g. through aggregation. In the case of image data, computer vision techniques are employed to extract information about humans performing activities, such as opening the fridge, holding a cup or drinking [8]. All atomic events and observations are mapped to a uniform semantic representation for interoperability and stored to Dem@Home’s Knowledge Base.

Fig. 1.
figure 1

Dem@Home architecture, sensors and clinical applications

Full size image

To obtain a more comprehensive image of an individual’s condition and its progression, driving clinical interventions, Dem@Home employs semantic interpretation to perform intelligent fusion and aggregation of atomic, sensor events to complex ones and identify problematic situations. It does so, with a hybrid combination of OWL 2 reasoning and SPARQL queries.

Dem@Home provides a simple pattern for modelling the context of complex activities, in other words, the semantics for activity recognition. Each activity context is described through class equivalence axioms that link them with lower-level observations of domain models (Fig. 1). The instantiation of this pattern is used by the underlying reasoner to classify context instances, generated during the execution of the protocol, as complex activities. The instantiation involves linking IADLs with context containment relations through class equivalence axioms. For example, given that the activity \( \tt{MakeTea} \) involves the observations \( \tt{TurnKettleOn} \), \( \tt{CupMoved} \), \( \tt{KettleMoved} \), \( \tt{TeaBagMoved} \), \( \tt{TurnKettleOff} \), \( \tt{HoldingCup} \), its semantics are defined as:

$$ \begin{aligned} MakeTea \equiv & Context\,{ \sqcap }\,\exists contains. TurnKettleOn\,{ \sqcap }\,\exists contains.CupMoved \\ \text{ } & { \sqcap }\,\exists contains.KettleMoved\,{ \sqcap }\,\exists contains. TeaBagMoved \\ \text{ } & { \sqcap }\,\exists contains.TurnKettleOff\,{ \sqcap }\,\exists contains.HoldingCup \\ \end{aligned} $$

According to clinical experts involved in the development of Dem@Home, highlighting problematic situations next to the entire set of monitored activities and metrics would further facilitate and accelerate clinical assessment. Dem@Home uses a set of pre-defined rules (expressed in SPARQL) with numerical thresholds that clinicians can adjust and personalize to each of the individuals in their care. Furthermore, each analysis is invoked for a period of time allowing different thresholds for different intervals e.g. before and after a clinical intervention. Problematic situations supported so far regard night sleep (short duration, many interruptions, too long to fall asleep), physical activity (low daily activity totals), missed activities (e.g. skipping daily lunch) and reoccurring problems (problems for consecutive days).

Dem@Home was evaluated in four home installations, in the residences of individuals living alone, clinically diagnosed with mild cognitive impairment or mild dementia, and maintained for four months. Complex activity recognition evaluated over the first month, achieved an average of 84 % recall and 88 % precision for daily tasks. Losses are attributed to the openness and uncertainty in the environment, e.g. when interleaving activities. However, with Dem@Home supporting clinical interventions, significant improvement was found in post-pilot clinical assessment in multiple domains, such as increase in physical condition, sleep duration decreased insomnia, utterly bringing about improvement in mood and cognitive state.