A scalable semantic framework for IoT healthcare applications

Abstract

IoT-based systems for early epidemic detection have not been investigated yet in the research. The state-of-the art in sensor technology and activity recognition makes it possible to automatically detect activities of daily living (ADL). Semantic reasoning over ADLs can discover anomalies and symptoms for disorders, hence diseases and epidemics. However, semantic reasoning is computationally rather expensive and therefore unusable for real-time monitoring in large scale applications, like early epidemic detection. To overcome this limitation, this paper proposes a new scalable semantic framework based on several semantic reasoning techniques that are distributed over a semantic middleware. To reduce the number of events to process during the semantic reasoning, a complex event processing (CEP) engine is used to detect abnormal events in ADL and to generate the associated symptom indicators. To demonstrate real-time detection and scalability, the proposed framework integrates a new extension of ADLSim, a discrete event simulator that simulates long-term sequences of ADL.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Abdallah ZS et al (2018) Activity recognition with evolving data streams: a review. ACM Comput Surv (CSUR) 51(4):71

    Article  Google Scholar 

  2. Alaya MB et al (2014) OM2M: extensible ETSI-compliant M2M service platform with self-configuration capability. Procedia Comput Sci 32:1079–1086

    Article  Google Scholar 

  3. Alessandra M et al (2016) Recognition of daily gestures with wearable inertial rings and bracelets. Sensors 16(8):1341

    Article  Google Scholar 

  4. Aran O et al (2016) Anomaly detection in elderly daily behavior in ambient sensing environments. In: International workshop on human behavior understanding. Springer, pp 51–67

  5. Avci A et al (2010) Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: 23rd international conference on architecture of computing systems (ARCS), pp 1–10

  6. Barbieri DF et al (2010) C-sparql: a continuous query language for rdf data streams. Int J Seman Comput 4(01):3–25

    Article  Google Scholar 

  7. Barret A et al (2014) Surveillance for outbreaks of gastroenteritis in elderly long-term care facilities in France, November 2010 to May 2012. Eurosurveillance 19(29):20859

    Article  Google Scholar 

  8. Bispo KA, Rosa NS, Cunha PR (2015) Sitrus: semantic infrastructure for wireless sensor networks. Sensors 15(11):27436–27469

    Article  Google Scholar 

  9. Curry E (2004) Message-oriented middleware. In: Middleware for communications. Workshop on distributed event-based systems (DEBS’04), pp 1–28

  10. Donnelly K (2006) Snomed-ct: the advanced terminology and coding system for ehealth. Stud Health Technol Inform 121:279

    Google Scholar 

  11. Eckert M et al (2011) A cep babelfish: languages for complex event processing and querying surveyed. In: Reasoning in event-based distributed systems. Springer, pp 47–70

  12. espertech (2006) Esper cep engine. http://www.espertech.com/. Accessed 16 May 2019

  13. Fernández-Llatas C et al (2011) Behaviour patterns detection for persuasive design in nursing homes to help dementia patients. In: Engineering in medicine and biology society, EMBC, 2011 annual international conference of the IEEE. IEEE, pp 6413–6417

  14. Ghayvat H et al (2018) Smart home based ambient assisted living: recognition of anomaly in the activity of daily living for an elderly living alone. In: 2018 IEEE (I2MTC). IEEE, pp 1–5

  15. Goller J et al (2004) Long-term features of norovirus gastroenteritis in the elderly. J Hosp Infect 58(4):286–291

    Article  Google Scholar 

  16. Gray J et al (1987) An outbreak of gastroenteritis in a home for the elderly associated with astrovirus type 1 and human calicivirus. J Med Virol 23(4):377–381

    MathSciNet  Article  Google Scholar 

  17. Haller A et al (2018) The modular ssn ontology: a joint w3c and ogc standard specifying the semantics of sensors, observations, sampling, and actuation. Semant Web 10(01):09–32

    Article  Google Scholar 

  18. Halvorsrud J, Örstavik I (1980) An epidemic of rotavirus-associated gastroenteritis in a nursing home for the elderly. Scand J Infect Dis 12(3):161–164

    Article  Google Scholar 

  19. Hochgatterer A et al (2011) Requirements for a behaviour pattern based assistant for early detection and management of neurodegenerative diseases. In: 2011 5th international conference on PervasiveHealth. IEEE, pp 346–353

  20. Hoque E, Dickerson RF, Preum SM, Hanson M, Barth A, Stankovic JA (2015) Holmes: a comprehensive anomaly detection system for daily in-home activities. In: International conference on distributed computing in sensor systems (DCOSS). IEEE

  21. Hsu HH, Chen CC (2010) Rfid-based human behavior modeling and anomaly detection for elderly care. Mobile Inf Syst 6(4):341–354

    Article  Google Scholar 

  22. Katz S et al (1970) Progress in development of the index of adl. Gerontologist 10:20–30

    Article  Google Scholar 

  23. Khrouf H et al (2016) Waves: big data platform for real-time rdf stream processing. In: SR workshop at ISWC

  24. Kibbe WA et al (2014) Disease ontology 2015 update: an expanded and updated database of human diseases for linking biomedical knowledge through disease data. Nucleic Acids Res 43:D1071–D1078

    Article  Google Scholar 

  25. Kim J, Lee JW (2014) OpenIoT: an open service framework for the Internet of Things. In: 2014 IEEE World forum on Internet of Things (WF-IoT). IEEE

  26. Kirk MD, Veitch MG, Hall GV (2010) Gastroenteritis and food-borne disease in elderly people living in long-term care. Clin Infect Dis 50(3):397–404

    Article  Google Scholar 

  27. Krishnan NC, Cook DJ (2014) Activity recognition on streaming sensor data. Pervasive Mobile Comput 10:138–154

    Article  Google Scholar 

  28. Kristiansen S et al (2016) Smooth and crispy: integrating continuous event proximity calculation and discrete event detection. In: Proceedings of the 10th ACM international conference on distributed and event-based systems. ACM, pp 153–160

  29. Kristiansen S et al (2018) An activity rule based approach to simulate adl sequences. IEEE Access 6:12551–12572

    Article  Google Scholar 

  30. Kwok T et al (2002) Parallel fuzzy c-means clustering for large data sets. In: European conference on parallel processing. Springer, pp 365–374

  31. Lampoltshammer AA, Thomas J (2014) Use of local intelligence to reduce energy consumption of wireless sensor nodes in elderly health monitoring systems. Sensors 14(3):4932–4947

    Article  Google Scholar 

  32. Le-Phuoc D et al (2011) A native and adaptive approach for unified processing of linked streams and linked data. In: International semantic web conference. Springer, pp 370–388

  33. Lohr C, Tanguy P, Kerdreux J (2015) xaal: a distributed infrastructure for heterogeneous ambient devices. J Intell Syst 24(3):321–331

    Article  Google Scholar 

  34. Lotfi A et al (2012) Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour. J Ambient Intell Hum Comput 3(3):205–218

    Article  Google Scholar 

  35. Margara A, Cugola G, Collavini D, Dell’Aglio D (2018) Efficient temporal reasoning on streams of events with dotr. In: Gangemi A, Navigli R, Vidal ME, Hitzler P, Troncy R, Hollink L, Tordai A, Alam M (eds) The semantic web. Springer International Publishing, New York, pp 384–399

    Google Scholar 

  36. Meditskos G, Dasiopoulou S, Kompatsiaris I (2016) MetaQ: a knowledge-driven framework for context-aware activity recognition combining SPARQL and OWL 2 activity patterns. Pervasive Mobile Comput. https://doi.org/10.1016/j.pmcj.2015.01.007

    Article  Google Scholar 

  37. Meng L, Miao C, Leung C (2017) Towards online and personalized daily activity recognition, habit modeling, and anomaly detection for the solitary elderly through unobtrusive sensing. Multimed Tools Appl 76(8):10779–10799

  38. Mohammed O et al (2012) Building a diseases symptoms ontology for medical diagnosis: an integrative approach. In: 2012 international conference on future generation communication technology (FGCT). IEEE, pp 104–108

  39. Noury N et al (2016) Building a spatial-temporal index to detect the global pattern deviations in daily activities of elderly subjects. In: 18th international conference on e-health networking, applications and services. IEEE

  40. Novák M et al (2012) Unobtrusive anomaly detection in presence of elderly in a smart-home environment. In: ELEKTRO, 2012. IEEE, pp 341–344

  41. Onofri L et al (2016) A survey on using domain and contextual knowledge for human activity recognition in video streams. Expert Syst Appl 63:97–111

    Article  Google Scholar 

  42. Perriot B et al (2014) Characterization of physical activity in copd patients: validation of a robust algorithm for actigraphic measurements in living situations. IEEE J Biomed Health Inform 18(4):1225–1231

    Article  Google Scholar 

  43. Pierleoni P et al (2015) A high reliability wearable device for elderly fall detection. IEEE Sens J 15(8):4544–4553

    Article  Google Scholar 

  44. Pires I et al (2016) From data acquisition to data fusion: a comprehensive review and a roadmap for the identification of activities of daily living using mobile devices. Sensors 16(2):184

    Article  Google Scholar 

  45. Quilitz B, Leser U (2008) Querying distributed rdf data sources with sparql. In: European semantic web conference. Springer, pp 524–538

  46. Razzaque MA et al (2016) Middleware for Internet of Things: a survey. IEEE Internet of Things J 3(1):70–95

    Article  Google Scholar 

  47. Rodríguez-Molina J et al (2013) SMArc: a proposal for a smart, semantic middleware architecture focused on Smart City energy management. Int Distrib Sens Netw 9(12):560418

  48. Ryoo MS, Aggarwal JK (2006) Recognition of composite human activities through context-free grammar based representation. Comput Vis Pattern Recogn IEEE Comput Soc Conf 2:1709–1718

    Google Scholar 

  49. Sareen S et al (2017) Secure internet of things-based cloud framework to control zika virus outbreak. Int J Technol Assess Health Care 33(1):11–18

    Article  Google Scholar 

  50. Sareen S, Sood SK, Gupta SK (2016) Iot-based cloud framework to control ebola virus outbreak. J Ambient Intell Hum Comput 18:1–18

    Google Scholar 

  51. Schriml LM (2018) Symptom ontology. https://www.ebi.ac.uk/ols/ontologies/symp. Accessed 26 Feb 2019

  52. Schriml LM et al (2011) Disease ontology: a backbone for disease semantic integration. Nucleic Acids Res 40(D1):D940–D946

    Article  Google Scholar 

  53. Serna A et al (2007) Modeling the progression of Alzheimer’s disease for cognitive assistance in smart homes. User Model User-Adapt Interact 17(4):415–438

    Article  Google Scholar 

  54. Strausbaugh LJ et al (2003) Infectious disease outbreaks in nursing homes: an unappreciated hazard for frail elderly persons. Clin Infect Dis 36(7):870–876

    Article  Google Scholar 

  55. Thakar AT, Pandya S (2017) Survey of iot enables healthcare devices. In: 2017 international conference on computing methodologies and communication (ICCMC). IEEE

  56. Ukil A et al (2016) Iot healthcare analytics: the importance of anomaly detection. In: 2016 IEEE 30th international conference on advanced information networking and applications (AINA). IEEE, pp 994–997

  57. van Asten L et al (2011) Unspecified gastroenteritis illness and deaths in the elderly associated with norovirus epidemics. Epidemiology 22:336–343

    Article  Google Scholar 

  58. Van Kasteren T et al (2008) Accurate activity recognition in a home setting. In: Proceedings of the 10th international conference on Ubiquitous computing. ACM, pp 1–9

  59. Varatharajan R et al (2017) Wearable sensor devices for early detection of Alzheimer disease using dynamic time warping algorithm. In: Cluster computing, pp 1–10

  60. Wolf P et al (2010) openaal-the open source middleware for ambient-assisted living (aal). In: AALIANCE conference, Malaga, Spain, pp 1–5

  61. Wu J et al (2018) Sensor fusion for recognition of activities of daily living. Sensors 18(11):4029

    Article  Google Scholar 

  62. Yu M et al (2012) A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment. IEEE Trans Inf Technol Biomed 16(6):1274–1286

    Article  Google Scholar 

  63. Zgheib R et al (2017) A flexible architecture for cognitive sensing of activities in ambient assisted living. In: IEEE 26th international conference on enabling technologies: WETICE. IEEE, pp 284–289

  64. Zgheib R et al (2019) Semantic middleware architectures for iot healthcare applications. In: Enhanced living environments. Springer, pp 263–294

  65. Ziaeefard M, Bergevin R (2015) Semantic human activity recognition: a literature review. Pattern Recogn 48(8):2329–2345

    Article  Google Scholar 

  66. Ziakas PD et al (2016) Prevalence and impact of clostridium difficile infection in elderly residents of long-term care facilities, 2011: a nationwide study. Medicine 95(31):e4187

    Article  Google Scholar 

Download references

Acknowledgements

This work has been funded by the COST Action IC 1303 AAPELE; the Research Council of Norway through the CESAR project (Grant no. 250239/O70); and the project IoTSec from the french Region Limousin

Author information

Affiliations

Authors

Corresponding author

Correspondence to Rita Zgheib.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Evaluation of epidemic simulation in ADLSim

Appendix 1: Evaluation of epidemic simulation in ADLSim

Fig. 6
figure6

Results from experiments set 1

To obtain reliable results from experiment sets 2 and 3, it is necessary to first ensure that our simulation models and SI queries work as intended. This section presents a set of experiments designed for this purpose, and discusses the results.

Experiment goals and design

The main goals of these experiments are to show that (1) the epidemic model behaves as intended, (2) the output from the SI queries correspond to the impact of the epidemic model, and (3) that the results vary realistically with the values of key parameters (listed in Table 1).

We conduct two experiments, Experiment 1.1 and 1.2. Each experiment includes one sub-experiment per combinations of the three ED and three DIP parameters shown in Table 1. This enables us to study epidemics ranging from very mild to very severe. We conduct each sub-experiment three times with different seeds and which results in \(3\times 3\times 3=27\) sub-experiments per experiment. In Experiment 1.1, we use highly conservative thresholds for SI event emissions. SI values are emitted only when they change to “very low” or “very high”. In Experiment 1.2, we increase the sensitivity of the queries to emit events whenever the value changes from “normal” to any other value. This way we can study the impact on the number of events emitted for further semantic reasoning. In addition, we generate every simulated day 24-h SI statistics regarding the entire apartment complex.

Results and discussions

To get a comprehensive understanding of our results, we present statistics across all runs and parameter values as well as plots that provide a deeper look at individual runs for a sub-set of the results.

Figure 6 presents the results from experiment set 1. Graphs (a), (b), (c), and (d) present results from the first run in Experiment 1.1 with ED = 30 and DIP = 1.5% (left) and 3% (right). The epidemic begins at simulated Day 20. Graphs (a) and (b) show the complex-wide 24-h SI statistics (y-axes) for all simulated days (x-axes), and Graphs (c) and (d) show the number of apartments (y-axes) with SI values “very low” and “very high” for all simulated days (x-axes). Graphs (e) and (f) show statistics across all runs for all parameter values from Experiment 1.1 and 1.2. They show the average (lines) and SD (error bars) of the number of emitted SI events (left y-axes) across all three runs per combination of DIP (x-axes) and ED (with different line colors). Note the differences in the left y-axes in Graphs (e) and (f). The box plots show the average number of the infected individual across all three runs (right y-axes) per combination of DIP (x-axes) and ED (with different box colors).

The results in Graphs (a), (b), (c), and (d) show a clear impact on the complex-wide SI of the simulated epidemic, namely that the SI values during the epidemic have significantly larger absolute values during the epidemic as well as before and after the epidemic. This effect is also clearly larger with DIP = 3% than with DIP = 1.5%. We also see that only the queries affected by the disease are impacted, i.e., the bottom five queries. Notice that although the epidemic lasts from Day 20 to Day 50, the main impact on the complex-wide SI values is found somewhat later. Three factors contribute to this. The most important factor is that a certain amount of time after the epidemic onset is required for a sufficient number of individuals to be infected to have a significant impact on the aggregate activity magnitudes in the whole apartment complex. Second, due to the gradual onset of symptoms, 3 days pass after an individual is infected until the full effect of the symptoms is manifested. Finally, at the end of the epidemic period, a certain number of individuals are still symptomatic for some time until recovery, which shows up in the SI queries. There are occasional, abnormal SI values before and after the epidemic explained by natural variations in behavior. These are trivially exposed as false positives by jointly considering a larger set of samples and/or samples from multiple SI queries. These results thus indicate that the complex-wide SI queries respond correctly to the simulated epidemic.

The results in graphs (c) and (d) show that also the apartment-specific SI values are as expected, i.e., the period with the epidemic results in a significantly larger number of apartments with indications of decreased activity levels reduced intake of food, and increased toileting. The effect is again clearly more pronounced with the largest DIP. We can thus conclude that also the apartment-specific SI work as intended.

Figure 6e, f show that the least restrictive SI thresholds (“high” and “low”, Fig. 6e) yield an order of magnitude more SI events than the most restrictive thresholds (“very high” and “very low”, Fig. 6f). This has a significant impact on the number of resources required for subsequent analysis. We also see that the epidemic is clearly detectable by visual inspection in graphs (a) and (b) even with the most restrictive thresholds. It is therefore advisable in this scenario to employ highly restrictive SI thresholds to minimize the emission rate of SI events. The box plots finally show that the number of infected apartments agrees well with Eq. 1, and that these numbers are proportional to the number of emitted SI. Thus, we find that the SI thresholds and the epidemic models work as intended.

Summary and key insights: Results from Experiment Set 1 demonstrate that the epidemic model and all aspects of the SI queries work as intended. We also find that using the most restrictive SI emission thresholds can significantly reduce the number of events emitted and thus save considerable amounts of event processing resources while preserving clearly detectable changes in SI values during the epidemic. We therefore decide to use these SI emission thresholds in experiment set 2.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zgheib, R., Kristiansen, S., Conchon, E. et al. A scalable semantic framework for IoT healthcare applications. J Ambient Intell Human Comput (2020). https://doi.org/10.1007/s12652-020-02136-2

Download citation

Keywords

  • Simulated activities
  • Middleware
  • Ontologies
  • Epidemic detection