Physiological Wireless Sensor Network for the Detection of Human Moods to Enhance Human-Robot Interaction

  • Francesco Semeraro
  • Laura FioriniEmail author
  • Stefano Betti
  • Gianmaria Mancioppi
  • Luca Santarelli
  • Filippo Cavallo
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 544)


Although it is already possible to issue utility services that use robots, these are still not perceived by society as capable of actually delivering them. One of the main motivations is the lack of a human-like behaviour in the interaction with the user. This is displayed both at physical and cognitive level. This work investigates the optimal sensor configuration in the recognition of three different moods, as it surely represents a crucial element in the enhancement of the human-robot interaction. Mainly focusing towards a future application in the field of assistive robotics, electrocardiogram, electrodermal activity and electroencephalographic signal were used as main informative channels, acquired through a wireless wearable sensor network. An experimental methodology was built to induce three different emotional states by means of social interaction. Collected data were classified with six supervised machine learning approaches, namely decision tree, induction rules and lazy, probabilistic and function-based classifiers. The results of this work revealed that the optimal configuration of sensors which maximizes the trade-off between accuracy and obtrusiveness is the one surveying cardiac and skin activities. This sensor configuration reached an accuracy of 87.07% in the best case.


Mood detection Physiological sensors MIP Social interaction 



This work was supported by the ACCRA Project, founded by the European Commission—Horizon 2020 Founding Programme (H2020-SCI-PM14-2016) and National Institute of Information and Communications Technology (NICT) of Japan under grant agreement No. 738251.


  1. 1.
    Cavallo F, Semeraro F, Fiorini L, Magyar G, Sinčák P, Dario P (2018) Emotion modelling for social robotics applications: a review. J Bionic Eng 15CrossRefGoogle Scholar
  2. 2.
    Isen AM (2000) Some perspectives on positive affect and self-regulation 11:184–187Google Scholar
  3. 3.
    Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective\physiological state. IEEE Trans Pattern Anal Mach Intell 23:1175–1191CrossRefGoogle Scholar
  4. 4.
    Bruno B, Mastrogiovanni F, Sgorbissa A (2013) Functional requirements and design issues for a socially assistive robot for elderly people with mild cognitive impairments. In: 22nd IEEE international symposium on robot and human interactive communication, pp 768–773Google Scholar
  5. 5.
    Koelstra S, Patras I (2013) Fusion of facial expressions and EEG for implicit affective tagging. Image Vis ComputGoogle Scholar
  6. 6.
    Betti S et al (2017) Evaluation of an integrated system of wearable physiological sensors for stress monitoring in working environments by using biological markers. IEEE Trans Biomed EngGoogle Scholar
  7. 7.
    Schumm J, Bächlin M, Setz C, Arnrich B, Roggen D, Tröster G (2008) Effect of movements on the electrodermal response after a startle event. In: Proceedings of 2nd international conference on pervasive computing technologies for healthcare, PervasiveHealth, 2008, pp 315–318Google Scholar
  8. 8.
    Chen M, Ma Y, Song J, Lai C-F, Hu B (2016) Smart clothing: connecting human with clouds and big data for sustainable health monitoring. Mob Netw Appl 21:825–845CrossRefGoogle Scholar
  9. 9.
    Nardelli M, Valenza G, Greco A, Lanata A, Scilingo EP (2014) Arousal recognition system based on heartbeat dynamics during auditory elicitationGoogle Scholar
  10. 10.
    Henriques R, Paiva A, Antunes C (2013) Accessing emotion patterns from affective interactions using electrodermal activity. In: 2013 humaine association conference on affective computing and intelligent interaction, pp 43–48. IEEEGoogle Scholar
  11. 11.
    Khezri M, Firoozabadi M, Sharafat AR (2015) Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals. Comput Methods Programs BiomedGoogle Scholar
  12. 12.
    Roberts NA, Tsai JL, Coan JA (2007) Emotion elicitation using dyadic interaction tasks. Handbook of emotion elicitation and assessment, pp 106–123Google Scholar
  13. 13.
    Koelstra S, Uhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3:18–31CrossRefGoogle Scholar
  14. 14.
    Kim J-H, Roberge R, Powell JB, Shafer AB, Williams WJ (2013) Measurement accuracy of heart rate and respiratory rate during graded exercise and sustained exercise in the heat using the Zephyr BioHarness TM. Int J Sports Med 34:497–501Google Scholar
  15. 15.
    Burns A, Greene BR, McGrath MJ, O’Shea TJ, Kuris B, Ayer SM, Stroiescu F, Cionca V (2010) SHIMMERTM—a wireless sensor platform for noninvasive biomedical research. IEEE Sens J 10:1527–1534CrossRefGoogle Scholar
  16. 16.
    Salabun W (2014) Processing and spectral analysis of the raw EEG signal from the MindWave. Prz Elektrotechniczny 169–173Google Scholar
  17. 17.
    Harmon-Jones E, Amodio DM, Zinner LR (2007) Social psychological methods of emotion elicitation. Handbook of emotion elicitation and assessment, pp 91–105Google Scholar
  18. 18.
    Bradley M, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25:49–59CrossRefGoogle Scholar
  19. 19.
    Kreibig S (2010) Autonomic nervous system activity in emotion: a review. Biol Psychol 3:394–421CrossRefGoogle Scholar
  20. 20.
    Lippman N, Stein KM, Lerman BB (1994) Comparison of methods for removal of ectopy in measurement of heart rate variability. Am J Physiol 267:H411–H418CrossRefGoogle Scholar
  21. 21.
    Acharya UR, Joseph KP, Kannathal N, Lim CM, Suri JS (2006) Heart rate variability: a review. Med Biol Eng Comput 44:1031–1051CrossRefGoogle Scholar
  22. 22.
    Mali B, Zulj S, Magjarevic R, Miklavcic D, Jarm T (2014) Matlab-based tool for ECG and HRV analysis. Biomed Signal Process Control 10:108–116CrossRefGoogle Scholar
  23. 23.
    Boucsein W (2012) Electrodermal activity. Springer Science + Business Media, LLCCrossRefGoogle Scholar
  24. 24.
    Lochner K, Eid M (2016) Successful emotions: how emotions drive cognitive performance. Successful emotions: how emotions drive cognitive performance, pp 43–67CrossRefGoogle Scholar
  25. 25.
    Wang H-M, Huang S-C (2012) SDNN/RMSSD as a surrogate for LF/HF: a revised investigation. Model Simul Eng 2012:1–8CrossRefGoogle Scholar
  26. 26.
    Kao F-C, Wang SR, Chang Y-J (2015) Brainwaves Analysis of Positive and Negative Emotions. WSEAS Trans. Inf. Sci. Appl. 12:200–208Google Scholar
  27. 27.
    Webb GI (1999) Decision tree grafting from the all-tests-but-one partition. IJCAI Int J Conf Artif Intell 2:702–707Google Scholar
  28. 28.
    Cohen WW (1995) Fast effective rule induction. In: Machine learning: proceedings of twelth international conferenceCrossRefGoogle Scholar
  29. 29.
    Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6:37–66Google Scholar
  30. 30.
    John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Eleventh conference on uncertainty in artificial intelligence, pp 338–345Google Scholar
  31. 31.
    Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KRK (2001) Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput 13:637–649CrossRefGoogle Scholar
  32. 32.
    White BW, Rosenblatt F (1963) Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Am J Psychol 76:705CrossRefGoogle Scholar
  33. 33.
    Tamura T, Maeda Y, Sekine M, Yoshida M (2014) Wearable photoplethysmographic sensors—past and present. Electronics 3:282–302CrossRefGoogle Scholar
  34. 34.
    Lee J, Matsumura K, Yamakoshi KI, Rolfe P, Tanaka S, Yamakoshi T (2013) Comparison between red, green and blue light reflection photoplethysmography for heart rate monitoring during motion. In: Annual international conference of the IEEE engineering in medicine and biology society EMBS, pp 1724–1727Google Scholar
  35. 35.
    Esposito D, Cavallo F (2015) Preliminary design issues for inertial rings in ambient assisted living applications. In: 2015 IEEE international instrumentation and measurement technology conference (I2MTC) proceedings, pp 250–255. IEEEGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Francesco Semeraro
    • 1
  • Laura Fiorini
    • 1
    Email author
  • Stefano Betti
    • 1
  • Gianmaria Mancioppi
    • 1
  • Luca Santarelli
    • 1
  • Filippo Cavallo
    • 1
  1. 1.The BioRobotics Institute, Scuola Superiore Sant’AnnaPontedera (Pisa)Italy

Personalised recommendations