Real-Time Interactive Multimodal Systems for Physiological and Emotional Wellbeing

  • Nimish BiloriaEmail author
  • Dimitra Dritsa
Part of the S.M.A.R.T. Environments book series (SMARTE)


There has been lately significant progress in the design of clinically assistive technologies for physiological and emotional wellbeing, driven by developments in Human Computer Interaction, Virtual Reality systems for rehabilitation and social interaction and Rehabilitation Robotics. The clinical, task-driven nature of such systems though often affects negatively the user acceptance of technology, resulting in lesser interactions with the user. At the same time, interactive environments which are not constructed for strictly medical applications, can also instigate interaction dialogues which generate physiological and emotional benefits for the user, while also incorporating a more playful dimension. As there is currently lack of communication channels between Clinically Assistive technologies and Socially Interactive Design Systems, the chapter attempts to merge these domains by identifying parameters related to physiological and emotional wellbeing that could inform the design of interactive systems for health and wellbeing at variable scales. These parameters are presented as a set of guidelines for Interaction design for healthcare and wellbeing, and the chapter elaborates on their practical application through three case studies: RoboZoo, Textrinium and Reflectego. All the presented case studies operate as public indoor or outdoor installations and have been tested in different contextual conditions, in Netherlands, Spain and France.


Architecture Real-time interaction Robotics User behaviour Tangible interaction 



We would specially like to thank the research and design team members involved in the RoboZoo, Textrinium and FLUID projects: Dr. Jia Rey Chang, Javid Jooshesh, Guang Yang, Jan Paclt, Chris Pydo, Kasper Siderius, Radoslaw Flis, Bob Heester, Esther Slagter, Marien Teeuw, Veronika, Laszlo, Ricardo Galli, Chrysostomos Tsaprailis, Leslie Che, Jiarui Sun, Yağ ız Söylev, Tanya Somova, Nick van Dorp, Hua Fan, Y. Lyu, Danny Cheng, R. Chheda. Additionally, we would like to thank the European Union Culture Grant for providing us with the opportunity and for funding such interactive environments, as well as the Swedish School of Textiles from the University of Boras, and especially Dr. Delia Dumitrescu, Dr. Hanna Landin and Marjan Kooroshnia, for the provision of guidance and facilities for the construction of the interactive textiles.


  1. Been-Lirn Duh, H., et al. (2010). Senior-friendly technologies: Interaction design for senior users. In CHI extended abstracts (pp. 4513–4516). New York: ACM.Google Scholar
  2. Bilda, Z., Candy, L., & Edmonds, E. (2007). An embodied cognition framework for interactive experience. CoDesign, 3(2), 123–137.CrossRefGoogle Scholar
  3. Brewer, B. R., et al. (2007). Poststroke upper extremity rehabilitation: A review of robotic systems and clinical results. Topics in Stroke Rehabilitation, 14(6), 22–44.CrossRefGoogle Scholar
  4. Bullivant, L. (2007). Alice in Technoland. Architectural Design, 75(4), 6–13.CrossRefGoogle Scholar
  5. Delbrück, T., et al. (2007). A tactile luminous floor for an interactive autonomous space. Robotics and Autonomous Systems, 55(6), 433–443.CrossRefGoogle Scholar
  6. Fasoli, S. E., et al. (2003). Effects of robotic therapy on motor impairment and recovery in chronic stroke. Archives of Physical Medicine and Rehabilitation, 84(4), 477–482.CrossRefGoogle Scholar
  7. Fernaeus, Y., & Sundström, P. (2012). The material move how materials matter in interaction design research. In Proceedings of the designing interactive systems conference (pp. 486–495). Newcastle Upon Tyne.Google Scholar
  8. Fiore, S. M., et al. (2013). Toward understanding social cues and signals in human–robot interaction: Effects of robot gaze and proxemic behavior. Frontiers in Psychology, 4, 859.CrossRefGoogle Scholar
  9. Gupta, A., et al. (2008). Design, control and performance of RiceWrist: A force feedback wrist exoskeleton for rehabilitation and training. The International Journal of Robotics Research, 27(2), 233–251.CrossRefGoogle Scholar
  10. Harris, Y., & Bongers, B. (2002). Approaches to creating interactivated spaces, from intimate to inhabited interfaces. Organized Sound, 7(3), 239–246.CrossRefGoogle Scholar
  11. Heerink, M., et al. (2010). Assessing acceptance of assistive social agent technology by older adults: The Almere model. International Journal of Social Robotics, 2(4), 361–375.CrossRefGoogle Scholar
  12. Heijden, H. (2004). User acceptance of hedonic information systems. MIS Quarterly, 28(4), 695–704.CrossRefGoogle Scholar
  13. Hirsch, T. et al. (2000). The ELDer project: Social, emotional, and environmental factors in the design of eldercare technologies. In CUU ’00 Proceedings on the 2000 conference on Universal Usability (pp. 72–79). Arlington, Virginia, USA.Google Scholar
  14. Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 437–446). New York: ACM.Google Scholar
  15. Javed, H. et al. (2015). Thomas and friends: Implications for the design of social robots and their role as social story telling agents for children with autism. In 2015 IEEE international conference on robotics and biomimetics (ROBIO) (pp. 1145–1150). Zhuhai, China.Google Scholar
  16. Kidd, C. D. et al. (2006). A sociable robot to encourage social interaction among the elderly. In Proceedings of the 2006 IEEE international conference on robotics and automation , (pp. 3972–3976). Orlando: IEEE.Google Scholar
  17. Krebs, H. I., et al. (2007). Robot-aided neurorehabilitation: A robot for wrist rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 15(3), 327–335.CrossRefGoogle Scholar
  18. Lahiri, U., et al. (2011). Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 19(4), 443–452.CrossRefGoogle Scholar
  19. Law, E. et al. (2017). A wizard-of-Oz study of curiosity in human-robot interaction. In 26th IEEE international symposium on robot and human interactive communication (RO-MAN) (pp. 607–614). Lisbon, Portugal.Google Scholar
  20. Lee, K. M., et al. (2006). Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people’s loneliness in human–robot interaction. International Journal of Human-Computer Studies, 64(10), 962–973.CrossRefGoogle Scholar
  21. Leite, I., et al. (2013). Social robots for long-term interaction: A survey. International Journal of Social Robotics, 5(2), 291–308.CrossRefGoogle Scholar
  22. Luo, X. et al. (2005). An augmented reality training environment for post-stroke finger extension rehabilitation. In Proceedings of the 2005 IEEE 9th international conference on rehabilitation robotics (pp. 329–332). Chicago: IEEE.Google Scholar
  23. Matarić, M. J., et al. (2007). Socially assistive robotics for post-stroke rehabilitation. Journal of Neuroengineering and Rehabilitation, 4, 5.CrossRefGoogle Scholar
  24. Oosterhuis, K. (2002). Programmable architecture. L’Arca Edizione.Google Scholar
  25. Patton, J., et al. (2008). KineAssist: Design and development of a robotic overground gait and balance therapy device. Topics in Stroke Rehabilitation, 15(2), 131–139.CrossRefGoogle Scholar
  26. Pennycott, A., et al. (2012). Towards more effective robotic gait training for stroke rehabilitation: A review. Journal of Neuroengineering and Rehabilitation, 9(1), 65.CrossRefGoogle Scholar
  27. Polygerinos, P. et al. (2015). Soft robotic glove for hand rehabilitation and task specific training. In 2015 IEEE international conference on robotics and automation (pp. 2913–2919).Google Scholar
  28. Steele, R., et al. (2009). Elderly persons’ perception and acceptance of using wireless sensor networks to assist healthcare. International Journal of Medical Informatics, 78(12), 788–801.CrossRefGoogle Scholar
  29. Sun, H., & Zhang, P. (2006). Causal relationships between perceived enjoyment and perceived ease of use: An alternative approach. Journal of the Association for Information Systems, 7(9), 618–645.CrossRefGoogle Scholar
  30. Taysheng, J. (2005). Advanced ubiquitous media for interactive space. Computer Aided Architectural Design Futures, 6, 341–350.Google Scholar
  31. Turolla, A., et al. (2013). Haptic-based neurorehabilitation in poststroke patients: A feasibility prospective multicentre trial for robotics hand rehabilitation. Computational and Mathematical Methods in Medicine, 2013, 895492.CrossRefGoogle Scholar
  32. Wada, K., & Shibata, T. (2007). Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house. IEEE Transactions on Robotics, 23(5), 972–980.CrossRefGoogle Scholar
  33. Yakub, F., et al. (2014). Recent trends for practical rehabilitation robotics, current challenges and the future. International Journal of Rehabilitation Research, 37(1), 9–21.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Faculty of Design Architecture BuildingUniversity of Technology SydneyUltimoAustralia

Personalised recommendations