Designing Mid-Air Gesture Interaction with Mobile Devices for Older Adults

  • Michela FerronEmail author
  • Nadia Mana
  • Ornella Mich
Part of the Human–Computer Interaction Series book series (HCIS)


This chapter presents a user-centered perspective on the design of effective mid-air gesture interaction with mobile technology for seniors. Starting from the basic characteristics of mid-air gesture interaction, we introduce the main design challenges of this interaction and report on a case study in which we implemented a user-centered design process focused on the engagement of older adults in each design stage. We present a series of studies in which we aimed at involving older users as active creators of the interaction, taking into account their feedback and values in the design process. Finally, we propose a set of recommendations for the design of mid-air gesture interaction for older adults. These recommendations are elaborated combining research on HCI, ageing and ergonomic principles, and the results from the user studies we conducted in the process.


  1. Bobeth J, Schmehl S, Kruijff E et al (2012) Evaluating performance and acceptance of older adults using freehand gestures for TV menu control. In: Proceedings of the 10th European conference on interactive TV and video. ACM, pp 35–44Google Scholar
  2. Bødker S (2006) When second wave HCI meets third wave challenges. In: Proceedings of the 4th Nordic conference on human-computer interaction: changing roles. ACM, pp 1–8Google Scholar
  3. Borg G (1998) Borg’s perceived exertion and pain scales. Human Kinetics, ChampaignGoogle Scholar
  4. Boring S, Jurmu M, Butz A (2009) Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays. In: Proceedings of the 21st annual conference of the Australian computer-human interaction special interest group: design: open 24/7. ACM, pp 161–168Google Scholar
  5. Cabreira AT, Hwang F (2016) How do novice older users evaluate and perform mid-air gesture interaction for the first time? Paper presented at the proceedings of the 9th Nordic conference on human-computer interaction, Gothenburg, SwedenGoogle Scholar
  6. Camunas-Mesa L, Zamarreno-Ramos C, Linares-Barranco A et al (2012) An event-driven multi-kernel convolution processor module for event-driven vision sensors. IEEE J Solid-State Circuits 47:504–517CrossRefGoogle Scholar
  7. Carbini S, Delphin-Poulat L, Perron L et al (2006) From a Wizard of Oz experiment to a real time speech and gesture multimodal interface. Sig Process 86:3559–3577zbMATHCrossRefGoogle Scholar
  8. Carreira M, Ting KLH, Csobanka P et al (2016) Evaluation of in-air hand gestures interaction for older people. Univers Access Inf Soc 16:1–20CrossRefGoogle Scholar
  9. Chen J, Proctor RW (2013) Response–Effect compatibility defines the natural scrolling direction. Hum Factors 55:1112–1129. Scholar
  10. Cockburn A, Quinn P, Gutwin C et al (2011) Air pointing: design and evaluation of spatial target acquisition with and without visual feedback. Int J Hum Comput Stud 69:401–414CrossRefGoogle Scholar
  11. Cronin D (2014) Usability of Micro- vs. Macro-gestures in camera-based gesture interaction. Diplomarbeit, California Polytechnic State University, San Luis Obispo. (Zitiert auf Seite 21)Google Scholar
  12. Delamare W, Janssoone T, Coutrix C et al (2016) Designing 3D gesture guidance: visual feedback and feedforward design options. In: Proceedings of the international working conference on advanced visual interfaces. ACM, pp 152–159Google Scholar
  13. Eaton C (1997) Electronic textbook on hand surgery.
  14. England D (2011) Whole body interaction: an introduction. In: Whole body interaction. Springer, London, pp 1–5CrossRefGoogle Scholar
  15. Ferron M, Mana N, Mich O (2015) Mobile for older adults: towards designing multimodal interaction. In: Proceedings of the 14th international conference on mobile and ubiquitous multimedia. ACM, pp 373–378Google Scholar
  16. Ferron M, Mana N, Mich O et al (2018) Design of multimodal interaction with mobile devices. Challenges for visually impaired and elderly users. In: 3rd international conference on human computer interaction theory and applications (HUCAPP). Funchal, Madeira. 140–146Google Scholar
  17. Gerling K, Dergousoff K, Mandryk R (2013) Is movement better? Ccomparing sedentary and motion-based game controls for older adults. In: Proceedings of graphics interface 2013. Canadian Information Processing Society, pp 133–140Google Scholar
  18. Gerling K, Livingston I, Nacke L et al (2012) Full-body motion-based game interaction for older adults. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 1873–1882Google Scholar
  19. Green P, Wei-Haas L (1985) The rapid development of user interfaces: experience with the wizard of Oz method. In: Proceedings of the human factors society annual meeting. Sage, Los Angeles, CA, pp 470–474CrossRefGoogle Scholar
  20. Gutman J (1982) A means-end chain model based on consumer categorization processes. J Mark 46:60–72CrossRefGoogle Scholar
  21. Harrison S, Sengers P, Tatar D (2011) Making epistemological trouble: third-paradigm HCI as successor science. Interact Comput 23:385–392CrossRefGoogle Scholar
  22. Hassani AZ, Van Dijk B, Ludden G et al (2011) Touch versus in-air hand gestures: evaluating the acceptance by seniors of human-robot interaction. In: International joint conference on ambient intelligence. Springer, pp 309–313Google Scholar
  23. Hincapié-Ramos JD, Guo X, Moghadasian P et al (2014) Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. In: Proceedings of the 32nd annual ACM conference on human factors in computing systems. ACM, pp 1063–1072Google Scholar
  24. Hurtienne J, Blessing L (2007) Design for intuitive use-testing image schema theory for user interface design. In: 16th international conference on engineering design. Citeseer, Paris, France. 829–830Google Scholar
  25. Hurtienne J, Stößel C, Sturm C et al (2010) Physical gestures for abstract concepts: Inclusive design with primary metaphors. Interact Comput 22:475–484. Scholar
  26. Jayroe TJ, Wolfram D (2012) Internet searching, tablet technology and older adults. Proc Am Soc Inf Sci Technol 49:1–3CrossRefGoogle Scholar
  27. Keir PJ, Bach JM, Rempel DM (1998) Effects of finger posture on carpal tunnel pressure during wrist motion. J Hand Surg 23:1004–1009CrossRefGoogle Scholar
  28. Kortum P (2008) HCI beyond the GUI: design for haptic, speech, olfactory, and other nontraditional interfaces. Morgan Kaufmann, BurlingtonGoogle Scholar
  29. Kurtz KH, Hovland CI (1953) The effect of verbalization during observation of stimulus objects upon accuracy of recognition and recall. J Exp Psychol 45:157CrossRefGoogle Scholar
  30. Maguire M (2001) Methods to support human-centred design. Int J Hum Comput Stud 55:587–634zbMATHCrossRefGoogle Scholar
  31. Mana N, Mich O, Ferron M (2017) How to increase older adults’ accessibility to mobile technology? The new ECOMODE camera. Paper presented at ForItAAL-Forum Italiano Ambient Assisted Living. Genoa, Italy.Google Scholar
  32. Markussen A, Jakobsen MR, Hornb K (2014) Vulture: a mid-air word-gesture keyboard. Paper presented at the proceedings of the 32nd annual ACM conference on human factors in computing systems, Toronto, ON, CanadaGoogle Scholar
  33. Nacenta MA, Kamber Y, Qiang Y et al (2013) Memorability of pre-designed and user-defined gesture sets. Paper presented at the proceedings of the SIGCHI conference on human factors in computing systems, Paris, FranceGoogle Scholar
  34. Ni T, Bowman DA, North C et al (2011) Design and evaluation of freehand menu selection interfaces using tilt and pinch gestures. Int J Hum Comput Stud 69:551–562CrossRefGoogle Scholar
  35. Nielsen M, Störring M, Moeslund TB et al (2004) A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri A, Volpe G (eds) Gesture-based communication in human-computer interaction. Springer, Heidelberg, pp 409–420CrossRefGoogle Scholar
  36. Pernice K, Nielsen J (2012) Web usability for senior citizens: design guidelines based on usability studies with people age 65 and older. Nielsen Norman GroupGoogle Scholar
  37. Reynolds TJ, Gutman J (1988) Laddering theory, method, analysis, and interpretation. J Advert Res 28:11–31Google Scholar
  38. Rico J, Brewster S (2010) Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 887–896Google Scholar
  39. Rogers Y, Marsden G (2013) Does he take sugar? Moving beyond the rhetoric of compassion. Interactions 20:48–57CrossRefGoogle Scholar
  40. Ruiz J, Li Y, Lank E (2011) User-defined motion gestures for mobile interaction. Paper presented at the proceedings of the SIGCHI conference on human factors in computing systems, Vancouver, BC, CanadaGoogle Scholar
  41. Ryan EB, Szechtman B, Bodkin J (1992) Attitudes toward younger and older adults learning to use computers. J Gerontol 47:P96–P101CrossRefGoogle Scholar
  42. Schiavo G, Mich O, Ferron M, Mana N (2017) Mobile multimodal interaction for older and younger users: exploring differences and similarities. In: Proceedings of the 16th international conference on mobile and ubiquitous multimedia. ACM, pp 407–414Google Scholar
  43. Schiavo G, Ferron M, Mich O et al (2016) Wizard of Oz studies with older adults: a methodological note. Int Rep Socio-Inform 13:93–100Google Scholar
  44. Vaidyanathan V, Rosenberg D (2014) “Will use it, because I want to look cool” a comparative study of simple computer interactions using touchscreen and in-air hand gestures. In: International conference on human-computer interaction. Springer, pp 170–181Google Scholar
  45. Vanden Abeele V, Zaman B (2009) Laddering the user experience! In: User experience evaluation methods in product development (UXEM 2009)-workshopGoogle Scholar
  46. Venkatesh V, Bala H (2008) Technology acceptance model 3 and a research agenda on interventions. Decis Sci 39:273–315CrossRefGoogle Scholar
  47. Vermeulen J, Luyten K, van den Hoven E, Coninx K (2013) Crossing the bridge over Norman’s gulf of execution: revealing feedforward’s true identity. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 1931–1940Google Scholar
  48. Wigdor D, Wixon D (2011) Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier, AmsterdamGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Fondazione Bruno KesslerTrentoItaly

Personalised recommendations