Observations on Experience and Flow in Movement-Based Interaction

  • Anton Nijholt
  • Marco Pasch
  • Betsy van Dijk
  • Dennis Reidsma
  • Dirk Heylen
Part of the Human-Computer Interaction Series book series (HCIS)


Movement-based interfaces assume that their users move. Users have to perform exercises, they have to dance, they have to golf or football, or they want to train particular bodily skills. Many examples of those interfaces exist, sometimes asking for subtle interaction between user and interface and sometimes asking for ‘brute force’ interaction between user and interface. Often these interfaces mediate between players of a game. Obviously, one of the players may be a virtual human. We embed this interface research in ambient intelligence and entertainment computing research, and the interfaces we consider are not only mediating, but they also ‘add’ intelligence to the interaction. Intelligent movement-based interfaces, being able to know and learn about their users, should also be able to provide means to keep their users engaged in the interaction. Issues that will be discussed in this chapter are ‘flow’ and ‘immersion’ for movement-based interfaces and we look at the possible role of interaction synchrony to measure and support engagement.


Game Design Game Experience Virtual Human Gaming Experience Game Element 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research has been supported by the Dutch National GATE project, funded by the Netherlands Organization for Scientific Research (NWO) and the Netherlands ICT Research and Innovation Authority (ICT Regie).


  1. 1.
    Babu, S., Zanbaka, C., Jackson, J., Chung, T.-O., Lok, B., Shin, M.C., Hodges, L.F.: Virtual human physiotherapist framework for personalized training and rehabilitation. In: Graphics Interface 2005, Victoria (2005)Google Scholar
  2. 2.
    Benford, S., Schnadelbach, H., Koleva, B., Gaver, B., Schmidt, A., Boucher, A., Steed, A., Anastasi, R., Greenhalgh, C., Rodden, T., Gellersen, H.: Sensible, sensible and desirable: a framework for designing physical interfaces. Technical Report Equator-03-003 (2003)Google Scholar
  3. 3.
    Bianchi-Berthouze, N., Kim, W.W., Patel, D.: Does body movement engage you more in digital game play? And why? In: Proceedings of Affective Computing and Intelligent Interaction 2007. Lecture Notes in Computer Science, vol. 4738, pp. 102–113. Springer, Heidelberg (2007)Google Scholar
  4. 4.
    Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRefGoogle Scholar
  5. 5.
    Brown, E., Cairns, P.: A grounded investigation of game immersion. In: Proceedings CHI 2004, pp. 1297–1300. Vienna (2004)Google Scholar
  6. 6.
    Cairns, P., Cox, A., Berthouze, N., Dhoparee, S., Jennett, C.: Quantifying the experience of immersion in games. In: Cognitive Science of Games and Gameplay, Workshop at Cognitive Science, Vancouver (2006)Google Scholar
  7. 7.
    Campbell, T., Fogarty, J.: Applying game design to everyday fitness applications. In: ACM CHI 2007 Workshop on Exertion Interfaces, San Jose (2007)Google Scholar
  8. 8.
    Cheng, K., Cairns, P. A.: Behaviour, realism and immersion in games. In: Proceedings CHI 2005, pp. 1272–1275. Portland (2005)Google Scholar
  9. 9.
    Cheok, A.D., Goh, K.H., Liu, W., Farbiz, F., Fong, S.W., Teo, S.L., Li, Y., Yang, X.: Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing. Personal Ubiquit. Comput. 8, 71–81 (2004)CrossRefGoogle Scholar
  10. 10.
    Clark, H.: Using Language. Cambridge University Press, Cambridge (1996)CrossRefGoogle Scholar
  11. 11.
    Consolvo, S., Everitt, K., Smith, I., Landay, J.A.: Design requirements for technologies that encourage physical activity. In: ACM Conference on Human Factors in Computing Systems (CHI 2006), pp. 457–466. Vancouver (2006)Google Scholar
  12. 12.
    Csikszentmihalyi, M.: Flow: the psychology of optimal experience. Harper & Row, New York (1990)Google Scholar
  13. 13.
    Davis, J.W., Bobick, A.F.: Virtual PAT: A Virtual Personal Aerobics Trainer. Technical Report: 436. MIT Media Laboratory (1998)Google Scholar
  14. 14.
    Ermi, L., Mäyrä, F.: Fundamental components of the gameplay experience: analysing immersion. In: de Castell, S., de Jenson, J. (eds.) Changing views: worlds in play, Selected Papers DiGRA Conference, pp. 15–27. Vancouver (2005)Google Scholar
  15. 15.
    Höysniemi, J., Aula, A., Auvinen, P., Hännikäinen, J., Hämäläinen, P.: Shadow boxer: a physically interactive fitness game. In: Third Nordic Conference on Human-Computer interaction (NordiCHI 2004), vol. 82, pp. 389–392. ACM, New York (2004)Google Scholar
  16. 16.
    Microsoft Kinect, Oct 2010
  17. 17.
    IJsselsteijn, W., Kort, Y., de Westerink, J., Jager, M., de Bonants, R.: Fun and sports: Enhancing the home fitness experience. In: Rauterberg M. (ed.) ICEC 2004. Lecture Notes in Artificial Intelligence, vol. 3166, pp. 46–56. Springer, Heidelberg (2004)Google Scholar
  18. 18.
    IJsselsteijn, W., Riva, G.: Being there: the experience of presence in mediated environments. In: Riva, G., Davide, F., IJsselsteijn, W. (eds.) Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments, pp. 3–16. IOS Press, Amsterdam (2003)Google Scholar
  19. 19.
    IJsselsteijn, W.A., van den Hoogen, W.M., Klimmt, C., de Kort, Y.A.W., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., Vorderer, P.: Measuring the experience of digital game enjoyment. In: Spink, E.J., Ballintijn, M.R., Bogers, N.D., Grieco, F., Loijens, L.W.S., Noldus, L.P.J.J., Smit, G., Zimmerman, P.H. (eds.), Proceedings of Measuring Behavior 2008, pp. 88–89. Maastricht (2008)Google Scholar
  20. 20.
    de Kort, Y.A.W., IJsselsteijn, W.A., Poels, K.: Digital games as social presence technology: development of the social presence in gaming questionnaire, PRESENCE 2007 Proceedings (Barcelona, Spain, 25–27 Oct 2007), pp. 195–203Google Scholar
  21. 21.
    Jackson, S.A., Csikszentmihalyi, M.: Flow in Sports: The Keys to Optimal Experiences and Performances. Human Kinetics, Leeds (1999)Google Scholar
  22. 22.
    Khoo, E.T., Cheok, A.D.: Age invaders: inter-generational mixed reality family game. Int. J. Virtual. Real. 5(2), 45–50 (2006)Google Scholar
  23. 23.
    Komura, T., Kuroda, A., Shinagawa, Y.: NiceMeetVR: facing professional baseball pitchers in the virtual batting cage. In: ACM Symposium on Applied Computing, pp. 1060–1065. Madrid (2002)Google Scholar
  24. 24.
    Larssen, A.T., Robertson, T., Loke, L., Edwards, J.: Special issue on movement-based interaction. Personal Ubiquit. Comput. 11(8), 607–701 (2004)CrossRefGoogle Scholar
  25. 25.
    Lindley, S., Le Couteur, J., Bianchi-Berthouze, N.: Stirring up experience through movement in game play: effects on engagement and social behaviour. In: ACM Conference on Human Factors in Computing Systems (CHI 2008), pp. 511–514. Florence (2008)Google Scholar
  26. 26.
    Mandryk, R.L., Atkins, M.S., Inkpen, K.M.: A continuous and objective evaluation of emotional experience with interactive play environments. In: ACM Conference on Human Factors in Computing Systems (CHI 2006), pp. 1027–1036. Montreal (2006)Google Scholar
  27. 27.
    Michalowski, M.P., Sabanovic, S., Kozima, H.: A dancing robot for rhythmic social interaction. In: Human Robot Interaction 2007, pp. 89–96. Arlington (2007)Google Scholar
  28. 28.
    Mokka, S., Väätänen, A., Heinilä, J., Välkkynen, P.: Fitness computer game with a bodily user interface. In: 2nd International Conference on Entertainment Computing,, pp. 1–3. Pittsburgh (2003)Google Scholar
  29. 29.
    Mueller, F., Agamanolis, S., Gibbs, M.R., Vetere, F.: Remote impact: shadowboxing over a distance. In: CHI 2008 Extended Abstracts on Human Factors in Computing Systems. ACM, New York (2008)Google Scholar
  30. 30.
    Mueller, F., Agamanolis, S., Picard, R.: Exertion interfaces: sports over a distance for social bonding and fun. In: ACM Conference on Human Factors in Computing Systems (CHI 2003), pp. 561–568. Ft. Lauderdale (2003)Google Scholar
  31. 31.
    Müller, F., Bianchi-Berthouze, N.: Evaluating exertion interfaces. In: Bernhaupt, R. (ed.) Evaluating User Experience in Games: Concepts and Methods, pp. 187–207. Springer, London (2010)CrossRefGoogle Scholar
  32. 32.
    Müller, F., Vetere, F., Gibbs, M.: Considerations for the design of networked exertion interactions. Int. J. Arts Technol. 3(4), 374–389 (2010)Google Scholar
  33. 33.
    Murray, J.H.: Hamlet on the Holodeck: The Future of Narrative in Cyberspace. MIT Press, Cambridge (1999)Google Scholar
  34. 34.
    Nacke, L.E.: Wiimote vs. controller: electroencephalographic measurement of affective gameplay interaction. In: Proceedings of FuturePlay 2010, pp. 183–190. Vancouver (2010)Google Scholar
  35. 35.
    Nacke, L.E., Stellmach, S., Lindley, C.: Electroencephalographic assessment of player experience: a pilot study in affective ludology. J. Simulation & Gaming. Published online, August 10, (2010)Google Scholar
  36. 36.
    Nijholt, A.: Playing and Cheating in Ambient Entertainment. In: Ma, L., Rauterberg, M., Nakatsu, R. (eds.) ICEC 2007. Lecture Notes in Computer Science, vol. 4740, pp. 415–420. Springer, Heidelberg (2007)Google Scholar
  37. 37.
    Nijholt, A., Reidsma, D., Welbergen, H., van Akker, H.J.A., op den Ruttkay, Z.M.: Mutually coordinated anticipatory multimodal interaction. In: Esposito, A., et al. (eds.) Nonverbal Features of Human-Human and Human-Machine Interaction. Lecture Notes in Computer Science, vol. 5042, pp. 73–93. Springer, Heidelberg (2008)Google Scholar
  38. 38.
    Nijholt, A., Tan, D.: Playing with your Brain: Brain-Computer Interfaces and Games. In: Bernhaupt, R., Tscheligi, M. (eds.) Proceedings International Conference on Advances in Computer Entertainment Technology, pp. 305–306. ACM, New York (2007)Google Scholar
  39. 39.
    Park, J.Y., Yi, J.H.: Gesture recognition based interactive boxing game. Int. J. Inf. Technol. 12(7), 36–44 (2006)Google Scholar
  40. 40.
    Picard, R.W., Daily, S.B.: Evaluating affective interactions: Alternatives to asking what users feel. Human factors in computing systems. In: Workshop on Innovative Approaches to Evaluating Affective Interfaces, Portland (2005)Google Scholar
  41. 41.
    Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)CrossRefGoogle Scholar
  42. 42.
    Reidsma, D., Welbergen, H., van Poppe, R., Bos, P., Nijholt, A.: Towards bi-directional dancing interaction. In: Harper, R., Rauterberg, M., Combetto, M. (eds.) ICEC 2006. Lecture Notes in Computer Science, vol. 4161, pp. 1–12. Springer, Heidelberg (2006)Google Scholar
  43. 43.
    Ruttkay, Z.M., van Welbergen, H.: On the timing of gestures of a virtual Physiotherapist. In: Lanyi, C.S. (ed.) 3rd Central European MM & VR Conference, pp. 219–224. Pannonian University Press, Hungary (2006)Google Scholar
  44. 44.
    Shaw, D., Gorely, T., Corban, R.: Sport and Exercise Psychology. BIOS Scientific Publishers, London (2005)Google Scholar
  45. 45.
    Sinclair, J., Hingston, P., Masek, M.: Considerations for the design of exergames. In: 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia, pp. 289–295. Perth (2007)Google Scholar
  46. 46.
    Sony Eyetoy, Oct 2010
  47. 47.
    Strömberg, H., Väätänen, A., Räty, V.: A group game played in interactive virtual space: design and evaluation. Designing Interactive Systems, pp. 56–63. New York (2002)Google Scholar
  48. 48.
    Sweetser, P., Wyeth, P.: Gameflow: a model for evaluating player enjoyment in games. Comput. Entertain. 3(3), 1–24 (2005)CrossRefGoogle Scholar
  49. 49.
    Tan Chua, P., Crivella, R., Daly, B., Hu, N., Schaaf, R., Ventura, D., Camill, T., Hodgins, J., Pausch, R.: Training for physical tasks in virtual environments: Tai Chi. Virtual Reality, pp. 87–94. IEEE, Los Alamitos (2003)Google Scholar
  50. 50.
    Tanaka, F., Suzuki, H.: Dance interaction with QRIO: a case study for non-boring interaction by using an entrainment ensemble model. In: IEEE International Workshop on Robot and Human Interactive Communication (RO-MAN 2004), pp. 419–424. Kurashiki (2004)Google Scholar
  51. 51.
    ter Maat, M., Ebbers, R., Reidsma, D., Nijholt, A.: Beyond the Beat: Modelling Intentions in a Virtual Conductor. In: INTETAIN ’08: Proceedings of the 2nd International Conference on INtelligent TEchnologies for Interactive enterTAINment, ACM Digital Library, Cancun (2008)Google Scholar
  52. 52.
    Tomida, T., Ishihara, A., Ueki, A., Tomari, Y., Fukushima, K., Inakage, M.: In: MiXer: the communication entertainment content by using “entrainment phenomenon” and “biofeedback”. Advances in Computer Entertainment Technology, pp. 286–287. New York (2007)Google Scholar
  53. 53.
    van den Hoogen, W.M., IJsselsteijn, W.A., de Kort, Y.A.W.: Exploring behavioral expressions of player experience in digital games. In: Nijholt, A., Poppe, R. (eds.), Proceedings of the Workshop on Facial and Bodily Expression for Control and Adaptation of Games, ECAG 2008, pp. 11–19. Amsterdam (2008)Google Scholar
  54. 54.
    Yim, J., Graham, T.C.N.: Using games to increase exercise motivation. In: Proceedings of the 2007 Conference on Future Play (Future Play ’07), pp. 166–173. New York (2007)Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Anton Nijholt
    • 1
  • Marco Pasch
    • 2
  • Betsy van Dijk
    • 1
  • Dennis Reidsma
    • 1
  • Dirk Heylen
    • 1
  1. 1.Human Media InteractionUniversity of TwenteEnschedeThe Netherlands
  2. 2.Faculty of InformaticsUniversity of LuganoLuganoSwitzerland

Personalised recommendations