Whole Body Interaction in Abstract Domains

  • Simon Holland
  • Katie Wilkie
  • Anders Bouwer
  • Mat Dalgleish
  • Paul Mulholland
Part of the Human-Computer Interaction Series book series (HCIS)


There is little dispute that Whole Body Interaction is a good fit of interaction style for some categories of application domain, such as the motion capture of gestures for computer games and virtual physical sports. This reflects the observation that in such applications the mapping between user gesture and the desired effect is, broadly speaking, the identity function. For more abstract application areas such as mathematics, programming and musical harmony, finding appropriate mappings between gesture and effect is less straightforward. The creation of appropriate whole body interaction designs for such abstract application areas remains challenging. However, this is not to argue that whole body interaction is unsuited to abstract domains. Indeed, there is evidence, outlined below, that whole body interaction offers excellent affordances for some highly abstract applications areas.


Body Interaction Abstract Domain Conceptual Metaphor Music Theory Harmonic Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Antle, A.N., Corness, G., Droumeva, M.: Human-computer-intuition? Exploring the cognitive basis for intuition in embodied interaction. Int. J. Arts Technol. 2(3), 235–254 (2009)CrossRefGoogle Scholar
  2. 2.
    Antle, A.N., Droumeva, M., Corness, G.: Playing with the sound maker: do embodied ­metaphors help children learn? In: Proceedings of the 7th International Conference on Interaction Design and Children, Chicago, pp. 178–185. Association for Computing Machinery, New York (2008)Google Scholar
  3. 3.
    Bach-y-Rita, P.: Brain Mechanisms in Sensory Substitution. Academic, New York (1972). Studies 64(1), 1–15 (2005)Google Scholar
  4. 4.
    Balzano, G.J.: The group-theoretic description of 12-fold and microtonal pitch systems. Comput. Music J. 4(4 Winter), 66–84 (1980)CrossRefGoogle Scholar
  5. 5.
    Bird, J., Holland, S., Marshall, P., Rogers, Y., Clark, A.: Feel the force: using tactile technologies to investigate the extended mind. In: Proceedings of Devices That Alter Perception Workshop (DAP 08), Seoul (2008)Google Scholar
  6. 6.
    Brower, C.: A cognitive theory of musical meaning. J. Music Theory 44(2), 323–379 (2000)CrossRefGoogle Scholar
  7. 7.
    Dourish, P.: Where the Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge (2001). ISBN 978-0-262-54178-7Google Scholar
  8. 8.
    Eitan, Z., Timmers, R.: Beethoven’s last piano sonata and those who follow crocodiles: cross-domain mappings of auditory pitch in a musical context. Cognition 114(3), 405–422 (2010)CrossRefGoogle Scholar
  9. 9.
    Fauconnier, G., Turner, M.: The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. Basic Books, New York (2002)Google Scholar
  10. 10.
    Gold, B.: Mathematical Association of America Online Book Review. Accessed Oct 2010
  11. 11.
    Held, R., Hein, A.V.: Adaptation of disarranged hand-eye coordination contingent upon re-afferent stimulation. Percept. Mot. Skills 8(1958), 87–90 (1958)CrossRefGoogle Scholar
  12. 12.
    Holland, S.: Learning about harmony with harmony space: an overview. In: Smith, M., Wiggins, G. (eds.) Music Education: An Artificial Intelligence Approach. Springer, London (1994)Google Scholar
  13. 13.
    Holland, S., Bouwer, A., Dalgleish, M., Hurtig, T.: Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit. In: Proceedings of TEI 2010, Boston, Cambridge (2010)Google Scholar
  14. 14.
    Holland, S., Marshall, P., Bird, J., Dalton, S., Morris, R., Pantidi, N., Rogers, Y., Clark, A.: Running up Blueberry Hill: prototyping whole body interaction in harmony space. In: Proceedings of the Third Conference on Tangible and Embodied Interaction, pp. 92–98. ACM, New York (2009)Google Scholar
  15. 15.
    Holland, S.: Artificial intelligence, education and music. Ph.D. thesis, Institute of Educational Technology, The Open University (1989)Google Scholar
  16. 16.
    Hurtienne, J., Blessing, L.: Design for intuitive use – testing image schema theory for user interface design. In: Proceedings of the 16th International Conference on Engineering Design, pp. 1–12. Paris (2007)Google Scholar
  17. 17.
    Hurtienne, J., Israel, J.H., Weber, K.: Cooking up real world business applications combining physicality, digitality, and image schemas. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, Bonn, pp. 239–246. ACM, New York (2008)Google Scholar
  18. 18.
    Johnson, M.: The philosophical significance of image schemas. In: Hampe, B., Grady, J. (eds.) From Perception to Meaning: Image Schemas in Cognitive Linguistics, pp. 15–33. Walter de Gruyter, Berlin (2005)CrossRefGoogle Scholar
  19. 19.
    Klemmer, S.R., Hartmann, B., Takayama, L.: How bodies matter: five themes for interaction design. In: Proceedings of the 6th Conference on Designing Interactive Systems (DIS’06), pp. 140–149. University Park, 26–28 June 2006.. ACM, New York (2006)Google Scholar
  20. 20.
    Lakoff, G.: Moral Politics: How Liberals and Conservatives Think. University of Chicago Press, Chicago (2002)Google Scholar
  21. 21.
    Lakoff, G., Johnson, M.: Metaphors We Live by. The University of Chicago, London (2003)Google Scholar
  22. 22.
    Lakoff, G., Núñez, R.: Where Mathematics Comes from: How the Embodied Mind Brings Mathematics into Being. Basic Books, New York (2000)MATHGoogle Scholar
  23. 23.
    Larson, S.: Musical forces and melodic patterns. Theory Pract. 22–23, 55–71 (1997–1998)Google Scholar
  24. 24.
    O’Regan, K., Noe, A.: A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24(5), 883–917 (2001)Google Scholar
  25. 25.
    Rohrer, T.: The body in space: dimensions of embodiment. In: Ziemke, T., Zlatev, J., Frank, R., Dirven, R. (eds.) Body, Language, and Mind: Embodiment, pp. 339–378. Walter de Gruyter, Berlin (2007)Google Scholar
  26. 26.
    Saslaw, J.: Forces, containers, and paths: the role of body-derived image schemas in the conceptualization of music. J. Music Theory 40(2 Autumn), 217–243 (1996)CrossRefGoogle Scholar
  27. 27.
    Wilkie, K., Holland, S., Mulholland, P.: What can the language of musicians tell us about music interaction design? Comput. Music J. 34(4), 34–48 (2010)CrossRefGoogle Scholar
  28. 28.
    Wilkie, K., Holland, S., Mulholland, P.: Evaluating musical software using conceptual metaphors. In: Proceedings of the 23rd British Computer Society Conference on Human Computer Interaction, Cambridge, pp. 232–237. British Computer Society, Cambridge (2009)Google Scholar
  29. 29.
    Zbikowski, L.M.: Metaphor and music theory: reflections from cognitive science. Music Theory Online 4, 1–8 (1998)Google Scholar
  30. 30.
    Zbikowski, L.M.: Conceptual models and cross-domain mapping: new perspective on theories of music and hierarchy. J. Music Theory 41(2 Autumn), 193–225 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Simon Holland
    • 1
  • Katie Wilkie
    • 1
  • Anders Bouwer
    • 2
  • Mat Dalgleish
    • 3
  • Paul Mulholland
    • 1
  1. 1.The Music Computing Lab, Department of ComputingThe Open UniversityMilton KeynesUK
  2. 2.Intelligent Systems Lab, Faculty of Science, Informatics InstituteUniversity of AmsterdamAmsterdamThe Netherlands
  3. 3.Department of Music (SPAL)University of WolverhamptonWolverhamptonUK

Personalised recommendations