Gestures in Human-Computer Interaction – Just Another Modality?

  • Antti Pirhonen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


The traditional framework in human-computer studies is based on a simple input-output model of interaction. In many cases, however, splitting interaction into input and output is not necessarily appropriate. Gestures work as a good example of a modality which is difficult or inappropriate to be conceptualised within the traditional input-output paradigm. In the search for a more appropriate interaction paradigm, gestures, as modality, have potential in working as a meta-modality, in terms of which all other modalities could be analysed. This paper proposes the use of gestures and gestural metaphors in a central role in interaction design, and presents a case study as an illustration of the point.


gesture metaphor human-computer interaction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Broadbent, D.E.: Perception and communication. Pergamon, London (1958)CrossRefGoogle Scholar
  2. 2.
    Horrey, W.J., Wickens, C.D.: Examining the impact of cell phone conversations on driving using meta-analytic techniques. Human Factors 48(1), 196–205 (2006)CrossRefGoogle Scholar
  3. 3.
    Card, S.K., Moran, T.P., Newell, A.: The psychology of human-computer interaction. Lawrence Erlbaum Associates, Hillsdale (1983)Google Scholar
  4. 4.
    Pirhonen, A., Brewster, S.: Metaphors and imitation. In: Workshop proceedings at PC-HCI 2001, Patras, Greece, December 7-9, pp. 27–32 (2001)Google Scholar
  5. 5.
    Pirhonen, A., Brewster, S., Holguin, C.: Gestural and audio metaphors as a means of control for mobile devices. In: Proceedings of CHI 2002, Minneapolis, Minnesota, April 20-25, pp. 291–298 (2002)Google Scholar
  6. 6.
    Pirhonen, A.: What do learning curves tell us about learnability? In: Vetere, F., Johnston, L., Kushinsky, R. (eds.) Proceedings of the HF2002 Human Factors Conference, Design for the whole person - integrating physical, cognitive and social aspects, November 25-27. Swinburne University of Technology (CD-ROM –format), Melbourne (2002)Google Scholar
  7. 7.
    Pirhonen, A.: From metaphors to simulations to idioms: Supporting the conceptualisation process. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 279–290. Springer, Heidelberg (2004)Google Scholar
  8. 8.
    Lakoff, G., Johnson, M.: Metaphors we live by. The University of Chicago Press, Chicago (1980)Google Scholar
  9. 9.
    Gallese, V., Lakoff, G.: The brain’s concepts: the role of the sensory-motor system in reason and language. Cogn. Neuropsychol. 22(2005), 455–479 (2005)CrossRefGoogle Scholar
  10. 10.
    Godøy, R.I.: Gestural-Sonorous Objects: Embodied Extensions of Schaeffer’s Conceptual Apparatus. Organised Sound 11(2) (2006)Google Scholar
  11. 11.
    Bernsen, N.O.: A toolbox of output modalities: Representing output information in multimodal interfaces. Esprit Basic Research Action 7040: The Amodeus Project, document TM/WP21 (1995)Google Scholar
  12. 12.
    Bernsen, N.O.: A taxonomy of input modalities. Esprit Basic Research Action 7040: The Amodeus Project, document TM/WP22 (1995)Google Scholar
  13. 13.
    Blattner, M., Sumikawa, D., Greenberg, R.: Earcons and icons: Their structure and common design principles. Human-Computer Interaction 4(1), 11–44 (1989)CrossRefGoogle Scholar
  14. 14.
    Brewster, S., Brown, L.: Non-visual information display using tactons. In: CHI 2004 extended abstracts on Human factors in computing systems, pp. 787–788. ACM, New York (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Antti Pirhonen
    • 1
  1. 1.Department of Computer Science and Information SystemsUniversity of JyväskyläFinland

Personalised recommendations