Abstract
Audio and haptics as interaction modalities share properties, which make them highly appropriate to be handled within a single conceptual framework. This paper outlines such framework, gaining ingredients from the literature concerning cross-modal integration and embodied cognition. The resulting framework is bound up with a concept of physical embodiment, which has been introduced within several scientific disciplines to reveal the role of bodily experience and the corresponding mental imagery as the core of meaning-creation. In addition to theoretical discussion, the contribution of the proposed approach in design is outlined.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Card, S.K., Moran, T.P., Newell, A.: The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, Hillsdale (1983)
Gardner, H.G.: The Mind’s New Science. Howard Gardner, New York (1985)
Broadbent, D.E.: Perception and Communication. Pergamon, London (1958)
Deutsch, J.A., Deutsch, D.: Attention: Some Theoretical Considerations. Psychological Review 70(1), 80–90 (1963)
Wickens, C.D.: Processing Resources in Attention. In: Parasuraman, R., Davies, D.R. (eds.) Varieties of Attention, pp. 63–102. Academic Press, Orlando, F (1984)
Pirhonen, A.: Semantics of Sounds and Images - Can They Be Paralleled? In: Proc. of International Conference on Auditory Display Schulich School of Music, pp. 319–325. McGill University Montreal, Canada (2007)
Pirhonen, A., Palomäki, H.: Sonification of Directional and Emotional Content: Description of Design Challenges. In: Proc. of International Conference on Auditory Display, IRCAM, Paris (2008)
McLuhan, M.: Understanding Media: the Extensions of Man. New American Library, New York (1966)
Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. The University of Illinois Press, Urbana (1949)
Edwards, A.D.N.: Redundancy and Adaptability. In: Edwards, A.D.N., Holland, S. (eds.) Multimedia Interface Design in Education. NATO ASI Series F: Computer and System Sciences, vol. 76, pp. 145–155. Springer, Heidelberg (1992)
Johnson, M., Rohrer, T.: We Are Live Creatures: Embodiment,American Pragmatism, and the Cognitive Organism. In: Zlatev, J., Ziemke, T., Frank, R., Dirven, R. (eds.) Body, Language, and Mind, vol. 1, pp. 17–54. Mouton de Gruyter, Berlin (2007)
Lakoff, G., Johnson, M.: Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought. Basic Books, New York (1999)
Gallese, V., Lakoff, G.: The Brain’s Concepts: The Role of the Sensory-Motor System in Reason and Language. Cognitive Neuropsychology 22, 455–479 (2005)
Liberman, A.M., Mattingly, I.G.: The Motor Theory of Speech Perception Revised. Cognition 21, 136 (1985)
Wilson, M., Knoblich, G.: The Case for Motor Involvement in Perceiving Conspecifics. Psychological Bulletin 1(3), 460–473 (2005)
Godoy, R.I.: Motor-Mimetic Music Cognition. Leonardo 36(4), 317–319 (2003)
Gallese, V.: Embodied Simulation: From Mirror Neuron Systems to Interpersonal Relations. In: Empathy and Fairness (Novartis Foundation Symposium), vol. 278, pp. 3–19. Wiley, Chichester (2006)
Johnson, M.: The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. University of Chicago, Chicago (1987)
Godøy, R.I.: Gestural Imagery in the Service of Musical Imagery. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 55–62. Springer, Heidelberg (2004)
Tagg, P.: Towards a Sign Typology of Music. In: Dalmonte, R., Baroni, M. (eds.) Secondo Convegno Europeo di Analisi Musicale, pp. 369–378. Universitá Degli Studi di Trento (1992)
Dourish, P.: Where the Action is: The Foundations of Embodied Interaction. MIT Press, Cambridge (2001)
Pirhonen, A.: To Simulate or to Stimulate? In Search of the Power of Metaphor in Design. In: Pirhonen, A., Isomäki, H., Roast, C., Saariluoma, P. (eds.) Future Interaction Design, pp. 105–123. Springer, London (2005)
Jensenius, A.: Action-Sound: Developing Methods and Tools to Study Music-Related Body Movement. Ph.D. Thesis. Department of Musicology, University of Oslo (2007)
Godøy, R. I.. Imagined action, excitation, and resonance. In: Godøy, R. I. and Jørgensen, H. (eds.): Musical Imagery. Swets and Zeitlinger, Lisse (2001) 237-250
Tuuri, K., Eerola, T.: Could Function-Specific Prosodic Cues Be Used as a Basis for Non-Speech User Interface Sound Design. In: Proc. of International Conference on Auditory Display, IRCAM, Paris (2008)
Csikszentmihalyi, M.: Creativity: Flow and the psychology of discovery and invention. HarperCollins, New York (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pirhonen, A., Tuuri, K. (2008). In Search for an Integrated Design Basis for Audio and Haptics. In: Pirhonen, A., Brewster, S. (eds) Haptic and Audio Interaction Design. HAID 2008. Lecture Notes in Computer Science, vol 5270. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87883-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-87883-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87882-7
Online ISBN: 978-3-540-87883-4
eBook Packages: Computer ScienceComputer Science (R0)