Abstract
In this paper we present results from an experiment in which infrared motion capture technology was used to record participants’ movement in synchrony to different rhythms and different sounds. The purpose was to determine the effects of the sounds’ spectral and temporal features on synchronization and gesture characteristics. In particular, we focused on the correlation between sounds and three gesture features: maximum acceleration, discontinuity, and total quantity of motion. Our findings indicate that discrete, discontinuous motion resulted in better synchronization, while spectral features of sound had a significant effect on the quantity of motion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Camurri, A., Lagerlöf, I., Volpe, G.: Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies 59(1), 213–225 (2003)
Caramiaux, B., Bevilacqua, F., Schnell, N.: Towards a Gesture-Sound Cross-Modal Analysis. In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS, vol. 5934, pp. 158–170. Springer, Heidelberg (2010)
Chen, J.L., Penhune, V.B., Zatorre, R.J.: The role of auditory and premotor cortex in sensori-motor transformations. Annals of the New York Academy of Sciences 1169(1), 15–34 (2008)
Dissanayake, E.: Antecedents of the Temporal Arts in Early Mother-Infant Interaction. In: Wallin, N.L., Merker, B., Brown, S. (eds.) The Origins of Music, pp. 389–410. MIT Press (1999)
Eitan, Z., Granot, R.Y.: How music moves: Musical parameters and listeners’ images of motion. Music Perception 23(3), 221–248 (2006)
Elliott, M., Welchman, A., Wing, A.: Being discrete helps keep to the beat. Experimental Brain Research 192(4), 731–737 (2009)
Gibson, J.J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, Inc. (1979)
Godøy, R.I.: Gestural-Sonorous Objects: Embodied Extensions of Schaeffer’s Con-ceptual Apparatus. Organised Sound 11(2), 149–157 (2006)
Godøy, R.I.: Gestural affordances of musical sound. In: Godøy, R.I., Leman, M. (eds.) Musical Gestures: Sound, Movement, and Meaning, pp. 103–125. Routledge (2010)
Godøy, R.I., Haga, E., Jensenius, A.R.: Playing “Air Instruments”: Mimicry of Sound-Producing Gestures by Novices and Experts. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 256–267. Springer, Heidelberg (2006)
Godøy, R.I., Haga, E., Jensenius, A.R.: Exploring music-related gestures by sound-tracing. A preliminary study. In: 2nd ConGAS International Symposium on Gesture Interfaces for Multimedia Systems, Leeds, UK (2006)
Grahn, J.A., Brett, M.: Rhythm and beat perception in motor areas of the brain. Journal of Cognitive Neuroscience 19, 893–906 (2007)
Hatten, R.: Interpreting Musical Gestures, Topics, and Tropes: Mozart, Beethoven, Schubert. Indiana University Press (2004)
Leman, M.: Music, gesture, and the formation of embodied meaning. In: Godøy, R.I., Leman, M. (eds.) Musical Gestures: Sound, Movement, and Meaning, pp. 126–153. Routledge (2010)
London, J.: Hearing in Time: Psychological Aspects of Musical Meter. Oxford University Press (2004)
Luck, G., Toiviainen, P.: Ensemble musicians’ synchronization with conductors’ gestures: An automated Feature-Extraction analysis. Music Perception 24(2), 189–200 (2006)
McNeill, D.: Gesture and Thought. University of Chicago Press (2005)
Merer, A., Ystad, S., Kronland-Martinet, R., Aramaki, M.: Semiotics of Sounds Evoking Motions: Categorization and Acoustic Features. In: Kronland-Martinet, R., Ystad, S., Jensen, K. (eds.) CMMR 2007. LNCS, vol. 4969, pp. 139–158. Springer, Heidelberg (2008)
Merker, B., Madison, G.S., Eckerdal, P.: On the role and origin of isochrony in human rhythmic entrainment. Cortex 45, 4–17 (2008)
Meyer, L.B.: Emotion and Meaning in Music. University of Chicago Press (1956)
Nettl, B.: An ethnomusicologist contemplates universals in musical sound and musical culture. In: Wallin, N.L., Merker, B., Brown, S. (eds.) The Origins of Music, pp. 463–472. MIT Press (1999)
Nymoen, K., Torrensen, J., Godøy, R.I., Jensenius, A.R.: A statistical approach to analyzing sound tracings. In: Ystad, S., et al. (eds.) Post-proceedings of Computer Music Modeling and Retrieval 2011. LNCS. Springer (forthcoming, 2012)
Repp, B.H.: Sensorimotor synchronization: A review of tapping literature. Psychonomic Bulletin & Review 12(6), 969–992 (2005)
Sethares, W.A.: Rhythm and Transforms. Springer (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kozak, M., Nymoen, K., Godøy, R.I. (2012). Effects of Spectral Features of Sound on Gesture Type and Timing. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, SE. (eds) Gesture and Sign Language in Human-Computer Interaction and Embodied Communication. GW 2011. Lecture Notes in Computer Science(), vol 7206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34182-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-34182-3_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34181-6
Online ISBN: 978-3-642-34182-3
eBook Packages: Computer ScienceComputer Science (R0)