Methods for Effective Sonification of Clarinetists’ Ancillary Gestures

  • Florian Grond
  • Thomas Hermann
  • Vincent Verfaille
  • Marcelo M. Wanderley
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


We present the implementation of two different sonifications methods of ancillary gestures from clarinetists. The sonifications are data driven from the clarinetist’s posture which is captured with a VICON motion tracking system. The first sonification method is based on the velocities of the tracking markers, the second method involves a principal component analysis as a data preprocessing step. Further we develop a simple complementary visual display with a similar information content to match the sonification. The effect of the two sonifications with respect to the movement perception is studied in an experiment where test subjects annotate the clarinetists performance represented by various combinations of the resulting uni- and multimodal displays.


sonification 3D movement data ancillary gestures multimodal displays 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cadoz, C., Wanderley, M.M.: Gesture - Music. In: Wanderley, M.M., Battier, M. (eds.) Trends in Gestural Control of Music. Ircam – Centre Pompidou (2000)Google Scholar
  2. 2.
    Wanderley, M.M., Vines, B.W., Middleton, N., McKay, C., Hatch, W.: The musical significance of clarinetists’ ancillary gestures: an exploration of the field. Journal of New Music Research 34(1), 97–113 (2005)CrossRefGoogle Scholar
  3. 3.
    Wanderley, M.M.: Quantitative analysis of non-obvious performer gestures. In: Gesture and Sign Language in Human-Computer Interaction, pp. 241–253 (2002)Google Scholar
  4. 4.
    Hermann, T., Höner, O., Ritter, H.: Acoumotion - an interactive sonification system for acoustic motion control. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 312–323. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Nusseck, M., Wanderley, M.M.: Music and motion—how music related ancillary body movements contribute to the experience of music. Music Perception 26(4) (2009)Google Scholar
  6. 6.
    Verfaille, V., Quek, O., Wanderley, M.M.: Sonification of musicians’ ancillary gestures. In: Proceedings of the 12th International Conference on Auditory Display, London, UK, June 20 -23 (2006)Google Scholar
  7. 7.
    Savard, A.: When gestures are perceived through sounds: A framework for sonification of musicians’ ancillary gestures. Master’s thesis, IDMIL CIRRMT McGill University (August 2008)Google Scholar
  8. 8.
    Effenberg, A.O.: Movement sonification: Effects on perception and action. IEEE Multim. 12(2), 56–69 (2005)Google Scholar
  9. 9.
    Goina, P., Polotti, M.: Elementary gestalts for gesture sonification. In: NIME 2008, pp. 150–153 (2009)Google Scholar
  10. 10.
    Ferrari, A., Benedetti, M.G., Pavan, E., Frigo, C., Bettinelli, D., Rabuffetti, M., Crenna, P., Leardini, A.: Quantitative comparison of five current protocols in gait analysis. GAIT POSTURE (2008)Google Scholar
  11. 11.
    de Campo, A.: Toward a data soni cation design space map. In: Scavone, G.P. (ed.) ICAD 2007, Schulich School of Music, McGill University, Schulich School of Music, McGill University, Montreal, Canada, pp. 342–347 (2007),
  12. 12.
  13. 13.
    Chion, M.: Un art sonore, le cinéma, le phrasé audio visuelle. Cahiers du cinéma, pp. 541–561 (2003)Google Scholar
  14. 14.
    Flückiger, B.: Sound Design Die virtuelle Klangwelt des Films. Schüren, Marburg (2001)Google Scholar
  15. 15.
    Schmidt, F.P.: Design and implementation of a realtime 3d graphics server. Master’s thesis, Bielefeld University, Germany (April 2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Florian Grond
    • 1
  • Thomas Hermann
    • 1
  • Vincent Verfaille
    • 2
  • Marcelo M. Wanderley
    • 2
  1. 1.Ambient Intelligence Group, CITECBielefeld University 
  2. 2.IDMIL, CIRMMTMcGill University 

Personalised recommendations