Conductor Tutoring Using the Microsoft Kinect

  • Andrea SalgianEmail author
  • Leighanne Hsu
  • Nathaniel Milkosky
  • David Vickerman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9475)


In this paper we present a system that uses the Microsoft Kinect to provide beginner conducting students real time feedback about their performance. Using upper body joint coordinates we detect common mistakes such as swaying, rocking, excessive hinge movement, and mirroring. We compute instant velocities to determine tempo and classify articulation as legato or staccato. Our experiments show that the system performs perfectly when detecting erroneous movements, correctly classifies articulation type most of the time, and can correctly determine tempo by counting the number of beats per minute. The system was well received by conducting students and their instructor, as it allows them to practice by themselves, without an orchestra.


Velocity Magnitude Hand Gesture Musical Piece Hand Velocity Conducting Technique 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Behringer, R.: Conducting digitally stored music by computer vision tracking. In: First International Conference on Automated Production of Cross Media Content for Multi-Channel Distribution (AXMEDIS05), Florence, Italy (2005)Google Scholar
  2. 2.
    Wilson, A., Bobick, A.: Realtime online adaptive gesture recognition. In: International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, Corfu, GreeceGoogle Scholar
  3. 3.
    Murphy, D., Andersen, T.H., Jensen, K.: Conducting audio files via computer vision. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 529–540. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  4. 4.
    Marrin, T., Paradiso, J.: The digital baton: a versatile performance instrument. In: International Computer Music Conference, Thessaloniki, Greece, pp. 313–316 (1997)Google Scholar
  5. 5.
    Nakra, T.M., Ivanov, Y., Smaragdis, P., Ault, C.: The ubs virtual maestro: an interactive conducting system. In: New Interfaces for Musical Expression (NIME), Pittsburgh, PA (2009)Google Scholar
  6. 6.
    Salgian, A., Pfirrmann, M., Nakra, T.M.: Follow the beat? Understanding conducting gestures from video. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Paragios, N., Tanveer, S.-M., Ju, T., Liu, Z., Coquillart, S., Cruz-Neira, C., Müller, T., Malzbender, T. (eds.) ISVC 2007, Part I. LNCS, vol. 4841, pp. 414–423. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  7. 7.
    Peng, L., Gerhard, D.: A wii-based gestural interface for computer-based conducting systems. In: New Interfaces for Musical Expression (NIME), Pittsburgh, PA (2009)Google Scholar
  8. 8.
    Ivanova, E., Wang, L., Fu, Y., Gadzala, J.: Maestro: a practice system to track, record, and observe for novice orchestral conductors. In: CHI 2014 Extended Abstracts on Human Factors in Computing SystemsGoogle Scholar
  9. 9.
    Library, M.: Kinect for windows sdk. Accessed 20 August 2015

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Andrea Salgian
    • 1
    Email author
  • Leighanne Hsu
    • 1
  • Nathaniel Milkosky
    • 1
  • David Vickerman
    • 2
  1. 1.Department of Computer ScienceThe College of New JerseyEwingUSA
  2. 2.Department of MusicThe College of New JerseyEwingUSA

Personalised recommendations