Recognition of Gesture Sequences in Real-Time Flow, Context of Virtual Theater

  • Ronan Billon
  • Alexis Nédélec
  • Jacques Tisseau
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


Our aim is to put on a short play featuring a real actor and a virtual actor, who will communicate through movements and choreography, with mutual synchronization. Gesture recognition in our context of Virtual Theater is mainly based on the ability of a virtual actor to perceive gestures made by a real actor. We present a method for real-time recognition. We use properties from Principal Component Analysis (PCA) to create signature for each gesture and a multiagent system to perform the recognition.


motion-capture gesture recognition virtual theatre synthetic actor 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Thomas, F., Johnston, O.: Disney Animation: The Illusion of Life. Abbeville Press, New York (1981)Google Scholar
  2. 2.
    Billon, R., Nédélec, A., Tisseau, J.: Gesture recognition in flow based on pca analysis using multiagent system. In: ACE 2008: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pp. 139–146. ACM, New York (2008)CrossRefGoogle Scholar
  3. 3.
    Cadoz, C.: Les réalités virtuelles. In: DOMINOS, Flammarion, Paris (1994)Google Scholar
  4. 4.
    Karam, M., Schraefel, M.C.: A taxonomy of gestures in human computer interactions. Technical report, Electronics and Computer Science, University of Southampton (2005)Google Scholar
  5. 5.
    Bobick, A., Wilson, A.: A state-based technique for the summarization and recognition of gesture. ICCV 00, 382 (1995)Google Scholar
  6. 6.
    Hong, P., Turk, M., Huang, T.S.: Constructing finite state machines for fast gesture recognition. In: Proc. 15th ICPR, pp. 691–694 (2000)Google Scholar
  7. 7.
    Sandberg, A.: Gesture recognition using neural networks. Technical report, KTH University, Sweden (1997)Google Scholar
  8. 8.
    Zhao, L.: Synthesis and Acquisition of Laban Movement Analysis Qualitative Parameters for Communicative Gestures. PhD thesis, University of Pennsylvania, USA (2001)Google Scholar
  9. 9.
    Yamato, J., Ohya, J., Ishii, K.: Recognizing human action in time-sequential images using hidden markov model. In: Computer Vision and Pattern Recognition, Proceedings CVPR 1992, June 15-18, pp. 379–385 (1992)Google Scholar
  10. 10.
    Lee, H.K., Kim, J.: An hmm-based threshold model approach for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10), 961–973 (1999)CrossRefGoogle Scholar
  11. 11.
    Kim, D., Song, J., Kim, D.: Simultaneous gesture segmentation and recognition based on forward spotting accumulative hmms. Pattern Recogn. 40(11), 3012–3026 (2007)zbMATHCrossRefGoogle Scholar
  12. 12.
    Rajko, S., Qian, G., Ingalls, T., James, J.: Real-time gesture recognition with minimal training requirements and on-line learning. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2007, June 2007, pp. 1–8 (2007)Google Scholar
  13. 13.
    Bevilacqua, F., Guédy, F., Schnell, N., Fléty, E., Leroy, N.: Wireless sensor interface and gesture-follower for music pedagogy. In: NIME 2007: Proceedings of the 7th international conference on New interfaces for musical expression, pp. 124–129. ACM, New York (2007)CrossRefGoogle Scholar
  14. 14.
    Kahol, K., Tripathi, P., Panchanathan, S.: Documenting motion sequences with a personalized annotation system. IEEE MultiMedia 13(1), 37–45 (2006)CrossRefGoogle Scholar
  15. 15.
    Campbell, L.W., Bobick, A.: Recognition of human body motion using phase space constraints. In: Proceedings of Fifth International Conference on Computer Vision, pp. 624–630 (1995)Google Scholar
  16. 16.
    Vasilescu, M.A.O.: Human motion signatures: Analysis, synthesis, recognition. In: ICPR 2002: Proceedings of the 16 th International Conference on Pattern Recognition (ICPR 2002), Washington, DC, USA, vol. 3, p. 30456. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  17. 17.
    Jenkins, O.C., Mataric, M.J.: Automated derivation of behavior vocabularies for autonomous humanoid motion. In: AAMAS 2003: Proceedings of the second international joint conference on Autonomous agents and multiagent systems, pp. 225–232. ACM, New York (2003)CrossRefGoogle Scholar
  18. 18.
    Forbes, K., Fiume, E.: An efficient search algorithm for motion data using weighted pca. In: SCA 2005: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, pp. 67–76. ACM, New York (2005)CrossRefGoogle Scholar
  19. 19.
    Müller, M., Röder, T.: Motion templates for automatic classification and retrieval of motion capture data. In: SCA 2006: Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, Aire-la-Ville, Switzerland, Switzerland, Eurographics Association, pp. 137–146 (2006)Google Scholar
  20. 20.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHCrossRefGoogle Scholar
  21. 21.
    Shlens, J.: A tutorial on principal component analysis. Systems Neurobiology Laboratory, University of California at San Diego (2005)Google Scholar
  22. 22.
    Dunham, J.G.: Optimum uniform piecewise linear approximation of planar curves. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8(1), 67–75 (1986)CrossRefGoogle Scholar
  23. 23.
    Wilson, A., Bobick, A., Cassell, J.: Recovering the temporal structure of natural gesture. In: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, October 1996, pp. 66–71 (1996)Google Scholar
  24. 24.
    Kendon, A.: How gestures can become like words. In: Cross-Cultural Perspectives in Nonverbal Communication, pp. 131–141. F. Poyatos Publishers (1988)Google Scholar
  25. 25.
    Berndt, D.J., Clifford, J.: Finding patterns in time series: a dynamic programming approach, pp. 229–248 (1996)Google Scholar
  26. 26.
    Aubry, M., Julliard, F., Gibet, S.: Apprentissage des paramètres d’un contrôleur pour la synthèse de mouvements réalistes. LISyC, rapport interne, Journées l’Aber Wrac’h des 29 et 30 mai 2008 (June 2008)Google Scholar
  27. 27.
    Pinhanez, C.: Computer theater. In: International Symposium of Electronic Arts (ISEA 1997), Chicago, Illinois (September 1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ronan Billon
    • 1
  • Alexis Nédélec
    • 1
  • Jacques Tisseau
    • 1
  1. 1.Laboratoire d’Informatique des SYstèmes ComplexesCERV: Centre Européen de Réalité Virtuelle 

Personalised recommendations