Body Posture Estimation in Sign Language Videos

  • François Lefebvre-Albaret
  • Patrice Dalle
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


This article deals with the posture reconstruction from a mono view video of a signed utterance. Our method makes no use of additional sensors or visual markers. The head and the two hands are tracked by means of a particle filter. The elbows are detected as convolution local maxima. A non linear filter is first used to remove the outliers, then some criteria using French Sign Language phonology are used to process the hand disambiguation. The posture reconstruction is achieved by using inverse kinematics, using a Kalman smoothing and the correlation between strong and week hand depth that can be noticed in the signed utterances. The article ends with a quantitative and qualitative evaluation of the reconstruction. We show how the results could be used in the framework of automatic Sign Language video processing.


Sign Language Posture Reconstruction Inverse Kinematics Mono Vision 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akyol, S., Alvarado, P.: Finding Relevant Image Content for mobile Sign Language Recognition. In: IASTED International Conference Signal Processing, Pattern Recognition and Application, pp. 48–52 (2001)Google Scholar
  2. 2.
    Battison, R.: Lexical borrowing in ASL. Linstok, Silver Spring (1978)Google Scholar
  3. 3.
    Brand, J., Mason, J.S.: A comparative assessment of three approaches to pixel-level human skin-detection. In: 15th ICPR, vol. 1, pp. 1056–1059 (2000)Google Scholar
  4. 4.
    Cuxac, C.: French Sign Language, the ways of Iconicity. In: Ophrys (ed.), Paris (2000)Google Scholar
  5. 5.
    Dalle, P.: High level models for sign language analysis by a vision system. In: Workshop on the Representation and Processing of Sign Language: Lexicographic Matters and Didactic Scenarios (LREC), Italy, ELDA, pp. 17–20 (2006)Google Scholar
  6. 6.
    Downton, A.C., Drouet, H.: Model-based image analysis for unconstrained human upper-body motion. In: ICIP, Venue, pp. 274–277 (1992)Google Scholar
  7. 7.
    Emmorey, K., Tversky, B., Taylor, H.A.: Using space to describe space: Perspective in speech, sign, and gesture. Spatial Cognition & Computation 2, 157–180 (2000)CrossRefGoogle Scholar
  8. 8.
    Fontmarty, M.: Vision et filtrage particulaire pour le suivi tridimentionnel de mouvement humain, Phd thesis, LAAS, University of Toulouse (2008)Google Scholar
  9. 9.
    Gianni, F., Collet, C., Dalle, P.: Robust tracking for processing of videos of communication’s gestures. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS (LNAI), vol. 5085, pp. 93–101. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Habili, N., Lim, C.C., Moini, A.: Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Transactions on Circuits and Systems for Video Technology 14(8), 1086–1097 (2004)CrossRefGoogle Scholar
  11. 11.
    Haritaoglu, I., Harwood, D., Davis, L.S.: Ghost: A human body part labeling system using silhouettes. In: ICPR, Brisbane, Australia, pp. 77–82 (1998)Google Scholar
  12. 12.
    Hienz, H., Grobel, K., Offner, G.: Real-time hand-arm motion analysis using a single video camera. In: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, Killington, USA, pp. 323–327 (1996)Google Scholar
  13. 13.
    Hruz, M., Campr, P., Zelezny, M.: Semi-automatic Annotation of Sign Language Corpora. In: Proceeding LREC 2008, Marrakech, Maroco (2008)Google Scholar
  14. 14.
    Jang, D.S., Jang, S.W., Choi, H.I.: 2D human body tracking with Structural Kalman filter. Pattern Recognition 35(10), 2041–2049 (2002)zbMATHCrossRefGoogle Scholar
  15. 15.
    Lenseigne, B., Gianni, F., Dalle, P.: Mono vision estimation of the arm posture using a biomechanical arm model, method and evaluation. In: 14th french-speeking congres on pattern recognition and artificial intelligence, RFIA Toulouse, France, AFRIF-AFIA, vol. (2), pp. 957–966 (2003)Google Scholar
  16. 16.
    Lenseigne, B., Gianni, F., Dalle, P.: A New Gesture Representation for Sign Language Analysis. In: LREC 2004 - Workshop on the Representation and Processing of Sign Language, Lisbonne, Portugal, pp. 85–90 (2004)Google Scholar
  17. 17.
    Li, P.H., Wang, H.J.: Object Tracking with Particle Filter Using Color Information. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 534–541. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Lichtenauer, J.F., Hendriks, E.A., Reinders, M.J.: 3D Visual Detection of Correct NGT Sign Production. In: 13th Annual Conference of the Advanced School for Computing and Imaging, Heijen, Netherlands (2007)Google Scholar
  19. 19.
    Maccormick, J., Blake, A.: A probabilistic exclusion principle for tracking multiple objects. International Journal of Computer Vision 39, 572–578 (1999)Google Scholar
  20. 20.
    Mahmoudi, F., Parviz, M.: Visual Hand Tracking Algorithms. In: GMAI 2006: Proceedings of the conference on Geometric Modeling and Imaging, pp. 228–232. IEEE Computer Society, Washington (2006)Google Scholar
  21. 21.
    Micilotta, A., Bowden, R.: View-based location and tracking of body parts for visual interaction. In: BMVC 2004, Kingston, pp. 849–858 (2004)Google Scholar
  22. 22.
    Noriega, P., Bernier, O.: Multicues 3D Monocular Upper Body Tracking using Constrained Belief Propagation. In: BMVC, Warwick, pp. 57–60 (2007)Google Scholar
  23. 23.
    Ong, S.C.W., Ranganath, S.: Automatic Sign Language Analysis, A Survey and the Future beyond Lexical Meaning. PAMI 27(6), 873–891 (2005)Google Scholar
  24. 24.
    Roberts, T.J., McKenna, S.J., Ricketts, I.W.: Human Pose Estimation Using learnt probabilistic region similarities and partial configurations. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 291–303. Springer, Heidelberg (2004)Google Scholar
  25. 25.
    Sherrah, J., Gong, S.: Resolving Visual Uncertainty and Occlusion through Probabilistic Reasoning. In: BMVC: Proceedings of the British Machine Vision Conference, Bristol, pp. 252–261 (2000)Google Scholar
  26. 26.
    Wang, J.J., Singh, S.: Video analysis of human dynamics: a survey. Real Time Imaging 9, 321–346 (2003)CrossRefGoogle Scholar
  27. 27.
    Wang, J., Chen, X., Gao, W.: Online selecting discriminative tracking features using particle filter. In: Conference on Computer Vision and Pattern Recognition, San Diego, USA, vol. 2, pp. 1037–1042. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  28. 28.
    Wang, H., Shindler, K.: Effective Appearance Model and Similarity Measure for Particle Filtering and Visual Tracking. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 606–618. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  29. 29.
    Yang, J., Timothy, R.M., Kim, H., Arora, J.S.: Abdel-Malek, K.: Multi-objective Optimization for Upper Body Posture Prediction. In: 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY (2004)Google Scholar
  30. 30.
    Zhou, H., Hu, H.S.: A Survey, Human Motion Tracking and Stroke Rehabilitation. Technical report, Dpt. of computer sciences, university of Essex, UK (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • François Lefebvre-Albaret
    • 1
  • Patrice Dalle
    • 1
  1. 1.IRIT : UPS-118 r. de NarbonneToulouse cedex 9

Personalised recommendations