Advertisement

Journal of Intelligent & Robotic Systems

, Volume 76, Issue 2, pp 283–296 | Cite as

A Dynamic Gesture and Posture Recognition System

  • Kyriakos Sgouropoulos
  • Ekaterini Stergiopoulou
  • Nikos Papamarkos
Article

Abstract

This paper presents a real time dynamic hand gesture and posture recognition system based on a neural network and a Hidden Markov Model. For skin color segmentation an adaptive online trained skin color model is used, while the hand posture recognition is accomplished through a likelihood-based classification technique of geometric features. A novel trajectory smoothing technique based on Self Organized Neural Network is introduced to improve HMM classification performance of dynamic gestures. The aim of the proposed system is the creation of a visual dictionary combining hand postures and dynamic gestures. The system has been successfully tested with many people under varying light conditions and different web cameras.

Keywords

Human computer interaction Hand posture Dynamic gesture Skin color detection Artificial neural network Hidden Markov Model 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Murthy, G., Jadon, R.: A review of vision based hand gestures recognition. Int. J. Inf. Technol. Knowl. Manag. 2(2), 405–410 (2009)Google Scholar
  2. 2.
    Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.-F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: gesture and speech. ACM Trans. Comput-Hum. Interact. (TOCHI). 9(3), 171–193 (2002)CrossRefGoogle Scholar
  3. 3.
    Van den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV) (2011)Google Scholar
  4. 4.
    Elmezain, M., Al-Hamadi, A., Appenrodt, J., Michaelis, B.: A hidden markov model-based continuous gesture recognition system for hand motion trajectory. In: 19th International Conference on Pattern Recognition, ICPR (2008)Google Scholar
  5. 5.
    Kollorz, E., Penne, J., Hornegger, J., Barke, A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3), 334–343 (2008)Google Scholar
  6. 6.
    Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin-color modeling and detection methods. Pattern Recognit. 40(3), 1106–1122 (2007)CrossRefMATHGoogle Scholar
  7. 7.
    Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon (2003)Google Scholar
  8. 8.
    Doulamis, N., Doulamis, A., Kosmopoulos, D.: Content-based decomposition of gesture videos. In: IEEE Workshop on Signal Processing Systems Design and Implementation (2005)Google Scholar
  9. 9.
    Araki, R., Gohshi, S., Ikenaga, T.: Real-time both hands tracking using CAMshift with motion mask and probability reduction by motion prediction. In: Signal & Information Processing Association Annual Summit and Conference, (APSIPA ASC) 2012. Asia-Pacific (2012)Google Scholar
  10. 10.
    Oikonomidis, I., Kyriazis, N., Argyros, A.: Efficient model-based 3d tracking of hand articulations using kinect. In: British Machine Vision Conference (2011)Google Scholar
  11. 11.
    Athitsos, V., Wang, H., Stefan, A.: A database-based framework for gesture recognition. Pers. Ubiquit. Comput. 14(6), 511–526 (2010)CrossRefGoogle Scholar
  12. 12.
    Malassiotis, S., Strintzis, M.: Real-time hand posture recognition using range data. Image Vis. Comput. 26(7), 1027–1037 (2008)CrossRefGoogle Scholar
  13. 13.
    Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comp. Vision Image Underst. 108(1), 52–73 (2007)CrossRefGoogle Scholar
  14. 14.
    Hasan, M.M., Mishra, P.K.: Hand gesture modeling and recognition using geometric features: a review. Can. J. Image Process. Comput. Vis. 3(1), 12–26 (2012)Google Scholar
  15. 15.
    Mitra, S., Acharya, T.: Gesture recognition: a survey. Syst. Man Cybern. C Appl. Rev. IEEE Trans. 37(3), 311–324 (2007)CrossRefGoogle Scholar
  16. 16.
    Wang, C.-C., Wang, K.-C.: Hand posture recognition using adaboost with SIFT for human robot interaction. In: Recent Progress in Robotics: Viable Robotic Service to Human, pp. 317–329. Springer (2008)Google Scholar
  17. 17.
    Kulkarni, V.S., Lokhande, S.: Appearance based recognition of american sign language using gesture segmentation. Int. J. Comput. Sci. Eng. 2(3), 560–565 (2010)Google Scholar
  18. 18.
    Caridakis, G., Karpouzis, K., Drosopoulos, A., Kollias, S.: SOMM: self organizing Markov map for gesture recognition. Pattern Recog. Lett. 31(1), 52–59 (2010)CrossRefGoogle Scholar
  19. 19.
    Stergiopoulou, E., Papamarkos, N.: Hand gesture recognition using a neural network shape fitting technique. Eng. Appl. Artif. Intell. 22(8), 1141–1158 (2009)CrossRefGoogle Scholar
  20. 20.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE (2001)Google Scholar
  21. 21.
    Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE (2002)Google Scholar
  22. 22.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media (2008)Google Scholar
  23. 23.
    Zhang, C., Zhang, Z.: A survey of recent advances in face detection. Microsoft Research (2010)Google Scholar
  24. 24.
    Yang, J., Lu, W., Waibel, A.: Skin-color modeling and adaptation. In: Computer Vision—ACCV’98, pp. 687–694 (1997)Google Scholar
  25. 25.
    Chai, D., Ngan, K.: Face segmentation using skin-color map in videophone applications. IEEE Trans. Circ. Syst. Video Technol. 9, 551–564 (1999)CrossRefGoogle Scholar
  26. 26.
    Suzuki, S., and others: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985)CrossRefMATHGoogle Scholar
  27. 27.
    Freeman, E., Brewster, S.: Messy tabletops: clearing up the occlusion problem. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems (2013)Google Scholar
  28. 28.
    Jain, A.: Fundamentals of Digital Image Processing. Prentice-Hall, Inc. (1989)Google Scholar
  29. 29.
    Ritter, G., Wilson, J.: Handbook of Computer Vision Algorithms in Image Algebra. CRC (2001)Google Scholar
  30. 30.
    Atsalakis, A., Papamarkos, N.: Color reduction and estimation of the number of dominant colors by using a self-growing and self-organized neural gas. Eng. Appl. Artif. Intell. 19(7), 769–786 (2006)CrossRefGoogle Scholar
  31. 31.
    Kohonen, T.: The self-organizing map. Proc. IEEE 78(9), 1464–1480 (1990)CrossRefGoogle Scholar
  32. 32.
    Fritzke, B., others: A growing neural gas network learns topologies. Adv. Neural Inf. Process. Syst. 7, 625–632 (1995)Google Scholar
  33. 33.
    Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994)CrossRefGoogle Scholar
  34. 34.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–286 (1989)CrossRefGoogle Scholar
  35. 35.
    Liu, N., Lovell, B.C., Kootsookos, P.J., Davis, R.I.: Model structure selection & training algorithms for an HMM gesture recognition system. In: IEEE (2004)Google Scholar
  36. 36.
    Wilson, A.D., Bobick, A.F.: Parametric hidden markov models for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21, 884–900 (1999)CrossRefGoogle Scholar
  37. 37.
    Abdul, Y.F., Wong, F.: Hidden Markov Model-based gesture recognition with overlapping hand-head/hand-hand estimated using kalman filter. In: IEEE (2012)Google Scholar
  38. 38.
    Kohavi, R., others: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence (1995)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Kyriakos Sgouropoulos
    • 1
  • Ekaterini Stergiopoulou
    • 1
  • Nikos Papamarkos
    • 1
  1. 1.Department of Electrical & Computer Engineering, Electric Circuits Analysis LaboratoryDemocritus University of ThraceXanthiGreece

Personalised recommendations