Advertisement

Hand gesture recognition using topological features

  • Narges Mirehi
  • Maryam TahmasbiEmail author
  • Alireza Tavakoli Targhi
Article
  • 33 Downloads

Abstract

Hand Gestures Recognition (HGR) is one of the main areas of research for Human Computer Interaction applications. Most existing approaches are based on local or geometrical properties of pixels. Still, there are some serious challenges on HGR methods such as sensitivity to rotation, scale, illumination, perturbation, and occlusion. In this paper, we study HGR from graph viewpoints. We introduce a set of meaningful shape features based on a graph constructed by Growing Neural Gas (GNG) algorithm. These features are constructed from topological properties of this graph. Graph properties in conserving topological features improve stability against different deformations, scale, and noise. We evaluate our method on NTU Hand Digits dataset with state-of-the-art methods. We also prepared a comprehensive dataset (SBU-1) for different hand gestures containing 2170 images. This dataset includes many possible deformations and variations and some articulations. Most of the existing datasets don’t capture these variations. We show the robustness of the algorithm to scale, rotation and noise, while preserving similar recognition rate in comparison with the state-of-the-art results.

Keywords

Hand gesture recognition Growing Neural Gas algorithm Topological features Adjacency matrix Linear Discriminant Analysis (LDA) NTU Hand Digits dataset 

Notes

Compliance with Ethical Standards

Conflict of Interest

Narges Mirehi declares that she has no conflict of interest. Maryam Tahmasbi declares that she has no conflict of interest. Alireza Tavakoli Targhi declares that he has no conflict of interest. The authors declare that they have no conflict of interest.

References

  1. 1.
    Bai X, Latecki LJ (2008) Path similarity skeleton graph matching. IEEE Trans Pattern Anal Mach Intell 30(7):1282–1292CrossRefGoogle Scholar
  2. 2.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522CrossRefGoogle Scholar
  3. 3.
    Bondy J, Murty U (2008) Graph theory, 2nd printing. SpringerGoogle Scholar
  4. 4.
    Bourke A, Obrien J, Lyons G (2007) Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. Gait Posture 26(2):194–199CrossRefGoogle Scholar
  5. 5.
    Bretzner L, Laptev I, Lindeberg T (2002) Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering. In: Proceedings. Fifth IEEE international conference on automatic face and gesture recognition. IEEE, pp 423–428Google Scholar
  6. 6.
    Chaudhary A, Raheja J, Das K, Raheja S (2011) A survey on hand gesture recognition in context of soft computing. In: International conference on computer science and information technology. Springer, pp 46–55Google Scholar
  7. 7.
    Cheng H, Dai Z, Liu Z, Zhao Y (2016) An image-to-class dynamic time warping approach for both 3d static and trajectory hand gesture recognition. Pattern Recognit 55:137–147CrossRefGoogle Scholar
  8. 8.
    Corera S, Krishnarajah N (2011) Capturing hand gesture movement: a survey on tools techniques and logical considerations. In: Proceedings of chi sparksGoogle Scholar
  9. 9.
    De Berg M, Van Kreveld M, Overmars M, Schwarzkopf OC (2000) Computational geometry. Springer, BerlinzbMATHCrossRefGoogle Scholar
  10. 10.
    Dong C, Leu MC, Yin Z (2015) American sign language alphabet recognition using microsoft kinect. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–52Google Scholar
  11. 11.
    Flórez F, García JM, García J, Hernández A (2002) Hand gesture recognition following the dynamics of a topology-preserving network. In: Proceedings. Fifth IEEE international conference on automatic face and gesture recognition. IEEE, pp 318–323Google Scholar
  12. 12.
    Fritzke B et al (1995) A growing neural gas network learns topologies. In: Advances in neural information processing systems, vol 7, pp 625–632Google Scholar
  13. 13.
    Fink O, Zio E, Weidmann U (2015) Novelty detection by multivariate kernel density estimation and growing neural gas algorithm. Mech Syst Signal Process 50:427–436CrossRefGoogle Scholar
  14. 14.
    Ge SS, Yang Y, Lee TH (2008) Hand gesture recognition and tracking based on distributed locally linear embedding. Image Vis Comput 26(12):1607–1620CrossRefGoogle Scholar
  15. 15.
    Hall ET et al (1959) The silent language, vol 3. Doubleday, New YorkGoogle Scholar
  16. 16.
    Holdstein Y, Fischer A (2008) Three-dimensional surface reconstruction using meshing growing neural gas (MGNG). Vis Comput 24(4):295–302CrossRefGoogle Scholar
  17. 17.
    Karam M (2006) Phd thesis: a framework for research and design of gesture-based humancomputer interactions. PhD thesis, University of SouthamptonGoogle Scholar
  18. 18.
    Kirac F, Kara YE, Akarun L (2014) Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data. Pattern Recogn Lett 50:91–100CrossRefGoogle Scholar
  19. 19.
    Kim TK, Cipolla R (2009) Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans Pattern Anal Mach Intell 31(8):1415–1428CrossRefGoogle Scholar
  20. 20.
    Klein HA (2012) The science of measurement: a historical survey, Courier Corporation, North ChelmsfordGoogle Scholar
  21. 21.
    Li S, Zhang H (2004) Floatboost learning and statistical face detection. IEEE Trans Pattern Anal Mach Intell 26(9):1112–1123CrossRefGoogle Scholar
  22. 22.
    Li Y, Wang X, Liu W, Feng B (2018) Deep attention network for joint hand gesture localization and recognition using static RGB-D images. Inf Sci 441:66–78MathSciNetCrossRefGoogle Scholar
  23. 23.
    Liu K, Kehtarnavaz N (2016) Real-time robust vision-based hand gesture recognition using stereo images. J Real-Time Image Proc 11(1):201–209CrossRefGoogle Scholar
  24. 24.
    Marcel S, Bernier O, Viallet JE, Collobert D (2000) Hand gesture recognition using input-output hidden markov models. In: Automatic face and gesture recognition. IEEE, pp 456–461Google Scholar
  25. 25.
    Maqueda AI, del-Blanco CR, Jaureguizar F, García N (2015) Human–computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns. Comput Vis Image Underst 141:126–37CrossRefGoogle Scholar
  26. 26.
    Martinetz T, Schulten K et al (1991) A “neuralgas” network learns topologies. University of Illinois at Urbana-ChampaignGoogle Scholar
  27. 27.
    Memo A, Zanuttigh P (2018) Head-mounted gesture controlled interface for human-computer interaction. Multimed Tools Appl 77(1):27–53CrossRefGoogle Scholar
  28. 28.
    Mittal A, Zisserman A, Torr PH (2011) Hand detection using multiple proposals. In: BMVC, pp 1–11Google Scholar
  29. 29.
    Molina J, Pajuelo JA, Escudero-Vi nolo M, Bescós J, Martínez JM (2014) A natural and synthetic corpus for benchmarking of hand gesture recognition systems. Mach Vis Appl 25(4):943–954CrossRefGoogle Scholar
  30. 30.
    Munib Q, Habeeb M, Takruri B, Al-Malik HA (2007) American sign language (asl) recognition based on hough transform and neural networks. Exp Syst Appl 32(1):24–37CrossRefGoogle Scholar
  31. 31.
    Nagi J, Ducatelle F, Di Caro GA, Cireşan D, Meier U, Giusti A, Nagi F, Schmidhuber J, Gambardella LM (2011) Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: IEEE international conference on signal and image processing applications (ICSIPA), vol 2011. IEEE, pp 342–347Google Scholar
  32. 32.
    Nai W, Liu Y, Rempel D, Wang Y (2017) Fast hand posture classification using depth features extracted from random line segments. Pattern Recogn 65:1–10CrossRefGoogle Scholar
  33. 33.
    Orts-Escolano S, Garcia-Rodriguez J, Morell V, Cazorla M, Perez JA, Garcia-Garcia A (2016) 3d surface reconstruction of noisy point clouds using growing neural gas: 3d object/scene reconstruction. Neural Process Lett 43(2):401–423CrossRefGoogle Scholar
  34. 34.
    Pattanaworapan K, Chamnongthai K, Guo JM (2016) Signer-independence finger alphabet recognition using discrete wavelet transform and area level run lengths. J Vis Commun Image Represent 38:658–677CrossRefGoogle Scholar
  35. 35.
    Peng B, Zhang L, Zhang D (2013) A survey of graph theoretical approaches to image segmentation. Pattern Recogn 46(3):1020–1038CrossRefGoogle Scholar
  36. 36.
    Phung SL, Bouzerdoum A, Chai D (2005) Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans Pattern Anal Mach Intell 27(1):148–154CrossRefGoogle Scholar
  37. 37.
    Plouffe G, Cretu AM (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–16CrossRefGoogle Scholar
  38. 38.
    Priyal SP, Bora PK (2013) A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments. Pattern Recogn 46(8):2202–2219zbMATHCrossRefGoogle Scholar
  39. 39.
    Pugeault N, Bowden R (2011) Spelling it out: realtime asl fingerspelling recognition. In: IEEE international conference on computer vision workshops (ICCV Workshops), 2011. IEEE, pp 1114--1119Google Scholar
  40. 40.
    Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54CrossRefGoogle Scholar
  41. 41.
    Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM international conference on Multimedia. ACM, pp 1093–1096Google Scholar
  42. 42.
    Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimedia 15(5):1110–20CrossRefGoogle Scholar
  43. 43.
    Siddiqi K, Shokoufandeh A, Dickinson SJ, Zucker SW (1999) Shock graphs and shape matching. Int J Comput Vis 35(1):13–32CrossRefGoogle Scholar
  44. 44.
    Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158CrossRefGoogle Scholar
  45. 45.
    Stergiopoulou E, Sgouropoulos K, Nikolaou N, Papamarkos N, Mitianoudis N (2014) Real time hand detection in a complex background. Eng Appl Artif Intell 35:54–70CrossRefGoogle Scholar
  46. 46.
    Sun Q, Liu H, Harada T (2017) Online growing neural gas for anomaly detection in changing surveillance scenes. Pattern Recogn 64:187–201CrossRefGoogle Scholar
  47. 47.
    Viola P, Jones M (2002) Fast and robust classification using asymmetric adaboost and a detector cascade. In: Advances in neural information processing systems, pp 1311–1318Google Scholar
  48. 48.
    Wang C, Liu Z, Zhu M, Zhao J, Chan SC (2017) A hand gesture recognition system based on canonical superpixel-graph. Signal Process Image Commun 58:87–98CrossRefGoogle Scholar
  49. 49.
    Wang Y, Yang R (2013) Real-time hand posture recognition based on hand dominant line using kinect. In: IEEE international conference multimedia and expo workshops (ICMEW). IEEE, pp 1–4Google Scholar
  50. 50.
    Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology. ACM, pp 159–168Google Scholar
  51. 51.
    Wu Y, Huang TS (2001) Hand modeling, analysis and recognition. IEEE Signal Process Mag 18(3):51–60CrossRefGoogle Scholar
  52. 52.
    Xie S, Pan J (2011) Hand detection using robust color correction and gaussian mixture model. In: Sixth international conference on image and graphics (ICIG). IEEE, pp 553–557Google Scholar
  53. 53.
    Zhang C, Yang X, Tian Y (2013) Histogram of 3d facets: a characteristic descriptor for hand gesture recognition. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–8Google Scholar
  54. 54.
    Zhang H, Fritts JE, Goldman SA (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Narges Mirehi
    • 1
  • Maryam Tahmasbi
    • 1
    Email author
  • Alireza Tavakoli Targhi
    • 1
  1. 1.Department of Computer ScienceShahid Beheshti UniversityTehranIran

Personalised recommendations