Hand Posture Recognition Using Convolutional Neural Network

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10657)


In this work we present a convolutional neural network-based algorithm for recognition of hand postures on images acquired by a single color camera. The hand is extracted in advance on the basis of skin color distribution. A neural network-based regressor is applied to locate the wrist. Finally, a convolutional neural network trained on 6000 manually labeled images representing ten classes is executed to recognize the hand posture in a sub-window determined on the basis of the wrist. We show that our model achieves high classification accuracy, including scenarios with different camera used in testing. We show that the convolutional network achieves better results on images pre-filtered by a Gabor filter.


Gesture recognition Biologically inspired computer vision Gabor filter Convolutional neural network 



This work was supported by Polish National Science Center (NCN) under a research grant 2014/15/B/ST6/02808.


  1. 1.
    Oyedotun, O., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017)CrossRefGoogle Scholar
  2. 2.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  3. 3.
    Kwolek, B.: Face detection using convolutional neural networks and Gabor filters. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3696, pp. 551–556. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  4. 4.
    Arel, I., Rose, D., Karnowski, T.: Research frontier: deep machine learning-a new frontier in artificial intelligence research. Comput. Intell. Mag. 5(4), 13–18 (2010)CrossRefGoogle Scholar
  5. 5.
    Tompson, J., Stein, M., Lecun, Y., Perlin, K.: Real-time continuous pose recovery of human hands using convolutional networks. ACM Trans. Graph. 33(5), 169:1–169:10 (2014)CrossRefGoogle Scholar
  6. 6.
    Nagi, J., Ducatelle, F., et al.: Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: IEEE ICSIP, pp. 342–347 (2011)Google Scholar
  7. 7.
    Barros, P., Magg, S., Weber, C., Wermter, S.: A multichannel convolutional neural network for hand posture recognition. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 403–410. Springer, Cham (2014). Google Scholar
  8. 8.
    Koller, O., Ney, H., Bowden, R.: Deep Hand: how to train a CNN on 1 million hand images when your data is continuous and weakly labelled. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3793–3802 (2016)Google Scholar
  9. 9.
    Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678 (2014)Google Scholar
  10. 10.
    Pisharady, P., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition. Comput. Vis. Image Underst. 141, 152–165 (2015)CrossRefGoogle Scholar
  11. 11.
    Pisharady, P., Vadakkepat, P., Loh, A.: Attention based detection and recognition of hand postures against complex backgrounds. IJCV 101(3), 403–419 (2013)CrossRefGoogle Scholar
  12. 12.
    Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. Int. J. Comput. Vis. 46(1), 81–96 (2002)CrossRefMATHGoogle Scholar
  13. 13.
    Hubel, D., Wiesel, T.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962)CrossRefGoogle Scholar
  14. 14.
    Petkov, N.: Biologically motivated computationally intensive approaches to image pattern recognition. Future Gener. Comput. Syst. 11(4–5), 451–465 (1995)CrossRefGoogle Scholar
  15. 15.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324 (1998)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.AGH University of Science and TechnologyKrakówPoland
  2. 2.National University of EngineeringLimaPeru

Personalised recommendations