Skip to main content
SpringerLink
Log in

3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories

  • Conference paper
  • First Online:
Understanding Human Activities Through 3D Sensors (UHA3DS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10188))

Abstract

Hand gesture recognition is recently becoming one of the most attractive field of research in Pattern Recognition. In this paper, a skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we consider the sequential data of hand geometric configuration to capture the hand shape variation, and explore the temporal character of hand motion. 3D Hand gesture are represented as a set of relevant spatiotemporal motion trajectories of hand-parts in an Euclidean space. Trajectories are then interpreted as elements lying on Riemannian manifold of shape space to capture their shape variations and achieve gesture recognition using a linear SVM classifier.

The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two different numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www-rech.telecom-lille.fr/DHGdataset.

References

  1. Kuznetsova, A., Leal-Taixé, L., Rosenhahn, B.: Real-time sign language recognition using a consumer depth camera. In: IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 83–90, December 2013

    Google Scholar 

  2. Wang, H., Wang, Q., Chen, X.: Hand posture recognition from disparity cost map. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7725, pp. 722–733. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37444-9_56

    Chapter  Google Scholar 

  3. Ren, Z., Yuan, J., Zhang, Z.: Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: ACM International Conference on Multimedia, MM 2011, pp. 1093–1096. ACM, New York (2011)

    Google Scholar 

  4. Cheng, H., Dai, Z., Liu, Z.: Image-to-class dynamic time warping for 3D hand gesture recognition. In: 2013 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, July 2013

    Google Scholar 

  5. Pugeault, N., Bowden, R.: Spelling it out: real-time ASL fingerspelling recognition. In: IEEE Computer Vision Workshops (ICCV Workshops), pp. 1114–1119, November 2011

    Google Scholar 

  6. Dong, C., Leu, M.C., Yin, Z.: American sign language alphabet recognition using microsoft kinect. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 44–52, June 2015

    Google Scholar 

  7. Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: IEEE International Conference on Image Processing (ICIP), pp. 1565–1569 (2014)

    Google Scholar 

  8. Kurakin, A., Zhang, Z., Liu, Z.: A real time system for dynamic hand gesture recognition with a depth sensor. In: 20th European Signal Processing Conference (EUSIPCO), pp. 1975–1979, August 2012

    Google Scholar 

  9. Zhang, C., Yang, X., Tian, Y.: Histogram of 3D facets: a characteristic descriptor for hand gesture recognition. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–8, April 2013

    Google Scholar 

  10. Escalera, S., et al.: ChaLearn looking at people challenge 2014: dataset and results. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 459–473. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_32

    Chapter  Google Scholar 

  11. Monnier, C., German, S., Ost, A.: A multi-scale boosted detector for efficient and robust gesture recognition. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 491–502. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_34

    Chapter  Google Scholar 

  12. Neverova, N., Wolf, C., Taylor, G.W., Nebout, F.: ModDrop: adaptive multi-modal gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell., April 2016

    Google Scholar 

  13. Ohn-Bar, E., Trivedi, M.M.: Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans. Intell. Transp. Syst. 15(6), 2368–2377 (2014)

    Article  Google Scholar 

  14. De Smedt, Q., Wannous, H., Vandeborre, J.P.: Skeleton-based dynamic hand gesture recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016

    Google Scholar 

  15. Lee, C.S., Elgammal, A.M.: Modeling view and posture manifolds for tracking. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

  16. Lui, Y.M.: Advances in matrix manifolds for computer vision. Image Vis. Comput. 30, 380–388 (2012)

    Article  Google Scholar 

  17. Harandi, M.T., Sanderson, C., Shirazi, S., Lovell, B.C.: Kernel analysis on grassmann manifolds for action recognition. Pattern Recogn. Lett. 34, 1906–1915 (2013)

    Article  Google Scholar 

  18. Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 588–595. IEEE Computer Society, Washington, DC (2014)

    Google Scholar 

  19. Slama, R., Wannous, H., Daoudi, M., Srivastava, A.: Accurate 3D action recognition using learning on the Grassmann manifold. Pattern Recogn. 48(2), 556–567 (2015)

    Article  Google Scholar 

  20. Slama, R., Wannous, H., Daoudi, M.: 3D human motion analysis framework for shape similarity and retrieval. Image Vis. Comput. 32(2), 131–154 (2014)

    Article  Google Scholar 

  21. Joshi, S.H., Klassen, E., Srivastava, A., Jermyn, I.: A novel representation for Riemannian analysis of elastic curves in \(R^n\). In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, pp. 1–7, June 2007

    Google Scholar 

  22. Karcher, H.: Riemannian center of mass and mollifier smoothing. Comm. Pure Appl. Math. 30, 509–541 (1977)

    Article  MathSciNet  Google Scholar 

  23. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)

    Article  Google Scholar 

  24. Oreifej, O., Liu, Z.: HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 716–723 (2013)

    Google Scholar 

  25. Ohn-Bar, E., Trivedi, M.M.: Joint angles similarities and HOG2 for action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2013, Portland, OR, USA, 23–28 June 2013, pp. 465–470 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Télécom Lille, CNRS, UMR 9189 - CRIStAL, 59000, Lille, France

    Quentin De Smedt & Jean-Philippe Vandeborre

  2. Télécom Lille, Univ. Lille, CNRS, UMR 9189 - CRIStAL, 59000, Lille, France

    Hazem Wannous

Authors
  1. Quentin De Smedt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hazem Wannous
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean-Philippe Vandeborre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hazem Wannous .

Editor information

Editors and Affiliations

  1. University of Lille, Lille, France

    Hazem Wannous

  2. University of Florence, Florence, Italy

    Pietro Pala

  3. IMT Lille Douai, Villeneuve-d’Ascq, France

    Mohamed Daoudi

  4. University of Alicante, Alicante, Spain

    Francisco Flórez-Revuelta

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

De Smedt, Q., Wannous, H., Vandeborre, JP. (2018). 3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories. In: Wannous, H., Pala, P., Daoudi, M., Flórez-Revuelta, F. (eds) Understanding Human Activities Through 3D Sensors. UHA3DS 2016. Lecture Notes in Computer Science(), vol 10188. Springer, Cham. https://doi.org/10.1007/978-3-319-91863-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91863-1_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91862-4

  • Online ISBN: 978-3-319-91863-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation