Skip to main content
SpringerLink
Log in

Fingerspelling Recognition Using Histogram of Oriented Point Cloud Vectors from Depth Data

  • Conference paper
  • First Online:
Proceedings of the 4th Brazilian Technology Symposium (BTSym'18) (BTSym 2018)

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 140))

Included in the following conference series:

Abstract

The high degree of freedom of hand movements produces a high variability of shapes and hand appearances that still challenges hand gesture recognition algorithms. This paper presents an approach to recognize sign language fingerspelling. Our approach, named histogram of oriented point cloud vectors (HOPC), is based on a new descriptor computed only from depth images. The segmented depth image is mapped into a 3D point cloud and divided into subspaces. In each subspace, 3D point vectors are mapped into their spherical coordinates around its centroid. Next, it is computed their orientations angles \(H_\varphi \) and \(H_\theta \) onto two cumulative histograms. Normalized histograms are concatenated to form the image descriptor and used to train a Support Vector Machine classifier (SVM). To assess the feasibility of our approach, we evaluated it on a public data-set of American Sign Language (ASL) composed of more than 60,000 images. Our experiments showed a recognition accuracy average of 99.46%, achieving the state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chanu, O.R., Pillai, A., Sinha, S., Das, P.: Comparative study for vision based and data based hand gesture recognition technique. In: 2017 International Conference on Intelligent Communication and Computational Techniques (ICCT), pp. 26–31. IEEE, Dec 2017

    Google Scholar 

  2. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893 (2005)

    Google Scholar 

  3. Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K.: Consumer Depth Cameras for Computer Vision: Research Topics and Applications. Springer Publishing Company, Incorporated (2013)

    Google Scholar 

  4. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Prentice-Hall, Inc. (2008)

    Google Scholar 

  5. Isaacs, J., Foo, S.: Hand pose estimation for American sign language recognition. In: Proceedings of the Thirty-Sixth Southeastern Symposium on System Theory, 2004, pp. 132–136. IEEE (2004)

    Google Scholar 

  6. Joudaki, S., Mohamad, D.B., Saba, T., Rehman, A., Al-Rodhaan, M., Al-Dhelaan, A.: Vision-based sign language classification: a directional review. IETE Tech Rev 31(5), 383–391 (2014)

    Article  Google Scholar 

  7. Keskin, C., Kirac, F., Kara, Y.E., Akarun, L.: Hand pose estimation and hand shape classification using multi-layered randomized decision forests. In: Proceedings of the 12th European Conference on Computer Vision—Volume Part VI. ECCV’12, pp. 852–863. Springer, Berlin, Heidelberg (2012)

    Chapter  Google Scholar 

  8. Kuznetsova, A., Leal-Taixe, L., Rosenhahn, B.: Real-time sign language recognition using a consumer depth camera. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 83–90. IEEE, Dec 2013

    Google Scholar 

  9. Li, S.Z., Yu, B., Wu, W., Su, S.Z., Ji, R.R.: Feature learning based on SAE-PCA network for human gesture recognition in RGBD images. Neurocomputing 151, 565–573 (2015)

    Article  Google Scholar 

  10. Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press (2012)

    Google Scholar 

  11. Otiniano-Rodríguez, K.C., Cámara-Chávez, G., Menotti, D.: Hu and Zernike moments for sign language recognition. In: Proceedings of the 2012 International Conference on Image Processing, Computer Vision, and Pattern Recognition. IPCV 2012, vol. 2, pp. 918–922 (2012)

    Google Scholar 

  12. Padden, C.A.: Learning fingerspelling twice: young signing children’s acquisition of fingerspelling. In: Schick, B., Marschark, M., Spencer, P.E. (eds.) Advances in the Sign Language Development of Deaf Children, pp. 189–201. Oxford University Press (2005)

    Google Scholar 

  13. Peach, D.: David Peach’s LearnSigns.com. http://www.learnsigns.com/sign-language-alphabet-asl/ (2018). Accessed on 24 June 2018

  14. Pugeault, N., Bowden, R.: Spelling it out: real-time ASL fingerspelling recognition. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1114–1119, Nov 2011

    Google Scholar 

  15. Rioux-Maldague, L., Giguere, P.: Sign language fingerspelling classification from depth and color images using a deep belief network. In: 2014 Canadian Conference on Computer and Robot Vision, pp. 92–97. IEEE, May 2014

    Google Scholar 

  16. Sahoo, A.K., Mishra, G.S., Ravulakollu, K.K.: Sign language recognition: state of the art. ARPN J. Eng. Appl. Sci. 9(2), 116–134 (2014)

    Google Scholar 

  17. Sandler, W., Lillo-Martin, D.: Sign Language and Linguistic Universals. Cambridge University Press (2006)

    Google Scholar 

  18. Suarez, J., Murphy, R.: Hand gesture recognition with depth images: a review. In: 2012 IEEE RO-MAN. pp. 411–417 (2012)

    Google Scholar 

  19. Tang, S., Wang, X., Lv, X., Han, T.X., Keller, J., He, Z., Skubic, M., Lao, S.: Histogram of oriented normal vectors for object recognition with a depth sensor. In: Proceedings of the 11th Asian Conference on Computer Vision—Volume Part II, pp. 525–538. Springer (2013)

    Google Scholar 

  20. Weerasekera, C.S., Jaward, M.H., Kamrani, N.: Robust ASL fingerspelling recognition using local binary patterns and geometric features. In: 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8. IEEE, Nov 2013

    Google Scholar 

  21. Xiaolong, Z., Wong, K.: Single-frame hand gesture recognition using color and depth kernel descriptors. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 2989–2992, Nov 2012

    Google Scholar 

  22. Zanuttigh, P., Marin, G., Dal Mutto, C., Dominio, F., Minto, L., Cortelazzo, G.M.: Time-of-Flight and Structured Light Depth Cameras. Springer International Publishing (2016)

    Google Scholar 

  23. Zheng, L., Liang, B., Jiang, A.: Recent advances of deep learning for sign language recognition. In: 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–7. IEEE, Nov 2017

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Technological Sciences and Engineering, University of Ayacucho Federico Froebel, Ayacucho, Peru

    José Elías Yauri Vidalón

  2. School of Electrical and Computer Engineering, University of Campinas, Campinas, SP, Brazil

    José Elías Yauri Vidalón & José Mario De Martino

Authors
  1. José Elías Yauri Vidalón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Mario De Martino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Elías Yauri Vidalón .

Editor information

Editors and Affiliations

  1. Faculty of Electrical and Computing Engineering, University of Campinas, Campinas, São Paulo, Brazil

    Yuzo Iano

  2. Faculty of Technology, University of Campinas, Campinas, São Paulo, Brazil

    Rangel Arthur

  3. Divisão de Engenharia Eletrônica, Instituto Tecnológico de Aeronáutica ITA, São José dos Campos, São Paulo, Brazil

    Osamu Saotome

  4. Universidade Federal Fluminense (UFF), Duque de Caxias, Rio de Janeiro, Brazil

    Vânia Vieira Estrela

  5. Faculty of Electrical and Computing Engineering, University of Campinas, Campinas, São Paulo, Brazil

    Hermes José Loschi

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yauri Vidalón, J.E., De Martino, J.M. (2019). Fingerspelling Recognition Using Histogram of Oriented Point Cloud Vectors from Depth Data. In: Iano, Y., Arthur, R., Saotome, O., Vieira Estrela, V., Loschi, H. (eds) Proceedings of the 4th Brazilian Technology Symposium (BTSym'18). BTSym 2018. Smart Innovation, Systems and Technologies, vol 140. Springer, Cham. https://doi.org/10.1007/978-3-030-16053-1_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-16053-1_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-16052-4

  • Online ISBN: 978-3-030-16053-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation