Skip to main content
SpringerLink
Log in

Towards On-Line Sign Language Recognition Using Cumulative SD-VLAD Descriptors

  • Conference paper
  • First Online:
Advances in Computing (CCC 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 885))

Included in the following conference series:

Abstract

On-line prediction of sign language gestures is nowadays a fundamental task to help and support multimedia interpretation of deaf communities. This work presents a novel approach to recover partial sign language gestures by cumulative coding different intervals of the video sequences. The method starts by computing volumetric patches that contain kinematic information from different appearance flow primitives. Then, several sequential intervals are learned to carry out the task of partial recognition. For each new video, a cumulative shape difference (SD)-VLAD representation is obtained at different intervals of the video. Each SD-VLAD descriptor recovers mean and variance motion information as signature of the computed gesture. Along the video, each partial representation is mapped to a support vector machine model to obtain a gesture recognition, being usable in on-line scenarios. The proposed approach was evaluated in a public dataset with 64 different classes, recorded in 3200 samples. This approach is able to recognize sign gestures using only 20% of the sequence with an average accuracy of 53.8% and with 60% of information, the 80% of accuracy was achieved. For complete sequences the proposed approach achieves 85% on average.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 41–48. IEEE (2009)

    Google Scholar 

  2. Vaughan, G.: Deafness and hearing loss (2018). http://www.who.int/en/news-room/fact-sheets/detail/deafness-and-hearing-loss

  3. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Tech. (TIST) 2(3), 27 (2011)

    Google Scholar 

  4. Chaudhry, R., Ravichandran, A., Hager, G., Vidal, R.: Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1932–1939. IEEE (2009)

    Google Scholar 

  5. Chen, X., Koskela, M.: Online RGB-D gesture recognition with extreme learning machines. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, pp. 467–474. ACM (2013)

    Google Scholar 

  6. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 428–441. Springer, Heidelberg (2006). https://doi.org/10.1007/11744047_33

    Chapter  Google Scholar 

  7. Duta, I.C., Uijlings, J.R., Ionescu, B., Aizawa, K., Hauptmann, A.G., Sebe, N.: Efficient human action recognition using histograms of motion gradients and VLAD with descriptor shape information. Multimed. Tools Appl. 76, 1–28 (2017)

    Article  Google Scholar 

  8. Fan, Z., Lin, T., Zhao, X., Jiang, W., Xu, T., Yang, M.: An online approach for gesture recognition toward real-world applications. In: Zhao, Y., Kong, X., Taubman, D. (eds.) ICIG 2017. LNCS, vol. 10666, pp. 262–272. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71607-7_23

    Chapter  Google Scholar 

  9. Jain, M., Jegou, H., Bouthemy, P.: Better exploiting motion for better action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2555–2562 (2013)

    Google Scholar 

  10. Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311. IEEE (2010)

    Google Scholar 

  11. Konecnỳ, J., Hagara, M.: One-shot-learning gesture recognition using HOG-HOF features. J. Mach. Learn. Res. 15, 2513–2532 (2014)

    MathSciNet  Google Scholar 

  12. Liu, Z., et al.: Real-time sign language recognition with guided deep convolutional neural networks. In: Proceedings of the 2016 Symposium on Spatial User Interaction, pp. 187–187. ACM (2016)

    Google Scholar 

  13. Martínez, F., Manzanera, A., Gouiffès, M., Braffort, A.: A Gaussian mixture representation of gesture kinematics for on-line sign language video annotation. In: Bebis, G. (ed.) ISVC 2015. LNCS, vol. 9475, pp. 293–303. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27863-6_27

    Chapter  Google Scholar 

  14. Masood, S., Srivastava, A., Thuwal, H.C., Ahmad, M.: Real-time sign language gesture (word) recognition from video sequences using CNN and RNN. In: Bhateja, V., Coello Coello, C.A., Satapathy, S.C., Pattnaik, P.K. (eds.) Intelligent Engineering Informatics. AISC, vol. 695, pp. 623–632. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-7566-7_63

    Chapter  Google Scholar 

  15. Neto, G.M.R., Junior, G.B., de Almeida, J.D.S., de Paiva, A.C.: Sign language recognition based on 3D convolutional neural networks. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 399–407. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_45

    Chapter  Google Scholar 

  16. Paulraj, M., Yaacob, S., Desa, H., Hema, C., Ridzuan, W.M., Ab Majid, W.: Extraction of head and hand gesture features for recognition of sign language. In: International Conference on Electronic Design, ICED 2008, pp. 1–6. IEEE (2008)

    Google Scholar 

  17. Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_11

    Chapter  Google Scholar 

  18. Ronchetti, F., Quiroga, F., Estrebou, C.A., Lanzarini, L.C., Rosete, A.: LSA64: an Argentinian sign language dataset. In: XXII Congreso Argentino de Ciencias de la Computación (CACIC 2016) (2016)

    Google Scholar 

  19. Tofighi, G., Monadjemi, S.A., Ghasem-Aghaee, N.: Rapid hand posture recognition using adaptive histogram template of skin and hand edge contour. In: 2010 6th Iranian Machine Vision and Image Processing (MVIP), pp. 1–5. IEEE (2010)

    Google Scholar 

  20. Wan, J., Ruan, Q., Li, W., Deng, S.: One-shot learning gesture recognition from RGB-D data using bag of features. J. Mach. Learn. Res. 14(1), 2549–2582 (2013)

    Google Scholar 

  21. Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 279–286. ACM (2011)

    Google Scholar 

  22. Zahedi, M., Keysers, D., Ney, H.: Appearance-based recognition of words in American sign language. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 511–519. Springer, Heidelberg (2005). https://doi.org/10.1007/11492429_62

    Chapter  Google Scholar 

Download references

Acknowledgments

“The authors acknowledge the Vicerrectoría de Investigación y Extensión of the Universidad Industrial de Santander for supporting this research registered by the project: Análisis de movimientos salientes en espacios comprimidos para la caracterización eficiente de videos multiespectrales, with VIE code 2347”.

Author information

Authors and Affiliations

  1. Grupo de investigación en ingeniería biomédica (GIIB), Motion Analysis and Computer Vision (MACV), Universidad Industrial de Santander (UIS), Bucaramanga, Colombia

    Jefferson Rodríguez & Fabio Martínez

Authors
  1. Jefferson Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jefferson Rodríguez .

Editor information

Editors and Affiliations

  1. Universidad Tecnológica de Bolívar, Cartagena, Colombia

    Jairo E. Serrano C.

  2. Universidad Tecnológica de Bolívar, Cartagena, Colombia

    Juan Carlos Martínez-Santos

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rodríguez, J., Martínez, F. (2018). Towards On-Line Sign Language Recognition Using Cumulative SD-VLAD Descriptors. In: Serrano C., J., Martínez-Santos, J. (eds) Advances in Computing. CCC 2018. Communications in Computer and Information Science, vol 885. Springer, Cham. https://doi.org/10.1007/978-3-319-98998-3_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98998-3_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98997-6

  • Online ISBN: 978-3-319-98998-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation