Skip to main content
SpringerLink
Log in

A Kinematic Gesture Representation Based on Shape Difference VLAD for Sign Language Recognition

  • Conference paper
  • First Online:
Computer Vision and Graphics (ICCVG 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11114))

Included in the following conference series:

Abstract

Automatic Sign language recognition (SLR) is a fundamental task to help with inclusion of deaf community in society, facilitating, noways, many conventional multimedia interactions. In this work is proposed a novel approach to represent gestures in SLR as a shape difference-VLAD mid level coding of kinematic primitives, captured along videos. This representation capture local salient motions together with regional dominant patterns developed by articulators along utterances. Also, the special VLAD representation allows to quantify local motion pattern but also capture shape of motion descriptors, that achieved a proper regional gesture characterization. The proposed approach achieved an average accuracy of 85,45% in a corpus data of 64 sign words captured in 3200 videos. Additionally, for Boston sign dataset the proposed approach achieve competitive results with \(82\%\) of accuracy in average.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ali, S., Shah, M.: Human action recognition in videos using kinematic features and multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 288–303 (2010)

    Article  Google Scholar 

  2. Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 41–48. IEEE (2009)

    Google Scholar 

  3. WHO Media Centre: Deafness and hearing loss, March 2018. http://www.who.int/mediacentre/factsheets/fs300/en/

  4. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  5. Chaudhry, R., Ravichandran, A., Hager, G., Vidal, R.: Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1932–1939. IEEE (2009)

    Google Scholar 

  6. Dalal, N., Triggs, B., Schmid, C.: Human detection using oriented histograms of flow and appearance. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 428–441. Springer, Heidelberg (2006). https://doi.org/10.1007/11744047_33

    Chapter  Google Scholar 

  7. Duta, I.C., Uijlings, J.R., Ionescu, B., Aizawa, K., Hauptmann, A.G., Sebe, N.: Efficient human action recognition using histograms of motion gradients and vlad with descriptor shape information. Multimedia Tools Appl. 76, 22445–22472 (2017)

    Article  Google Scholar 

  8. Jain, M., Jegou, H., Bouthemy, P.: Better exploiting motion for better action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2555–2562 (2013)

    Google Scholar 

  9. Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3304–3311. IEEE (2010)

    Google Scholar 

  10. Konecnỳ, J., Hagara, M.: One-shot-learning gesture recognition using HOG-HOF. J. Mach. Learn. Res. 15, 2513–2532 (2014)

    MathSciNet  Google Scholar 

  11. Martínez, F., Manzanera, A., Gouiffès, M., Braffort, A.: A Gaussian mixture representation of gesture kinematics for on-line sign language video annotation. In: Bebis, G., et al. (eds.) ISVC 2015. LNCS, vol. 9475, pp. 293–303. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27863-6_27

    Chapter  Google Scholar 

  12. Paulraj, M., Yaacob, S., Desa, H., Hema, C., Ridzuan, W.M., Ab Majid, W.: Extraction of head and hand gesture features for recognition of sign language. In: International Conference on Electronic Design, ICED 2008, pp. 1–6. IEEE (2008)

    Google Scholar 

  13. Peng, X., Wang, L., Wang, X., Qiao, Y.: Bag of visual words and fusion methods for action recognition: comprehensive study and good practice. Comput. Vis. Image Underst. 150, 109–125 (2016)

    Article  Google Scholar 

  14. Perronnin, F., Sánchez, J., Mensink, T.: Improving the Fisher Kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15561-1_11

    Chapter  Google Scholar 

  15. Ronchetti, F., Quiroga, F., Estrebou, C.A., Lanzarini, L.C., Rosete, A.: LSA64: an Argentinian sign language dataset. In: XXII Congreso Argentino de Ciencias de la Computación (CACIC 2016) (2016)

    Google Scholar 

  16. Wan, J., Ruan, Q., Li, W., Deng, S.: One-shot learning gesture recognition from RGB-D data using bag of features. J. Mach. Learn. Res. 14(1), 2549–2582 (2013)

    Google Scholar 

  17. Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 279–286. ACM (2011)

    Google Scholar 

  18. Zahedi, M., Keysers, D., Deselaers, T., Ney, H.: Combination of tangent distance and an image distortion model for appearance-based sign language recognition. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 401–408. Springer, Heidelberg (2005). https://doi.org/10.1007/11550518_50

    Chapter  Google Scholar 

  19. Zahedi, M., Keysers, D., Ney, H.: Appearance-based recognition of words in American sign language. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 511–519. Springer, Heidelberg (2005). https://doi.org/10.1007/11492429_62

    Chapter  Google Scholar 

  20. Zaki, M.M., Shaheen, S.I.: Sign language recognition using a combination of new vision based features. Pattern Recogn. Lett. 32(4), 572–577 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

The authors acknowledge the Vicerrectoría de Investigación y Extensión of the Universidad Industrial de Santander for supporting this research registered by the project: Análisis de movimientos salientes en espacios comprimidos para la caracterización eficiente de videos multiespectrales, with VIE code 2347”

Author information

Authors and Affiliations

  1. Grupo de investigación en ingeniería biomédica (GIIB), Bucaramanga, Colombia

    Jefferson Rodríguez & Fabio Martínez

  2. Motion Analysis and Computer Vision (MACV), Bucaramanga, Colombia

    Jefferson Rodríguez & Fabio Martínez

  3. Universidad Industrial de Santander (UIS), Bucaramanga, Colombia

    Jefferson Rodríguez & Fabio Martínez

Authors
  1. Jefferson Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jefferson Rodríguez .

Editor information

Editors and Affiliations

  1. Faculty of Applied Informatics and Mathematics, Warsaw University of Life Sciences, Warsaw, Poland

    Leszek J. Chmielewski

  2. Faculty of Applied Informatics and Mathematics, Warsaw University of Life Sciences, Warsaw, Poland

    Ryszard Kozera

  3. Faculty of Applied Informatics and Mathematics, Warsaw University of Life Sciences, Warsaw, Poland

    Arkadiusz Orłowski

  4. Institute of Computer Science, Silesian University of Technology, Gliwice, Poland

    Konrad Wojciechowski

  5. Technion, Israel Institute of Technology, Haifa, Israel

    Alfred M. Bruckstein

  6. University of Groningen, Groningen, The Netherlands

    Nicolai Petkov

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rodríguez, J., Martínez, F. (2018). A Kinematic Gesture Representation Based on Shape Difference VLAD for Sign Language Recognition. In: Chmielewski, L., Kozera, R., Orłowski, A., Wojciechowski, K., Bruckstein, A., Petkov, N. (eds) Computer Vision and Graphics. ICCVG 2018. Lecture Notes in Computer Science(), vol 11114. Springer, Cham. https://doi.org/10.1007/978-3-030-00692-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00692-1_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00691-4

  • Online ISBN: 978-3-030-00692-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation