Advertisement

Using Convolutional Neural Networks to Distinguish Different Sign Language Alphanumerics

  • Stephen GreenEmail author
  • Ivan Tyukin
  • Alexander Gorban
Conference paper
Part of the Proceedings of the International Neural Networks Society book series (INNS, volume 1)

Abstract

Using Convolutional Neural Networks (CNN)’s to create Deep Learning systems that turns Sign Language into text has been a vital tool in breaking communication barriers between deaf-mute people. Conventional research on this subject concerns training networks to recognize alphanumerical gestures and produce their textual equivalents.

A problem with current methods is that images are scarce, with little variation in available gestures, often skewed towards skin tones and hand sizes that makes a significant subset of gestures hard to detect. Current identification programs are only trained in a single language despite there being over two-hundred known variants so far. This presents a limitation for traditional exploitation for the state of current technologies such as CNN’s, due to their large number of required parameters.

This work presents a technology that aims to resolve this issue by combining a pretrained legacy AI system for a generic object recognition task with a corrector method to uptrain the legacy network. As a result, a program is created that can receive finger spelling from multiple tactile languages and deduct the corresponding alphanumeric and its language which no other neural network has been able to replicate.

Keywords

Convolutional Neural Networks Sign language Deep Learning Legacy AI 

References

  1. 1.
    World Health Organization: Deafness and hearing loss. http://www.who.int/mediacentre/factsheets/fs300/en/
  2. 2.
  3. 3.
  4. 4.
    British Deaf Association. https://bda.org.uk/help-resources/
  5. 5.
  6. 6.
  7. 7.
    Mohandes, M., Aliyu, S., Deriche, M.: Prototype Arabic Sign language recognition using multi-sensor data fusion of two leap motion controllers. In: 2015 IEEE 12th International Multi-conference on Systems, Signals & Devices (SSD15), Mahdia, pp. 1–6 (2015)Google Scholar
  8. 8.
    Abhishek, K.S., Qubeley, L.C.F., Ho, D.: Glove-based hand gesture recognition sign language translator using capacitive touch sensor. In: 2016 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC). IEEE (2016)Google Scholar
  9. 9.
    Yang, H.-D.: Sign language recognition with the Kinect sensor based on conditional random fields. Sensors (2014)Google Scholar
  10. 10.
    Kumar, V.K., Goudar, R.H., Desai, V.T.: Sign language unification: the need for next generation deaf education. Procedia Comput. Sci. 48, 673–678 (2015).  https://doi.org/10.1016/j.procs.2015.04.151CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Efficiency of Shallow Cascades for Improving Deep Learning AI Systems.  https://doi.org/10.1109/IJCNN.2018.8489266
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
    Tyukin, I.Y., Gorban, A.N., Green, S., Prokhorov, D.: Fast construction of correcting ensembles for legacy artificial intelligence systems: algorithms and a case studyGoogle Scholar
  20. 20.
    Jackson, D.: Stopping rules in principal components analysis: a comparison of heuristical and statistical approaches. Ecology 74(8), 2204–2214 (1993)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Stephen Green
    • 1
    Email author
  • Ivan Tyukin
    • 1
    • 2
  • Alexander Gorban
    • 1
    • 2
  1. 1.Leicester UniversityLeicesterUK
  2. 2.Lobachevsky UniversityNizhny NovgorodRussia

Personalised recommendations