ASL Recognition and Conversion to Speech

  • Simran KharpudeEmail author
  • Vaishnavi Hardikar
  • Gautam Munot
  • Omkar Lonkar
  • Vanita Agarwal
Conference paper
Part of the Lecture Notes on Data Engineering and Communications Technologies book series (LNDECT, volume 44)


The deaf & dumb (or the Mute community) find it a tedious task to converse with ordinary people through sign language. This stands as a hindrance in even the most basic of their activities. It affects their personal development, interpersonal relations and limits the contributions they could otherwise make to society. The prime motive of this project is to provide an easy to use platform for the hard of hearing people to express themselves despite the sign language barrier. We aim to achieve this motive through gesture recognition. Using gesture recognition, we compute the mathematical interpretation of human hand gestures to recognize the signs conveyed by American Sign Language. The system enables real-time hand gesture and speech recognition and provides an innovative and simpler mode of communication for the mute people.


American sign language Background subtraction OpenCV Python speech recognition Convolutional neural network Tensorflow-GPU Real-time 


  1. 1.
    Hussain, I., Talukdar, A.K., Sarma, K.K.: Hand gesture recognition system with real-time palm tracking. In: 11th IEEE India Conference: Emerging Trends and Innovation in Technology, INDICON 2014. Institute of Electrical and Electronics Engineers Inc. (2014)Google Scholar
  2. 2.
    Elmahgiubi, M., Ennajar, M., Drawil, N., Elbuni, M.S.: Sign language translator and gesture recognition. In: 2015 Global Summit on Computer & Information Technology (GSCIT), pp. 1–6 (2015)Google Scholar
  3. 3.
    Loke, P., et al.: Indian sign language converter system using an android app. In: Proceedings of the International Conference on Electronics, Communication and Aerospace Technology, ICECA, January 2017. Institute of Electrical and Electronics Engineers Inc., pp. 436–439 (2017)Google Scholar
  4. 4.
    Gupta, M., Meha, G., Prateek, D.: Sign language to speech converter using neural networks. Int. J. Comput. Sci. Emerg. Technol. 14(3) 2044–6004 (2010)Google Scholar
  5. 5.
    Butte, A., Jadhav, S., Meher, S.: Hand Gestures And Speech Recognition System For Deaf-Dumb (2018)Google Scholar
  6. 6.
    About American Sign Language., August 2016
  7. 7.
    Sign Language Recognition. (2018)
  8. 8.
    Background subtraction. Accessed 06 April 2017
  9. 9.
    OpenCV documentation.
  10. 10.
  11. 11.
    Gonzales, R.C., Woods, R.E.: Digital Image ProcessingGoogle Scholar
  12. 12.
    Raschka, S.: Python Machine Learning (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Simran Kharpude
    • 1
    Email author
  • Vaishnavi Hardikar
    • 1
  • Gautam Munot
    • 1
  • Omkar Lonkar
    • 1
  • Vanita Agarwal
    • 1
  1. 1.College of EngineeringPuneIndia

Personalised recommendations