Abstract
The American Sign Language (ASL) was developed in the early 19th century in the American School for Deaf, United States of America. It is a natural language inspired by the French sign language and is used by around half a million people around the world with a majority in North America. The Deaf Culture views deafness as a difference in human experience rather than a disability, and ASL plays an important role in this experience. In this project, we have used Convolutional Neural Networks to create a robust model that understands 29 ASL characters (26 alphabets and 3 special characters). We further host our model locally over a real-time video interface which provides the predictions in real-time and displays the corresponding English characters on the screen like subtitles. We look at the application as a one-way translator from ASL to English for the alphabet. We conceptualize this whole procedure in our paper and explore some useful applications that can be implemented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adithya, V., Vinod, P.R., Gopalakrishnan, U.: Artificial neural network based method for Indian sign language recognition. In: 2013 IEEE Conference on Information & Communication Technologies, Thuckalay, Tamil Nadu, India, pp. 1080–1085 (2013)
Ragab, A., Ahmed, M., Chau, S.-C.: Sign language recognition using hilbert curve features. In: Kamel, M., Campilho, A. (eds.) ICIAR 2013. LNCS, vol. 7950, pp. 143–151. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39094-4_17
ASL Alphabet. Image dataset for alphabets in the American sign language. https://www.kaggle.com/grassknoted/asl-alphabet
Emil, M., Chen, P.Q., Georganas, N.D.: Real-time vision-based hand gesture recognition using haar-like features (2007)
Cooper, H., Ong, E.-J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13(1), 2205–2231 (2012)
Bradski, G.: The OpenCV library. DR DOBBS J. Softw. Tools 120, 122 (2000)
Dardas, N.H., Georganas, N.D.: Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE Trans. Instrum. Measurment (2011)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)
Nikam, A.S., Ambekar, A.G.: Sign language recognition using image based hand gesture recognition techniques. In: 2016 Online International Conference on Green Engineering and Technologies (IC-GET), Coimbatore, pp. 1–5 (2016)
Ren, Z., Meng, J., Yuan, J., Zhang, Z.: Robust hand gesture recognition with the kinect sensor. In: Proceedings of the 19th ACM International Conference on Multimedia (MM 2011), pp. 759–760. ACM, New York (2011)
Sawant, S.N., Kumbhar, M.S.: Real time sign language recognition using PCA. In: 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies, Ramanathapuram, pp. 1412–1415 (2014)
Raheja, J.L., Mishra, A., Chaudhary, A.: Pattern Recognit. Image Anal. 26, 434 (2016). https://doi.org/10.1134/S1054661816020164
Starner, T.E.: Visual recognition of American sign language using hidden Markov models. Master’s thesis, Massachusetts Institute of Technology, Cambridge (1995)
Madhuri, Y., Anitha, G., Anburajan, M.: Vision-based sign language translation device. In: 2013 International Conference on Information Communication and Embedded Systems, ICICES 2013, pp. 565–568 (2013). https://doi.org/10.1109/ICICES.2013.6508395
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). http://www.tensorflow.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sinha, S., Singh, S., Rawat, S., Chopra, A. (2019). Real Time Prediction of American Sign Language Using Convolutional Neural Networks. In: Singh, M., Gupta, P., Tyagi, V., Flusser, J., Ören, T., Kashyap, R. (eds) Advances in Computing and Data Sciences. ICACDS 2019. Communications in Computer and Information Science, vol 1045. Springer, Singapore. https://doi.org/10.1007/978-981-13-9939-8_3
Download citation
DOI: https://doi.org/10.1007/978-981-13-9939-8_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9938-1
Online ISBN: 978-981-13-9939-8
eBook Packages: Computer ScienceComputer Science (R0)