Abstract
Recognition of the sign language is one of the most important milestone in image recognition field. Such systems can help deaf people to communicate with the world. We feel privileged to present a new method which translates from American Sign Language (ASL) fingerspelling into a letter using Convolutional Neural Network and transfer learning. The method is using Google pre-trained model named MobileNet V1 which was trained on the ImageNet image database. Our model was trained on the dataset from Surrey University. We developed a useful model not only for desktop computers but it is also possible to apply it into mobile systems, because of low memory consumption.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Escalera S., Baró X., Gonzà lez J., Bautista M. A., Madadi M., Reyes M., Ponce-López V., Escalante H. J., Shotton J., Guyon I.: ChaLearn looking at people challenge 2014: dataset and results. In: Proceedings of Computer Vision - ECCV 2014 Workshops, pp. 459–473 (2015)
Garcia, B., Viesca, S.A.: Real-time american sign language recognition with convolutional neural networks. In: CS231n: Convolutional Neural Networks for Visual Recognition (2016)
Grobel K., Assan M.: Isolated sign language recognition using hidden Markov models. In: Proceedings of IEEE SMC. Computational Cybernetics and Simulation, vol. 1, pp. 162–167 (1997). https://doi.org/10.1109/ICSMC.1997.625742
Hasanuzzaman, M., Ampornaramveth, V., Zhang, T., Bhuiyan, M. A., Shirai , Y., Ueno, H.: Real-time vision-based gesture recognition for human robot interaction. In: Proceedings of IEEE International Conference on Robotics and Biomimetics, pp. 413–418 (2004). https://doi.org/10.1109/ROBIO.2004.1521814
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Hartwig, A.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017). CoRR http://arxiv.org/abs/1704.04861
Kadous, M.W.: Machine recognition of Auslan signs using PowerGloves: towards large-lexicon recognition of sign language computer science engineering. In: Proceedings of the Workshop on the Integration of Gesture in Language and Speech, pp. 165–174 (1996)
Kraiss, K.F.: Non-intrusive sign language recognition for human computer interaction. In: Proceedings of IFAC/IFIP/IFORS/IEA symposium on analysis, design and evaluation of human machine systems (2004)
Liang, R.H., Ouhyoung, M.: A sign language recognition system using hidden Markov model and context sensitive search. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 59–66 (1996)
Liu, Q., Zhang, N., Yang, W., Wang, S., Cui, Z., Chen, X., Chen, L.: A Review of Image Recognition with Deep Convolutional Neural Network. In: Intelligent Computing Theories and Application, pp. 69–80 (2017)
Mehdi, S.A., Khan, Y.N.: Sign language recognition using sensor gloves. In: Proceedings of ICONIP 2002. vol. 5, pp. 2204–2206 (2002)
Mitchell, R.E., Young, T.A., Bachleda, B., Karchmer, M.A.: How Many People Use ASL in the United States? Sign Lang. Stud. 6(3), 306–335 (2006)
Murakami, K., Taguchi, H.: Gesture recognition using recurrent neural networks. In: Proceedings of the SIGCHI, pp. 237–242 (1991)
Pigou, L., Dieleman, S., Kindermans, P.J., Schrauwen, B.: Sign language recognition using convolutional neural networks. In: Proceedings of Computer Vision - ECCV 2014 Workshops. Springer International Publishing, pp. 572–578 (2015)
Segen, J., Kumar, S.: Shadow gestures: 3D hand pose estimation using a single camera. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 485 (1999). https://doi.org/10.1109/CVPR.1999.786981
Starner, T., Pentland, A.: Real-time american sign language recognition from video using hidden Markov models. In: Proceedings of ISCV, pp. 265–270 (1995). https://doi.org/10.1109/ISCV.1995.477012
Pugeault, N., Bowden, R.: Spelling it out: real–time ASL fingerspelling recognition. In: Proceedings ICCV 2011: 1st IEEE Workshop on Consumer Depth Cameras for Computer Vision, pp. 1114–1119 (2011). https://doi.org/10.1109/ICCVW.2011.6130290
Waldron, M.B., Kim, S.: Isolated ASL sign recognition system for deaf persons. In: IEEE Transactions on Rehabilitation Engineering, vol. 3, no. 3, pp. 261–271 (1995). https://doi.org/10.1109/86.413199
Yang, H.D.: Sign language recognition with the kinect sensor based on conditional random fields. (Sens. Basel, Switz.) 15, 135–147 (2015). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4327011/
Core ML. https://developer.apple.com/documentation/coreml. Accessed 30 05 018
List of sign languages by number of native signers. https://en.wikipedia.org/wiki/List_of_sign_languages_by_number_of_native_signers. Accessed 30 05 2018
Abadi, M., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems (2015). https://www.tensorflow.org/
Acknowledgments
This research was partially supported by Polish Ministry of Science and Higher Education. Calculations have been carried out using resources provided by Wroclaw Centre for Networking and Supercomputing (http://wcss.pl), grant No. 469.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kotarski, S., Maleszka, B. (2019). An Efficient Method for Sign Language Recognition from Image Using Convolutional Neural Network. In: Choroś, K., Kopel, M., Kukla, E., Siemiński, A. (eds) Multimedia and Network Information Systems. MISSI 2018. Advances in Intelligent Systems and Computing, vol 833. Springer, Cham. https://doi.org/10.1007/978-3-319-98678-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-98678-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98677-7
Online ISBN: 978-3-319-98678-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)