Abstract
Around 5% of people across the globe have difficulty in speaking or are unable to speak. So, to overcome this difficulty, sign language came into the picture. It is a method of non-verbal communication which is usually used by deaf and mute people. Another problem that arises with sign language is that people without hearing or speaking problems do not learn this language. This problem is severe as it creates a barrier between them. To resolve this issue, this paper makes use of computer vision and machine learning along with Convolutional Neural Network. The objective of this paper is to facilitate communication among deaf and mute and other people. For achieving this objective, a system is built to convert hand gestures to voice with gesture understanding and motion capture. This system will be helpful for deaf and mute people as it will increase their communication with other people.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bhat, S., Amruthesh, M., Ashik, Das, C., Sujith: Translating Indian sign language to text and voice messages using flex sensors. Int. J. Adv. Res. Comput. Commun. Eng. (4), 430–434 (2015)
Corballis, M.C.: The gestural origins of language. WIREs Cogn. Sci. 1, 2–7 (2010)
Xu, J., Gannon, P., Emmorey, K., Smith, J., Braun, A.: Symbolic gestures and spoken language are processed by a common neural system. Proc. Natl. Acad. Sci. U.S.A. 106(49), 20664–20669 (2009)
Chakravorty, P.: What is a signal. IEEE Signal Process. Mag. 35(5), 175–177 (2018)
Tang, J., Rangayyan, R., Yao, J., Yang, Y.: Digital image processing and pattern recognition techniques for detection of cancer. Pattern Recogn. 42(6), 1015–1016 (2009)
Klette, R.: Concise Computer Vision, pp. 1–429. Springer, Heidelberg (2014). https://doi.org/10.1007/978-1-4471-6320-6
Morris, T.: Computer Vision and Image Processing, pp. 1–320. Palgrave Macmillan. Springer (2014)
Garg, P., Aggarwal, N., Sofat, S.: Vision based hand gesture recognition. World Acad. Sci. Eng. Technol. Int. J. Comput. Electr. Autom. Control Inf. Eng. 3(1), 186–191 (2009)
Li, Y., Chen, X.: Sign-component-based framework for chinese sign language recognition using accelerometer and sEMG data. IEEE Trans. Biomed. Eng. 59(10), 2695–2704 (2012)
Chen, F., Fu, C., Huang, C.: Hand gesture recognition using a real-time tracking method and hidden Markov models. Image Vis. Comput. 21, 745–758 (2003)
Quek, F.: Towards a vision based hand gesture interface. In: Proceedings of Virtual Reality Software and Technology, pp. 17–31. ACM (1994)
Wikipedia: Markov model (2018). https://en.wikipedia.org/w/index.php?
Cranenburgh, S.V., Alwoshee, A.: An artificial neural network based approach to investigate travellers’ decision rules. Transp. Res. Part C: Emerg. Technol. 98(1), 152–166 (2019)
Jeanguillaume, C., Colliex, C.: Spectrum image: the next step in EELS digital acquisition and processing. Ultramicroscopy 28(1), 252–257 (1989)
Zhang, W., Itoh, K., et al.: Parallel distributed processing model with local space-invariant and its optical architecture. Appl. Opt. 29(32), 4790–4797 (1990)
Jimenez-Fernandez, V.M., et al.: Exploring the use of two-dimensional piecewise-linear functions as an alternative model for representing and processing grayscale-images. J. Appl. Res. Technol. 14(5), 311–318 (2016)
Ding, L., Goshtasby, A.: On the canny edge detector. Pattern Recogn. 34(3), 721–725 (2001)
Coa, Z., Wei, Z., Zhang, G.: A no-reference sharpness metric based on the notion of relative blur for Gaussian blurred images. J. Vis. Commun. Image Represent. 25(7), 1763–1773 (2014)
Ji, S., Xu, W., Yang, M., Yu, K.: 3D convolutional neural networks for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 221–231 (2013)
Johnston, L.: What are webcam frame rates? (2018). https://www.lifewire.com/webcam-framerates
Shinde, S.S., Autee, R.M., Bhosale, V.K.: Real time two way communication approach for hearing impaired and dumb person based on image processing. In: IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), pp. 1–5. IEEE (2016)
Seo, Y., Shin, K.S.: Hierarchical convolutional neural networks for fashion image classification. Expert Syst. Appl. 116, 328–339 (2018)
Shi, D., Zhu, L., Cheng, Z., Li, Z., Zhang, H.: Unsupervised multi-view feature extraction with dynamic graph learning. J. Vis. Commun. Image Represent. 56, 256–264 (2018)
Cai, J., Luo, J., Wang, S., Yang, S.: Feature selection in machine learning: a new perspective. NeuroComputing 300, 70–79 (2018)
Xue, Y., Wang, Q., Yin, X.: A unified approach to sufficient dimension reduction. J. Stat. Plan. Infer. 197, 168–179 (2018)
Di, H., Gao, D.: Gray-level transformation and Canny edge detection for 3D seismic discontinuity enhancement. Comput. Geo Sci. 72, 192–200 (2014)
Gaur, R., Chouhan, V.S.: Classifiers in Image processing. Int. J. Future Revolut. Comput. Sci. Commun. Eng. 3(6), 22–24 (2017)
Rehm, M., Bee, N., André, E.: Wave like an egyptian – accelerometer based gesture recognition for culture specific interactions. Br. Comput. Soc. 1, 13–22 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Mishra, S.K., Sinha, S., Sinha, S., Bilgaiyan, S. (2019). Recognition of Hand Gestures and Conversion of Voice for Betterment of Deaf and Mute People. In: Singh, M., Gupta, P., Tyagi, V., Flusser, J., Ören, T., Kashyap, R. (eds) Advances in Computing and Data Sciences. ICACDS 2019. Communications in Computer and Information Science, vol 1046. Springer, Singapore. https://doi.org/10.1007/978-981-13-9942-8_5
Download citation
DOI: https://doi.org/10.1007/978-981-13-9942-8_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9941-1
Online ISBN: 978-981-13-9942-8
eBook Packages: Computer ScienceComputer Science (R0)