Abstract
It is often observed that people who are speech or hearing impaired find it difficult to communicate with others. According to WHO [5], over 5% of the global population has disabling hearing loss, while many more are speech impaired. These people mainly rely on sign languages for their daily communication. Sign language is quite complicated for an average person to understand, which makes the world less accessible to a person who has acquired this disability. Hence, to solve this existing problem and make the world more accessible for such people, we propose GestTalk, a smart glove specifically designed to enable speech-impaired people to communicate with others by translating their performed gesture to speech with the help of machine learning and IoT. We are using hardware-based glove loaded with home-made flex sensor to capture data points and machine learning algorithm to map these gestures to speech dynamically in real time. Usage of home-made flex sensor, each costing under 10 INR, plays the vital role in capturing finger movement during the gesture, and certainly makes GestTalk a cost-effective gadget. We were able to achieve ~95% of prediction accuracy on the trained sample with our minimal prototype. We present this case study to show how GestTalk can aid the hearing/speech impaired by enabling them to communicate with the world via speech cheaply and economically.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
DTW Wikipidea. https://en.wikipedia.org/wiki/Dynamictimewarping.
Hand gesture interpretation using sensing glove integrated with machinelearning algorithms. World Academy of Science, Engineering and Technology International Journal of Mechanical and Mechatronics Engineering, 10(11), 1860–1860 (2016). https://waset.org/publications/10005809/hand-gesture-interpretation-using-sensingglove-integrated-with-machine-learning-algorithms.
OurGitRepo,https://github.com/sabyasachi087/aml-gesture-recognition/blob/master/gesturerecognitionprojectreport.ipynb.
Pypi Library, https://github.com/pierre-rouanet/dtw.
WHO Report on Hearing Disability. (2016). http://www.who.int/mediacentre/factsheets/fs300/en/.
Acknowledgements
This prototype was accomplished as a part of a global hackathon conducted in Persistent Systems. The authors would like to acknowledge the efforts of all the team members of BiT’s Please (Samaikya Akarapu, Abhijeet Pal, Nikita Shah, Anirudh Ghosh & Nisha Kumari). Special thanks to Sagar Inamdar for helping us immensely in building the glove.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Shanbhag, V., Prabhune, A., Choudhury, S.R., Jain, H. (2020). GestTalk—Real-Time Gesture to Speech Conversion Glove. In: Sharma, N., Chakrabarti, A., Balas, V. (eds) Data Management, Analytics and Innovation. Advances in Intelligent Systems and Computing, vol 1016. Springer, Singapore. https://doi.org/10.1007/978-981-13-9364-8_28
Download citation
DOI: https://doi.org/10.1007/978-981-13-9364-8_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9363-1
Online ISBN: 978-981-13-9364-8
eBook Packages: EngineeringEngineering (R0)