Advertisement

Improvements in a Wearable Device for Sign Language Translation

  • Francesco PezzuoliEmail author
  • Dario Corona
  • Maria Letizia Corradini
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 973)

Abstract

Nowadays a commercial product for sign language translation is still not available. This paper presents our latest results towards this goal, presenting a functional prototype called Talking Hands. Talking Hands uses a data-glove to detect the hand movements of the user, and a smartphone application to gather all the data and translates them into voice, using a speech synthesizer. Talking Hands implements the most suitable solutions for a massive production without penalizing its reliability. This paper presents the improvements of the last prototype in terms of hardware, software and design, together with a preliminary analysis for the translation of dynamic gestures through this device.

Keywords

Sign Language Recognition Deaf Data-glove Gesture recognition 

Notes

Acknowledgments

This work is supported by Limix S.r.l. (www.limix.it). Limix is an Italian start-up and spin-off of the University of Camerino. The intellectual property of Talking Hands and its different parts (hardware, software, design) is of Limix S.r.l.

References

  1. 1.
    Perkins, R., Battle, T., Edgerton, J., Mcneill, J.: A survey of barriers to employment for individuals who are deaf. J. Am. Deaf. Rehabil. Assoc. 49(1), 66–85 (2015)Google Scholar
  2. 2.
    Kim, H., Lee, S., Lee, D., Choi, S., Ju, J., Myung, H.: Real-time human pose estimation and gesture recognition from depth images using superpixels and SVM classifier. Sensors (Switzerland) 15(6), 2410–12427 (2015)Google Scholar
  3. 3.
    Hirafuji Neiva, D., Zanchettin, C.: Gesture recognition: a review focusing on sign language in a mobile context. Expert Syst. Appl. 103, 159–183 (2018)CrossRefGoogle Scholar
  4. 4.
    Cooper, H., Pugeault, N., Bowden, R.: Reading the signs: a video based sign dictionary. In: IEEE International Conference Computer Vision workshops, ICCV 2011, Barcelona (2011)Google Scholar
  5. 5.
    Starner, T., Weaver, J., Pentland, A.: Real time american sign language recognition using desk and wearable computer based video. IEEE Trans. Pattern Anal. Mach. Intell. 20(12) (1998)CrossRefGoogle Scholar
  6. 6.
    Kelly, D., McDonald, J., Markham, C.: A person independent system for recognition of hand postures used in sign language. Pattern Recognit. Lett. 31, 1359–1368 (2010)CrossRefGoogle Scholar
  7. 7.
    Yoon, H.S., Soh, J., Bae, Y.J., Seung Yang, H.: Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognit. 37(4), 1491–1501 (2001)CrossRefGoogle Scholar
  8. 8.
    Ahmed, M.A., Zaidan, B.B., Zaidan, A.A., Salih, M.M., Bin Lakulu, M.M.: A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors (Switzerland) 18(7) (2018)CrossRefGoogle Scholar
  9. 9.
    Bajpai, D., Porov, U., Srivastav, G., Sachan, N.: Two way wireless data communication and American sign language translator glove for images text and speech display on mobile phone. In: Proceedings 2015 5th International Conference on Communication Systems and Network Technologies. CSNT 2015, pp. 578–585 (2015)Google Scholar
  10. 10.
    Bukhari, J., Rehman, M., Malik, S.I., Kamboh, A.M., Salman, A.: American sign language translation through sensory glove: SignSpeak. Int. J. u- e-Serv. Sci. Technol. 8, 131–142 (2015)CrossRefGoogle Scholar
  11. 11.
    Shukor, A.Z., Miskon, M.F., Jamaluddin, M.H., Bin Ali Ibrahim, F., Asyraf, M.F., Bin Bahar, M.B.: A new data glove approach for malaysian sign language detection. In: 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), vol. 76, pp. 60–67 (2015)CrossRefGoogle Scholar
  12. 12.
    Seymour, M., Tsoeu, M.: A mobile application for South African Sign Language (SASL) recognition, pp. 1–5 (2015)Google Scholar
  13. 13.
    Kau, L.J., Su, W.L., Yu, P.J., Wei S.J.: A real-time portable sign language translation system. In: 2015 IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 1–4 (2015)Google Scholar
  14. 14.
    Devi, S., Deb, S.: Low cost tangible glove for translating sign gestures to speech and text in Hindi language. In: 3rd International Conference on Computational Intelligence & Communication Technology (CICT), pp. 1–5 (2017)Google Scholar
  15. 15.
    Pezzuoli, F., Corona, D., Corradini, M.L., Cristofaro, A.: Development of a wearable device for sign language translation. In: International Workshop on Human-Friendly Robotics (HFR2017), pp. 115–126 (2017)Google Scholar
  16. 16.
    Akhmadeev, K., Rampone, E., Yu, T., Aoustin, Y., Le Carpentier E.: A testing system for a real-time gesture classification using surface EMG. In: 20th IFAC World Congress, pp. 11498–11503 (2017)CrossRefGoogle Scholar
  17. 17.
    Kouichi, M., Hitomi, T.: Gesture recognition using recurrent neural networks. In: ACM Conference on Human factors in computing systems: reaching through technology (1999)Google Scholar
  18. 18.
    Vogler, C.: American sign language recognition: reducing the complexity of the task with phoneme-based modeling and parallel hidden markov models. University of Pennsylvania (2003)Google Scholar
  19. 19.
    Li, X.: Gesture recognition based on fuzzy C-Means clustering algorithm. Department of Computer Science, The University of Tennessee, KnoxvilleGoogle Scholar
  20. 20.
    Nagi, J., et al.: Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: 2011 International Conference on Signal and Image Processing and Applications (ICSIPA), pp. 342–347 (2011)Google Scholar
  21. 21.
    Huynh, D.Q.: Metrics for 3D rotations: comparison and analysis. J. Math. Imaging Vis. 35, 155–164 (2009)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ong, S.C.W., Ranganath, S.: Automatic sign language analysis: a survey and the future beyond lexical meaning. IEEE Trans. Pattern Anal. Mach. Intell. 27(6) (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Francesco Pezzuoli
    • 1
    Email author
  • Dario Corona
    • 1
  • Maria Letizia Corradini
    • 1
  1. 1.School of Science and Technology, Mathematics DivisionUniversity of CamerinoCamerinoItaly

Personalised recommendations