Communication Support System of Smart Glasses for the Hearing Impaired

  • Daiki Watanabe
  • Yoshinori TakeuchiEmail author
  • Tetsuya Matsumoto
  • Hiroaki Kudo
  • Noboru Ohnishi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10896)


In this research, we propose a novel system for displaying captions of conversation content on smart glasses. We propose natural communication using smart glasses that is more like a conversation between hearing people than conventional communication methods such as sign language or handwriting. The system translates spoken words into text and displays them on the screen of the smart glasses. It equips four microphones, localizes the direction of the sound source, and distinguishes the angular direction of the sound source. Using signals from four microphones, the system can enhance voices. The voice enhancement technique is required to improve the automatic speech recognition rate in noisy environments. Experimental results showed that the system can estimate the angular direction of a voice and recognize more than 90% of words that are spoken. The subject experiment showed that the proposed system had a similarity score close to an existing smartphone application.


Smart glasses Automatic speech recognition Sound localization Voice enhancement Hearing impaired 


  1. 1.
    Suemitsu, K., Zenpo, K., Mizutani, K., Wakatsuki, N.: Captioning of conversational information on single channel microphone array. In: IEICE HCG Symposium, A-4-5, pp. 322–326 (2016). (in Japanese)Google Scholar
  2. 2.
  3. 3.
    KoeTra App Support Page. Accessed 31 Jan 2018

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Daiki Watanabe
    • 1
  • Yoshinori Takeuchi
    • 2
    Email author
  • Tetsuya Matsumoto
    • 3
  • Hiroaki Kudo
    • 3
  • Noboru Ohnishi
    • 3
  1. 1.Graduate School of Information ScienceNagoya UniversityNagoyaJapan
  2. 2.Department of Information Systems, School of InformaticsDaido UniversityNagoyaJapan
  3. 3.Graduate School of InformaticsNagoya UniversityNagoyaJapan

Personalised recommendations