Design and Implementation of Speech Recognition System Based on Gesture Control

  • Zuliang WangEmail author
  • Chuangle Cao
  • Suying Zhang
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1088)


Gesture is a basic human characteristic and an indispensable part of interpersonal communication. Gesture recognition voice playback system is designed for deaf and mute people, which can help them communicate with normal people more conveniently. The system is divided into hardware part and software part. The hardware part includes data acquisition and analysis. The raspberry pie system is adopted. The raspberry pie system mainly includes three hardware modules: camera module, wireless network card module and serial communication module. The software part uses Python language to program the raspberry pie, and PyCharm is used as the programming software. The gestures made by deaf-mute people can be converted into text information through this system and displayed on the display screen. At the same time, the voice can be played, so that the normal people and deaf-mute people can understand each other. The experimental results show that the speech recognition system based on gesture control can achieve the function of gesture acquisition, and can play gesture information through loudspeakers, and display gesture information on the screen, which provides a convenient communication between deaf and mute people and normal people.


Gesture recognition Camera module Voice playback Serial communication 



This study was funded by Key Research and Development Plan Project of Shaanxi Provincial Science & Technology Department (Program No. 2018ZDXM-NY-014).


  1. 1.
    Xiujuan, Chai, and Wang Kongqiao. 2016. Gesture recognition based on partial local texture description. High-tech Communication 20 (5): 487–492.Google Scholar
  2. 2.
    Hua, Liu, Tian Zhansheng, and Feng Yufei. 2018. Intelligent home voice control system based on raspberry pie. Manufacturing Automation 40 (10): 128–131.Google Scholar
  3. 3.
    Gaofeng, Chen Xiong, and Chen Wanqiu. 2016. Video detection and tracking system based on raspberry B + microprocessor. Television Technology 39 (19): 105–108.Google Scholar
  4. 4.
    Youshu, Hu. 2015. Summary of gesture recognition technology. China Science and Technology Information 1 (2): 41–42.Google Scholar
  5. 5.
    Bo, Yuan, and Cha Chendong. 2018. Current situation and prospect of gesture recognition technology. Scientific and Technological Innovation 32 (09): 22–26.Google Scholar
  6. 6.
    Zhensen, Gao, Wang Lei, Meng Fanqiang, and Liu Mingmin. 2018. Gesture recognition system based on FDC2214 capacitance sensor. Electronic Technology 10 (21): 65–71.Google Scholar
  7. 7.
    Yuheng, Luo, Wang Yang, and Liu Wei. 2018. A non-contact gesture recognition device. Science and Technology and Innovation 21 (10): 15–18.Google Scholar
  8. 8.
    Ziyang, Liu, Liu Zhongfu, Zhao Hongyu, Liu Guanchu, Guo Xin, and Wu Yi. 2018. Intelligent old and disabled assistance system based on speech and gesture control. Shanxi Electronic Technology 15 (05): 25–28.Google Scholar
  9. 9.
    Xiaoyan, Zhou. 2018. Research on gesture recognition algorithm for interactive teaching interface, 12–19. Jinan: Jinan University.Google Scholar
  10. 10.
    Wan, Zhou. 2016. Acoustic modeling of speech recognition based on deep neural network, 04–13. Anhui: China University of Science and Technology.Google Scholar
  11. 11.
    Talking about genealogy. 2016. Application of gesture recognition and human-computer interaction based on fingertip information, 11–21. Beijing Jiaotong University.Google Scholar
  12. 12.
    Xuewen, Yang. 2015. Research on real-time search method of user basic gestures oriented to interactive semantics, 23–26. Jinan: Jinan University.Google Scholar
  13. 13.
    Chao, Wang. 2016. Interactive gestures for mobile applications, 10–15. Beijing: Beijing Institute of Fashion.Google Scholar
  14. 14.
    Jiasheng, Yu. Application of gesture recognition technology in software. Electronic technology in software.Google Scholar
  15. 15.
    A Study of Shi Mengchu. 2017. Python Language. China New Communications 22 (07): 24–28.Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.Xijing UniversityXi’an ShaanxiChina

Personalised recommendations