Accurate Acoustic Based Gesture Classification with Zero Start-Up Cost

  • Haojun Ai
  • Liangliang Han
  • Yifeng WangEmail author
  • Liang Liao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11336)


Acoustic gesture recognition based on the Doppler effect has garnered much research attention. The accuracy of gesture recognition and potential false positives are the main factors that limit the widespread use of gestures. To this end, we propose a novel gesture classification method based on the acoustic Doppler effect that does not require any custom hardware, simply a speaker and one microphone on a laptop. An effective sound field is built by a high frequency sound wave from the speaker, and the wave reflected by hand motion is captured by the microphone. We design a set with five features, three of them are stable and invariant to different people, so even new users can operate our system with zero start-up cost and no training. The remaining two features are highly correlated with the velocity and the range to computer of the gestures, which can reduce the potential false positives in detection. Besides, a classifier is designed depending on multistage decision rules to identify the 11 kinds of defined gestures. The experiment result about user experience feedback of HCI shows that our system has good usability performance. And the numerical experiments with 10 users show that our system can not only keep less potential false positives, but also achieve a classification accuracy of up to 99.09%.


Doppler effect Gesture classification Acoustic HCI 



We thank the participants for participating the user study. This work is partially supported by The National Key Research and Development Program of China (2016YFB0502201).


  1. 1.
    Ai, H., Men, Y., Han, L., Li, Z., Liu, M.: High precision gesture sensing via quantitative characterization of the doppler effect. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 973–978. IEEE (2016)Google Scholar
  2. 2.
    Asadzadeh, P., Kulik, L., Tanin, E.: Gesture recognition using RFID technology. Pers. Ubiquit. Comput. 16(3), 225–234 (2012)CrossRefGoogle Scholar
  3. 3.
    Aumi, M.T.I., Gupta, S., Goel, M., Larson, E., Patel, S.: Doplink: using the doppler effect for multi-device interaction. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 583–586. ACM (2013)Google Scholar
  4. 4.
    Bevan, N., Curson, I.: Methods for measuring usability. In: Howard, S., Hammond, J., Lindgaard, G. (eds.) Human-Computer Interaction INTERACT 1997. ITIFIP, pp. 672–673. Springer, Boston, MA (1997). Scholar
  5. 5.
    Cabral, M.C., Morimoto, C.H., Zuffo, M.K.: On the usability of gesture interfaces in virtual reality environments. In: Proceedings of the 2005 Latin American Conference on Human-Computer Interaction, pp. 100–108. ACM (2005)Google Scholar
  6. 6.
    Chen, K.Y., Ashbrook, D., Goel, M., Lee, S.H., Patel, S.: Airlink: sharing files between multiple devices using in-air gestures. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 565–569. ACM (2014)Google Scholar
  7. 7.
    Fu, B., Karolus, J., Grosse-Puppendahl, T., Hermann, J., Kuijper, A.: Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones. In: Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction, p. 8. ACM (2015)Google Scholar
  8. 8.
    Gupta, S., Morris, D., Patel, S., Tan, D.: Soundwave: using the doppler effect to sense gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1911–1914. ACM (2012)Google Scholar
  9. 9.
    Jeong, J., Jang, Y.: Max-min hand cropping method for robust hand region extraction in the image-based hand gesture recognition. Soft Comput. 19(4), 815–818 (2015)CrossRefGoogle Scholar
  10. 10.
    Kellogg, B., Talla, V., Gollakota, S.: Bringing gesture recognition to all devices. NSDI 14, 303–316 (2014)Google Scholar
  11. 11.
    Molchanov, P., Gupta, S., Kim, K., Kautz, J.: Hand gesture recognition with 3D convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–7 (2015)Google Scholar
  12. 12.
    Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 409–420. Springer, Heidelberg (2004). Scholar
  13. 13.
    Paramonov, P., Sutula, N.: Simplified scoring methods for HMM-based speech recognition. Soft Comput. 20(9), 3455–3460 (2016)CrossRefGoogle Scholar
  14. 14.
    Pittman, C., Wisniewski, P., Brooks, C., LaViola Jr, J.J.: Multiwave: doppler effect based gesture recognition in multiple dimensions. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1729–1736. ACM (2016)Google Scholar
  15. 15.
    Pittman, C.R., LaViola Jr, J.J.: Multiwave: complex hand gesture recognition using the doppler effect. In: Proceedings of the 43rd Graphics Interface Conference, pp. 97–106. Canadian Human-Computer Communications Society (2017)Google Scholar
  16. 16.
    Qifan, Y., Hao, T., Xuebing, Z., Yin, L., Sanfeng, Z.: Dolphin: ultrasonic-based gesture recognition on smartphone platform. In: 2014 IEEE 17th International Conference on Computational Science and Engineering (CSE), pp. 1461–1468. IEEE (2014)Google Scholar
  17. 17.
    Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)CrossRefGoogle Scholar
  18. 18.
    Seddon, N., Bearpark, T.: Observation of the inverse doppler effect. Science 302(5650), 1537–1540 (2003)CrossRefGoogle Scholar
  19. 19.
    Suk, H.I., Sin, B.K., Lee, S.W.: Hand gesture recognition based on dynamic bayesian network framework. Pattern Recogn. 43(9), 3059–3072 (2010)CrossRefGoogle Scholar
  20. 20.
    Xiao, Q., Siqi, L.: Motion retrieval based on dynamic Bayesian network and canonical time warping. Soft Comput. 21(1), 267–280 (2017)CrossRefGoogle Scholar
  21. 21.
    Xiao, Q., Song, R.: Motion retrieval based on motion semantic dictionary and HMM inference. Soft Comput. 21(1), 255–265 (2017)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Haojun Ai
    • 1
    • 2
    • 3
  • Liangliang Han
    • 4
  • Yifeng Wang
    • 1
    Email author
  • Liang Liao
    • 5
    • 6
  1. 1.School of Cyber Science and EngineeringWuhan UniversityWuhanChina
  2. 2.Key Laboratory of Aerospace Information Security and Trusted ComputingMinistry of EducationBeijingChina
  3. 3.Collaborative Innovation Center of Geospatial TechnologyWuhanChina
  4. 4.Aerospace System Engineering ShanghaiShanghaiPeople’s Republic of China
  5. 5.ChangZhou Municipal Public Security BureauChangzhouChina
  6. 6.Key Laboratory of Police Geographic Information TechnologyMinistry of Public SecurityBeijingChina

Personalised recommendations