Accurate Acoustic Based Gesture Classification with Zero Start-Up Cost
Acoustic gesture recognition based on the Doppler effect has garnered much research attention. The accuracy of gesture recognition and potential false positives are the main factors that limit the widespread use of gestures. To this end, we propose a novel gesture classification method based on the acoustic Doppler effect that does not require any custom hardware, simply a speaker and one microphone on a laptop. An effective sound field is built by a high frequency sound wave from the speaker, and the wave reflected by hand motion is captured by the microphone. We design a set with five features, three of them are stable and invariant to different people, so even new users can operate our system with zero start-up cost and no training. The remaining two features are highly correlated with the velocity and the range to computer of the gestures, which can reduce the potential false positives in detection. Besides, a classifier is designed depending on multistage decision rules to identify the 11 kinds of defined gestures. The experiment result about user experience feedback of HCI shows that our system has good usability performance. And the numerical experiments with 10 users show that our system can not only keep less potential false positives, but also achieve a classification accuracy of up to 99.09%.
KeywordsDoppler effect Gesture classification Acoustic HCI
We thank the participants for participating the user study. This work is partially supported by The National Key Research and Development Program of China (2016YFB0502201).
- 1.Ai, H., Men, Y., Han, L., Li, Z., Liu, M.: High precision gesture sensing via quantitative characterization of the doppler effect. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 973–978. IEEE (2016)Google Scholar
- 3.Aumi, M.T.I., Gupta, S., Goel, M., Larson, E., Patel, S.: Doplink: using the doppler effect for multi-device interaction. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 583–586. ACM (2013)Google Scholar
- 5.Cabral, M.C., Morimoto, C.H., Zuffo, M.K.: On the usability of gesture interfaces in virtual reality environments. In: Proceedings of the 2005 Latin American Conference on Human-Computer Interaction, pp. 100–108. ACM (2005)Google Scholar
- 6.Chen, K.Y., Ashbrook, D., Goel, M., Lee, S.H., Patel, S.: Airlink: sharing files between multiple devices using in-air gestures. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 565–569. ACM (2014)Google Scholar
- 7.Fu, B., Karolus, J., Grosse-Puppendahl, T., Hermann, J., Kuijper, A.: Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones. In: Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction, p. 8. ACM (2015)Google Scholar
- 8.Gupta, S., Morris, D., Patel, S., Tan, D.: Soundwave: using the doppler effect to sense gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1911–1914. ACM (2012)Google Scholar
- 10.Kellogg, B., Talla, V., Gollakota, S.: Bringing gesture recognition to all devices. NSDI 14, 303–316 (2014)Google Scholar
- 11.Molchanov, P., Gupta, S., Kim, K., Kautz, J.: Hand gesture recognition with 3D convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–7 (2015)Google Scholar
- 12.Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 409–420. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24598-8_38CrossRefGoogle Scholar
- 14.Pittman, C., Wisniewski, P., Brooks, C., LaViola Jr, J.J.: Multiwave: doppler effect based gesture recognition in multiple dimensions. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1729–1736. ACM (2016)Google Scholar
- 15.Pittman, C.R., LaViola Jr, J.J.: Multiwave: complex hand gesture recognition using the doppler effect. In: Proceedings of the 43rd Graphics Interface Conference, pp. 97–106. Canadian Human-Computer Communications Society (2017)Google Scholar
- 16.Qifan, Y., Hao, T., Xuebing, Z., Yin, L., Sanfeng, Z.: Dolphin: ultrasonic-based gesture recognition on smartphone platform. In: 2014 IEEE 17th International Conference on Computational Science and Engineering (CSE), pp. 1461–1468. IEEE (2014)Google Scholar