Abstract
Emerging depth sensors and new interaction paradigms enable to create more immersive and intuitive Natural User Interfaces by recognizing body gestures. One of the vision-based devices that has received plenty of attention is the Leap Motion Controller (LMC). This device models the 3D position of hands and fingers and provides more than 50 features such as palm center and fingertips. In spite of the fact that the LMC provides such useful information of the hands, developers still have to deal with the hand gesture recognition problem. For this reason, several researchers approached this problem using well-known machine learning techniques used for gesture recognition such as SVM for static gestures and DTW for dynamic gestures. At this point, we propose an approach that applies a resampling technique based on fast Fourier Transform algorithm to feed a CNN+LSTM neural network in order to identify both static and dynamic gestures. As far as our knowledge, there is no full dataset based on the LMC that includes both types of gestures. Therefore, we also introduce the Hybrid Hand Gesture Recognition database, which consists of a large set of gestures generated with the LMC, including both type of gestures with different temporal sizes. Experimental results showed the robustness of our approach in terms of recognizing both type of gestures. Moreover, our approach outperforms other well-known algorithms of the gesture recognition field.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, Y., Ding, Z., Chen, Y.L., Wu, X.: Rapid recognition of dynamic hand gestures using leap motion. In: 2015 IEEE International Conference on Information and Automation, pp. 1419–1424. IEEE (2015)
Cheng, H., Yang, L., Liu, Z.: Survey on 3D hand gesture recognition. IEEE Trans. Circuits Syst. Video Technol. 26(9), 1659–1673 (2015)
Cheok, M.J., Omar, Z., Jaward, M.H.: A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybernet. 10(1), 131–153 (2019)
Chuan, C.H., Regina, E., Guardino, C.: American sign language recognition using leap motion sensor. In: 2014 13th International Conference on Machine Learning and Applications (ICMLA), pp. 541–544. IEEE (2014)
Fok, K.Y., Ganganath, N., Cheng, C.T., Chi, K.T.: A real-time ASL recognition system using leap motion sensors. In: 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), pp. 411–414. IEEE (2015)
Khelil, B., Amiri, H.: Hand gesture recognition using leap motion controller for recognition of Arabic sign language. In: 3rd International conference ACECS 2016 (2016)
López, G., Quesada, L., Guerrero, L.A.: A gesture-based interaction approach for manipulating augmented objects using leap motion. In: Cleland, I., Guerrero, L., Bravo, J. (eds.) IWAAL 2015. LNCS, vol. 9455, pp. 231–243. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26410-3_22
Lu, W., Tong, Z., Chu, J.: Dynamic hand gesture recognition with leap motion controller. IEEE Signal Process. Lett. 23(9), 1188–1192 (2016)
Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1565–1569. IEEE (2014)
Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with jointly calibrated leap motion and depth sensor. Multimed. Tools Appl. 75(22), 14991–15015 (2016)
McCartney, R., Yuan, J., Bischof, H.P.: Gesture recognition with the leap motion controller. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV). p. 3. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp) (2015)
Mohandes, M., Aliyu, S., Deriche, M.: Arabic sign language recognition using the leap motion controller. In: 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), pp. 960–965. IEEE (2014)
Sainath, T.N., Vinyals, O., Senior, A., Sak, H.: Convolutional, long short-term memory, fully connected deep neural networks. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (2015)
Sarkar, A.R., Sanyal, G., Majumder, S.: Hand gesture recognition systems: a survey. Int. J. Comput. Appl. 71(15), 25–37 (2013)
Wang, Q., Xu, Y., Chen, Y.L., Wang, Y., Wu, X.: Dynamic hand gesture early recognition based on hidden semi-markov models. In: 2014 IEEE International Conference on Robotics and Biomimetics. IEEE (2014)
Acknowledgements
This work has been supported by CONICET PIP No. 112-201501-00030CO (2015–2017). In addition, we gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan GPU used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Alonso, D.G., Teyseyre, A., Berdun, L., Schiaffino, S. (2019). A Deep Learning Approach for Hybrid Hand Gesture Recognition. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds) Advances in Soft Computing. MICAI 2019. Lecture Notes in Computer Science(), vol 11835. Springer, Cham. https://doi.org/10.1007/978-3-030-33749-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-33749-0_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33748-3
Online ISBN: 978-3-030-33749-0
eBook Packages: Computer ScienceComputer Science (R0)