A Survey on Hand Gesture Recognition Using Machine Learning and Infrared Information

  • Rubén NogalesEmail author
  • Marco E. Benalcázar
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1194)


The research consists in the hand gestures are movements that convey information and thus complement oral communication or by themselves constitute a form of communication between people. The function of a hand gesture recognition system is to identify the type of movement, from a given set of movements, and the instant when that movement is performed. Gesture recognition systems have multiple applications including sign language translation, bionics, human-machine interaction, gamming, and virtual reality. For this reason, hand gesture recognition is a problem where many researchers have focused their attention too. In this context, in this paper, we present a systematic literature review for hand gesture recognition using ma-chine learning and infrared information. This work has been made because there is no work in the scientific literature that reviews gesture recognition systems based on machine learning and infrared information. In this work, we answer the research question: what is the architecture of the proposed models for hand gesture recognition based on machine learning and infrared information? For answering this research question, we used Kitchenham methodology. Finally, in this work, we also present trends and gaps with respect to the problem analyzed.


Hand gesture recognition Machine learning Infrared information Survey Systematic literature review 



The authors gratefully acknowledge the financial support provided by Escuela Politécnica Nacional for the development of the research project PIJ-16-13, and the Universidad Técnica de Ambato by development research project PFISEI24 “Integración de Machine Learning y Visión por Computadora para la Manipulación de Objetos Aplicados al Youbot Kuka”. Finally, to Doctoral program of the Departamento de Informática y Ciencias de la Computación of the Escuela Politécnica Nacional.


  1. 1.
    Rohith, H.R., Gowtham, S., Chandra, A.S.: Hand gesture recognition in real time using IR sensor. Int. J. Pure Appl. Math. 114(7), 111–121 (2017)Google Scholar
  2. 2.
    Al-Shamayleh, A.S., Ahmad, R., Abushariah, M.A.: A systematic literature review on vision based gesture recognition techniques. Multimed. Tools Appl. 77, 28121–28184 (2018)CrossRefGoogle Scholar
  3. 3.
    Groenewald, C., Anslow, C., Islam, J., Rooney, C., Passmore, P.J., Wong, B.L.: Understanding 3D mid-air hand gestures with interactive surfaces and displays: a systematic literature review, pp. 1–13 (2016)Google Scholar
  4. 4.
    Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141, 152–165 (2015)CrossRefGoogle Scholar
  5. 5.
    Suarez, J., Murphy, R.R.: Hand gesture recognition with depth images: a review (2012)Google Scholar
  6. 6.
    Sathyanarayana, S.: Vision-based patient monitoring: a comprehensive review of algorithms and technologies. J. Ambient Intell. Humaniz. Comput. 9, 225–251 (2015)CrossRefGoogle Scholar
  7. 7.
    Madeo, R.C., Lima, C.A., Peres, S.M.: Studies in automated hand gesture analysis: an overview of functional types and gesture phases (2016)Google Scholar
  8. 8.
    Chouhan, S.S., Kaul, A.: Soft computing approaches for image segmentation a survey. Multimed. Tools Appl. 77, 28483–28537 (2018)CrossRefGoogle Scholar
  9. 9.
    Dominio, F., Donadeo, M., Marin, G., Zanuttigh, P., Cortelazzo, G.M.: Hand gesture recognition with depth data. In: Proceedings of the 4th ACM/IEEE, pp. 9–16 (2013)Google Scholar
  10. 10.
    Xu, Y., Wang, Q., Bai, X., Chen, Y.L., Wu, X.: A novel feature extracting method for dynamic gesture recognition based on support vector machine. In: Proceedings of the IEEE International Conference on Information, pp. 437–441 (2014)Google Scholar
  11. 11.
    Jais, H.M., Mahayuddin, Z.R., Arshad, H.: A review on gesture recognition using Kinect. In: The 5th International Conference on Electrical Engineering and Informatics 2015, pp. 594–599 (2015)Google Scholar
  12. 12.
    Plouffe, G., Cretu, A.-M.: Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans. Instrum. Meas. 65(2), 305–316 (2016)CrossRefGoogle Scholar
  13. 13.
    Czuszynski, K., Ruminski, J., Wtorek, J.: Pose classification in the gesture recognition using the linear optical sensor. In: Proceedings - 2017 10th International Conference on Human System Interactions, HSI 2017, pp. 18–24 (2017)Google Scholar
  14. 14.
    Park, S., Ryu, M., Chang, J.Y., Park, J.: A hand posture recognition system utilizing frequency difference of infrared light. In: Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, pp. 65–68 (2014)Google Scholar
  15. 15.
    Jangyodsuk, P., Conly, C., Athitsos, V.: Sign language recognition using dynamic time warping and hand shape distance based on histogram of oriented gradient features. In: Proceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments - PETRA 2014, pp. 1–6 (2014)Google Scholar
  16. 16.
    Doan, H. G., Vu, H., Tran, T.H.: Recognition of hand gestures from cyclic hand movements using spatial-temporal features. In: ACM International Conference Proceeding Series, pp. 260–267 (2015)Google Scholar
  17. 17.
    Wei, L., Tong, Z., Chu, J.: Dynamic hand gesture recognition with leap motion controller. IEEE Signal Process. Lett. 23(9), 1188–1192 (2016)CrossRefGoogle Scholar
  18. 18.
    Naguri, C.R., Bunescu, R.C.: Recognition of dynamic hand gestures from 3D motion data using LSTM and CNN architectures. In: Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017, pp. 1130 1133 (2018)Google Scholar
  19. 19.
    Sagayam, K.M., Hemanth, D.J.: A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition. Comput. Intell. 35, 59–81 (2019)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Du, Y., Liu, S., Feng, L., Chen, M., Wu, J.: Hand gesture recognition with leap motion and kinect devices, pp. 1–6 (2014)Google Scholar
  21. 21.
    Benmoussa, M., Mahmoudi, A.: Machine learning for hand gesture recognition using bag-of-words. In: 2018 International Conference on Intelligent Systems and Computer Vision, ISCV 2018, pp. 1–7 (2018)Google Scholar
  22. 22.
    Vamsikrishna, K.M., Dogra, D.P., Desarkar, M.S.: Computer vision assisted palm rehabilitation with supervised learning. IEEE Trans. Biomed. Eng. 9294(i), 1–10 (2015)Google Scholar
  23. 23.
    Almasre, M.A., Al-Nuaim, H.: Recognizing arabic sign language gestures using depth sensors and a KSVM classifier, pp. 146–151 (2016)Google Scholar
  24. 24.
    Avola, D., Bernardi, M., Cinque, L., Foresti, G.L., Massaroni, C.: Exploiting recurrent neural networks and leap motion controller for the recognition of sign language and semaphoric hand gestures. IEEE Trans. Multimed. PP(8), 1 (2018)Google Scholar
  25. 25.
    Liu, F., Du, B., Wang, Q., Wang, Y., Zeng, W.: Hand gesture recognition using kinect via deterministic learning, pp. 2127–2132 (2017)Google Scholar
  26. 26.
    Khalifa, A.B.: A comprehensive leap motion database for hand gesture recognition, no. July 2013, pp. 514–519 (2016)Google Scholar
  27. 27.
    Rossol, N., Cheng, I., Basu, A.: A multisensor technique for gesture recognition through intelligent skeletal pose analysis. IEEE Trans. Hum.-Mach. Syst. 46, 1–10 (2015)Google Scholar
  28. 28.
    Plouffe, G., Cretu, A.M.: Static and dynamic hand gesture recognition in depth data using dynamic time warping, pp. 1–12 (2015)Google Scholar
  29. 29.
    Bai, X., Li, C.: Dynamic hand gesture recognition based on depth information. In: 2018 International Conference on Control, Automation and Information Sciences (ICCAIS), (Iccais), pp. 216–221 (2018)Google Scholar
  30. 30.
    Lai, K., Yanushkevich, S.N.: CNN + RNN depth and skeleton based dynamic hand gesture recognition. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3451–3456 (2018)Google Scholar
  31. 31.
    Li, C.: Hand gesture recognition based on wavelet invariant moments, no. 61403302, pp. 459–464 (2017)Google Scholar
  32. 32.
    Clark, A., Moodley, D.: A system for a hand gesture-manipulated virtual reality environment. In: Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists on - SAICSIT 2016, pp. 1–10 (2016)Google Scholar
  33. 33.
    Jiang, F., Zhang, S., Wu, S., Gao, Y., Zhao, D.: Multi-layered gesture recognition with kinect. In: Escalera, S., Guyon, I., Athitsos, V. (eds.) Gesture Recognition. TSSCML, pp. 387–416. Springer, Cham (2017). Scholar
  34. 34.
    Hsiao, D.Y., Sun, M., Ballweber, C., Cooper, S.: Proactive sensing for improving hand pose estimation, pp. 2348–2352 (2016)Google Scholar
  35. 35.
    Ye, Y., Nurmi, P.: Gestimator - shape and stroke similarity based gesture recognition categories and subject descriptors, pp. 219–226 (2015)Google Scholar
  36. 36.
    Tang, A., Lu, K., Wang, Y., Huang, J., Li, H.: A real-time hand posture recognition system using. ACM Trans. Intell. Syst. Technol. (TIST) 6(2), 1–23 (2015)CrossRefGoogle Scholar
  37. 37.
    Wang, C., Liu, Z., Zhu, M., Zhao, J., Chan, S.: A hand gesture recognition system based on canonical superpixel-graph. Signal Process.: Image Commun. 58, 87–98 (2017)Google Scholar
  38. 38.
    Núñez, J.C., Cabido, R., Pantrigo, J.J., Montemayor, A.S., Vélez, J.F.: Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recognit. 76, 80–94 (2018)CrossRefGoogle Scholar
  39. 39.
    Cho, Y., Lee, A., Park, J., Ko, B., Kim, N.: Enhancement of gesture recognition for contactless interface using a personalized classifier in the operating room. Comput. Methods Programs Biomed. 161, 39–44 (2018)CrossRefGoogle Scholar
  40. 40.
    Kumar, P., Gauba, H., Roy, P.P., Dogra, D.P.: A multimodal framework for sensor based sign language recognition. Neurocomputing 259, 21–38 (2016)CrossRefGoogle Scholar
  41. 41.
    Hong, C., Zeng, Z., Xie, R., Zhuang, W., Wang, X.: Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 142, 223–230 (2018)CrossRefGoogle Scholar
  42. 42.
    Cheng, H., Dai, Z., Liu, Z., Zhao, Y.: An image-to-class dynamic time warping approach for both 3D static and trajectory hand gesture recognition. Pattern Recognit. 55, 137–147 (2016)CrossRefGoogle Scholar
  43. 43.
    Liang, W., Guixi, L.: Dynamic and combined gestures recognition based on multifeature fusion in a complex environment. J. China Univ. Posts Telecommun. 22(2), 81–88 (2015)CrossRefGoogle Scholar
  44. 44.
    Inoue, K., Shiraishi, T., Yoshioka, M., Yanagimoto, H.: Depth sensor based automatic hand region extraction by using time-series curve and its application to japanese finger-spelled sign language recognition. Procedia - Procedia Comput. Sci. 60, 371–380 (2015)CrossRefGoogle Scholar
  45. 45.
    Yang, J., Horie, R.: An improved computer interface comprising a recurrent neural network and a natural user interface. Procedia - Procedia Comput. Sci. 60, 1386–1395 (2015)CrossRefGoogle Scholar
  46. 46.
    Tao, W., Leu, M.C., Yin, Z.: American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion. Eng. Appl. Artif. Intell. 76(February), 202–213 (2018)Google Scholar
  47. 47.
    Leite, D.Q., et al.: Hand gesture recognition from depth and infrared Kinect data for CAVE applications interaction. Multimed. Tools Appl. 76, 20423–20455 (2017)CrossRefGoogle Scholar
  48. 48.
    Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with jointly calibrated leap motion and depth sensor. Multimed. Tools Appl. 75(22), 14991–15015 (2016)CrossRefGoogle Scholar
  49. 49.
    Zeng, W., Wang, C., Wang, Q.: Hand gesture recognition using leap motion via deterministic learning. Multimed. Tools Appl. 77, 28185–28206 (2018)CrossRefGoogle Scholar
  50. 50.
    Jiang, X., Gang, Z., Carlo, X.: Virtual grasps recognition using fusion of leap motion and force myography. Virtual Reality 22(4), 297–308 (2018)CrossRefGoogle Scholar
  51. 51.
    Quesada, L., López, G., Guerrero, L.: Automatic recognition of the American sign language fingerspelling alphabet to assist people living with speech or hearing I pairments. J. Ambient Intell. Hum. Comput. 8(4), 625–635 (2017)CrossRefGoogle Scholar
  52. 52.
    Ma, C., Wang, A., Chen, G., Chi, X.: Hand joints-based gesture recognition for noisy dataset using nested interval unscented Kalman filter with LSTM network. Vis. Comput. 34(6), 1053–1063 (2018)CrossRefGoogle Scholar
  53. 53.
    Lee, G.C., Yeh, F.H., Hsiao, Y.H.: Kinect-based Taiwanese sign-language recognition system. Multimed. Tools Appl. 75(1), 261–279 (2016)CrossRefGoogle Scholar
  54. 54.
    Benalcázar, M.E.: Machine learning for computer vision: a review of theory and algorithms (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Facultad de Ingeniería en Sistemas Electrónica e IndustrialUniversidad Técnica de AmbatoAmbatoEcuador
  2. 2.Departamento de Informática y Ciencias de la ComputaciónEscuela Politécnica NacionalQuitoEcuador

Personalised recommendations