Skip to main content

Advertisement

SpringerLink
Log in

An Efficient Human Computer Interaction through Hand Gesture Using Deep Convolutional Neural Network

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

This paper focuses on the achievement of effective human–computer interaction using only webcam by continuous locating or tracking and recognizing the hand region. We detected the region of interest (ROI) in the captured image range and classify hand gestures for specific tasks. Firstly, background subtraction is used based on the main frame captured by webcam, and some preprocessing are done, and then YCrCb skin segmentation is used on RGB subtracted image. The ROI is detected using Haar cascade classifier for hand palm detection. Next, kernelized correlation filters tracking algorithm is used to avoid noise or background influences for tracking the ROI, and the median-flow tracking algorithm is used for depth tracking. The ROI is converted to a binary channel (black and white), resized to 54 × 54. Then gesture recognition is done using a 2D convolutional neural network (CNN) by entering the preprocessed ROI on the architecture. Two predictions are made based on skin segmented frame and image dilated frame, and gesture is recognized from the maximum value of those two predictions. The tracking and recognition process is continued until the ROI is presented on the frames. Finally, after validation, the proposed system has successfully obtained a recognition rate of 98.44%, which is usable for the practical and real-time application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. G Coleman, R Ward. Gesture recognition performance, applications and features. 4th ed. New York : Nova Science Publishers, Inc; 2018.

  2. A Voulodimos, N Doulamis, A Doulamis, E Protopapadakis. Deep learning for computer vision: a brief review. Comput Intell Neurosci. 2018;2018:1–13. https://doi.org/10.1155/2018/7068349.

  3. Rezwanul Haque M, Milon Islam M, Saeed Alam K, Iqbal H. A computer vision based lane detection approach. Int J Image, Graph Signal Process. 2019;11:27–34. https://doi.org/10.5815/ijigsp.2019.03.04.

    Article  Google Scholar 

  4. Dabre K, Dholay S (2014) Machine learning model for sign language interpretation using webcam images. In: 2014 International conference on circuits, systems, communication and information technology applications, CSCITA 2014

  5. Hasan H, Abdul-Kareem S. Static hand gesture recognition using neural networks Artif. Intell Rev. 2014;41:147–81. https://doi.org/10.1007/s10462-011-9303-1.

    Article  Google Scholar 

  6. Molchanov P, Yang X, Gupta S, Kim K, Tyree S, Kautz J (2016) online detection and classification of dynamic hand gestures with recurrent 3d convolutional neural Networks. In: proceedings of the IEEE computer society conference on computer vision and pattern Recognition

  7. Haria A, Subramanian A, Asokkumar N, Poddar S, Nayak JS. Hand gesture recognition for human computer interaction. Procedia Comp Sci. 2017;115:367–74.

    Article  Google Scholar 

  8. Zhang Z. Microsoft kinect sensor and its effect. IEEE Multimed. 2012;19(2):4–10. https://doi.org/10.1109/MMUL.2012.24.

    Article  Google Scholar 

  9. Memo A, Minto L, Zanuttigh P (2015) Exploiting silhouette descriptors and synthetic data for hand gesture recognition. In: Italian chapter conference 2015—smart tools and apps in computer graphics, STAG 2015

  10. Keskin C, Kiraç F, Kara YE, Akarun L (2011) Real time hand pose estimation using depth sensors. In: proceedings of the IEEE international conference on computer Vision

  11. Rigatti SJ. Random Forest J Insur Med. 2017. https://doi.org/10.17849/insm-47-01-31-39.1.

    Article  Google Scholar 

  12. Rahman MA, Sadi MS, Islam MM, Saha P (2019) Design and Development of Navigation Guide for Visually Impaired People. In: 2019 IEEE International conference on biomedical engineering, computer and information technology for health (BECITHCON). IEEE, pp 89–92

  13. Khanom M, Sadi MS, Islam MM (2019) A comparative study of walking assistance Tools developed for the visually impaired people. 1st Int conf adv sci eng robot technol 2019, ICASERT 2019 2019:1–5. https://doi.org/10.1109/ICASERT.2019.8934566

  14. Rahman MM, Islam M, Ahmmed S. “BlindShoe”: an electronic guidance system for the visually impaired people. J Telecommun Electron Comp Eng (JTEC). 2019;11:49–54.

    Google Scholar 

  15. Habib A, Islam MM, Kabir MN, Mredul MB, Hasan M. Staircase detection to guide visually impaired people: a hybrid approach. Rev d’Intell Artif. 2019;33:327–34. https://doi.org/10.18280/ria.330501.

    Article  Google Scholar 

  16. Islam MM, Sadi MS, Zamli KZ, Ahmed MM. Developing walking assistants for visually impaired people: a review. IEEE Sens J. 2019;19:2814–28. https://doi.org/10.1109/JSEN.2018.2890423.

    Article  Google Scholar 

  17. Alam N, Islam M, Habib A, Mredul MB. Staircase detection systems for the visually impaired people : a review. Int J Comp Sci Inf Secur (IJCSIS). 2018;16:13–18.

    Google Scholar 

  18. Kamal MM, Bayazid AI, Sadi MS, Islam MM, Hasan N (2017) Towards developing walking assistants for the visually impaired people. In: 2017 IEEE region 10 humanitarian technology Conference (R10-HTC). IEEE, pp 238–241

  19. Islam MM, Sadi MS, Islam MM, Hasan MK (2018) A New Method for Road Surface Detection. In: 2018 4th International conference on electrical engineering and information & communication technology (iCEEiCT). IEEE, pp 624–629

  20. Islam MM, Sadi MS (2018) Path Hole Detection to Assist the Visually Impaired People in Navigation. In: 2018 4th International conference on electrical engineering and information & communication technology (iCEEiCT). IEEE, pp 268–273

  21. Islam MM, Neom NH, Imtiaz MS, Nooruddin S, Islam MR, Islam MR. A review on fall detection systems using data from smartphone sensors. Ing des Syst d’Inf. 2019;24:569–76. https://doi.org/10.18280/isi.240602.

    Article  Google Scholar 

  22. Islam MM, Hasan MK, Billah MM, Uddin MM (2017) Development of smartphone-based student attendance system. In: 2017 IEEE region 10 humanitarian technology conference (R10-HTC). IEEE, pp 230–233

  23. Nooruddin S, Milon Islam M, Sharna FA. An IoT based device-type invariant fall detection system. Internet Things. 2020;9:100130. https://doi.org/10.1016/j.iot.2019.100130.

    Article  Google Scholar 

  24. Rahaman A, Islam M, Islam M, Sadi M, Nooruddin S. Developing IoT based smart health monitoring systems: a review. Rev d’Int Artif. 2019;33:435–40. https://doi.org/10.18280/ria.330605.

    Article  Google Scholar 

  25. Islam MM, Rahaman A, Islam MR. Development of smart healthcare monitoring system in IoT environment. SN Comput Sci. 2020;1:185. https://doi.org/10.1007/s42979-020-00195-y.

    Article  Google Scholar 

  26. Brutzer S, Höferlin B, Heidemann G (2011) Evaluation of background subtraction techniques for video surveillance. In: proceedings of the IEEE computer society conference on computer vision and pattern recognition

  27. Shaik KB, Ganesan P, Kalist V, Sathish B, Jenitha JMM. Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Comp Sci. 2015;57:41–8. https://doi.org/10.1016/j.procs.2015.07.362.

    Article  Google Scholar 

  28. Tsagaris A, Manitsaris S. Colour space comparison for skin detection in finger gesture recognition. Int J Adv Eng Technol. 2013;6(4):1431.

    Google Scholar 

  29. Soo S. Object detection using Haar-cascade Classifier. Inst Comp Sci Univ Tartu. 2014;2(3):1–12.

  30. Fisher R, Perkins S, Walker A, Wolfart E (2003) Spatial Filters—Gaussian Smoothing. Image Process. Learn. Resour.

  31. Tang M, Yu B, Zhang F, Wang J (2018) High-speed tracking with Multi-kernel correlation Filters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition

  32. Grigorev A, Derevitskii I, Bochenina K. Analysis of special transport behavior using computer vision analysis of video from traffic cameras. Commun Comput Inform Sci. 2018;858:289–301. https://doi.org/10.1007/978-3-030-02843-5_23.

  33. Cheng C, Parhi KK. Fast 2D convolution algorithms for convolutional neural networks. IEEE Trans Circuits Syst I Regul Pap. 2020. https://doi.org/10.1109/TCSI.2020.2964748.

    Article  Google Scholar 

  34. Lecun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.

    Article  Google Scholar 

  35. Haque S, Sadi MS, Rafi MEH, Islam MM, Hasan MK (2020) Real-Time Crowd Detection to Prevent Stampede. pp 665–678

  36. Muhammad LJ, Islam MM, Sharif US, Ayon SI. Predictive data mining models for novel coronavirus (COVID-19) Infected Patients Recovery. SN Comput Sci. 2020;1:216. https://doi.org/10.1007/s42979-020-00216-w.

  37. Hasan MK, Islam MM, Hashem MMA (2016) Mathematical model development to detect breast cancer using multigene genetic programming. In: 2016 5th International conference on informatics, electronics and vision (ICIEV). IEEE, pp 574–579

  38. Das S, Sadi MS, Ahsanul Haque M, Islam MM (2019) A machine learning approach to protect electronic devices from damage using the concept of outlier. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT). IEEE, pp 1–6

  39. Haque MR, Islam MM, Iqbal H, Reza MS, Hasan MK (2018) Performance evaluation of random forests and artificial neural networks for the classification of Liver Disorder. In: 2018 International conference on computer, communication, chemical, material and electronic engineering (IC4ME2). IEEE, pp 1–5

  40. Islam MM, Iqbal H, Haque MR, Hasan MK (2017) Prediction of breast cancer using support vector machine and K-nearest neighbors. In: 2017 IEEE Region 10 Humanitarian technology conference (R10-HTC). IEEE, pp 226–229

  41. Islam Ayon S, Milon Islam M. Diabetes prediction: a deep learning approach. Int J Inf Eng Electron Bus. 2019;11:21–7. https://doi.org/10.5815/ijieeb.2019.02.03.

    Article  Google Scholar 

  42. Milon Islam M, Kabir MN, Sadi MS, Morsalin MI, Haque A, Wang J. A novel approach towards tamper detection of digital holy quran generation. Lect Notes Electr Eng. 2020;632:297–308.

    Article  Google Scholar 

  43. Ayon SI, Islam MM, Hossain MR. Coronary artery heart disease prediction: a comparative study of computational intelligence techniques. IETE J Res. 2020. https://doi.org/10.1080/03772063.2020.1713916.

    Article  Google Scholar 

  44. Hasan M, Islam MM, Zarif MII, Hashem MMA. Attack and anomaly detection in IoT sensors in IoT sites using machine learning approaches. Internet Things. 2019;7:100059. https://doi.org/10.1016/j.iot.2019.100059.

    Article  Google Scholar 

Download references

Acknowledgment

The authors would like to thank the Department of Computer Science and Engineering (CSE), Khulna University of Engineering & Technology (KUET) to facilitate the work.

Funding

No funding sources.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna, 9203, Bangladesh

    Md. Milon Islam, Md. Repon Islam & Md. Saiful Islam

Authors
  1. Md. Milon Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Md. Repon Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Md. Saiful Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md. Milon Islam.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advances in Computational Approaches for Artificial Intelligence, Image Processing, IoT and Cloud Applications” guest edited by Bhanu Prakash K N and M. Shivakumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, M.M., Islam, M.R. & Islam, M.S. An Efficient Human Computer Interaction through Hand Gesture Using Deep Convolutional Neural Network. SN COMPUT. SCI. 1, 211 (2020). https://doi.org/10.1007/s42979-020-00223-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-020-00223-x

Keywords

Search

Navigation