Skip to main content
SpringerLink
Log in

Head Movement Mouse Control Using Convolutional Neural Network for People with Disabilities

  • Conference paper
  • First Online:
13th International Conference on Theory and Application of Fuzzy Systems and Soft Computing — ICAFS-2018 (ICAFS 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 896))

Abstract

In this paper, computer mouse control with head movements and eye blinks was proposed for people with impaired spinal cord injury. The head mouse control is based on finding and predicting eye states and direction of the head. This human computer interface (HCI) is an assistant system for people with physical disabilities who are suffering from motor neuron diseases or severe cerebral palsy. By moving the head to the right, left, up and down moves the mouse pointer and sends mouse button commands using eye blinks. Here, left eye-blink triggers left mouse button, right eye-blink triggers right mouse button and double-eyed sends “holds” command. In this system, eye blink and head movement used same Convolutional Neural Network (CNN) architecture with different number of classes (output). In head movement part, CNN has 5 outputs, (forward, up, down, left and right), in eye-blink part CNN has 2 output either opened or closed. This combined system allows people with down to neck paralyzed, to control computer using head movement and eye blinking. The test results reveal that this system is robust and accurate. This invention allows people with disabilities to use computer with head movements and eye blinks without using any extra hardware.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Vasa, J.J.: Electronic aids for the disabled and the elderly. Med. Instrument. 16(5), 263–264 (1982)

    Google Scholar 

  2. Chen, Y.L., Tang, F.T., Chang, W.H., Wong, M.K., Shih, Y.Y., Kuo, T.S.: The new design of an infrared-controlled human computer interface for the disabled. IEEE Trans. Rehab. Eng. 7, 474–481 (1999)

    Article  Google Scholar 

  3. Computer access solutions for the motor impaired, 12 April 1999. http://www.orin.com/access/

  4. King, D.E.: Dlib-ml: a machine learning toolkit. JMLR 10, 1755–1758 (2009)

    Google Scholar 

  5. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  6. Hecht-Nielsen, R.: Theory of the backpropagation neural network. Neural Netw. 1, 445 (1988). https://doi.org/10.1016/0893-6080(88)90469-8

    Article  Google Scholar 

  7. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  8. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV 2014, ECCV 2014. Lecture Notes in Computer Science, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  9. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of ICLR 2015, pp. 1–14 (2014)

    Google Scholar 

  10. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015). https://doi.org/10.1109/cvpr.2015.7298594

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/cvpr.2016.90

  12. Lawrence, S., Giles, C., Tsoi, A.C., Back, A.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8(1), 98–113 (1997). https://doi.org/10.1109/72.554195

    Article  Google Scholar 

  13. Ciresan, D., Meier, U., Gambardella, L., Schmidhuber, J.: Convolutional neural network committees for handwritten character classification. In: Proceedings of 2011 International Conference on Document Analysis and Recognition, pp. 3207–3220 (2011). https://doi.org/10.1109/icdar.2011.229

  14. Simard, P., Steinkraus, D., Platt, J.: Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of Seventh International Conference on Document Analysis and Recognition, pp. 958–963. https://doi.org/10.1109/icdar.2003.1227801

  15. Jiao, L., Zhang, S., Li, L., Liu, F., Ma, W.: A modified convolutional neural network for face sketch synthesis. Pattern Recogn. 76, 125–136 (2018). https://doi.org/10.1016/j.patcog.2017.10.025

    Article  Google Scholar 

  16. Chudzik, P., Majumdar, S., Calivá, F., Al-Diri, B., Hunter, A.: Microaneurysm detection using fully convolutional neural networks. Comput. Methods Programs Biomed. 158, 185–192 (2018). https://doi.org/10.1016/j.cmpb.2018.02.016

    Article  Google Scholar 

  17. Li, J., Feng, J., Kuo, C.: Deep convolutional neural network for latent fingerprint enhancement. Sig. Process. Image Commun. 60, 52–63 (2018). https://doi.org/10.1016/j.image.2017.08.010

    Article  Google Scholar 

  18. Hussain, S., Anwar, S., Majid, M.: Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282, 248–261 (2018). https://doi.org/10.1016/j.neucom.2017.12.032

    Article  Google Scholar 

  19. Baldominos, A., Saez, Y., Isasi, P.: Evolutionary convolutional neural networks: an application to handwriting recognition. Neurocomputing 283, 38–52 (2018). https://doi.org/10.1016/j.neucom.2017.12.049

    Article  Google Scholar 

  20. Ferreira, A., Giraldi, G.: Convolutional neural network approaches to granite tiles classification. Expert Syst. Appl. 84, 1–11 (2017). https://doi.org/10.1016/j.eswa.2017.04.053

    Article  Google Scholar 

  21. Abiyev, R., Ma’aitah, M.K.S.: Deep convolutional neural networks for chest diseases detection. J. Healthc. Eng. 2018 (2018)

    Google Scholar 

  22. Wachinger, C., Reuter, M., Klein, T.: DeepNAT: deep convolutional neural network for segmenting neuroanatomy. Neuroimage 170, 434–445 (2018). https://doi.org/10.1016/j.neuroimage.2017.02.035

    Article  Google Scholar 

  23. Viola, P., Jones, M.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  24. Kingma, D.P., Ba, J.L.: ADAM: a method for stochastic optimization. In: ICLR 2015 (2015)

    Google Scholar 

  25. TensorFlow. https://www.tensorflow.org/. Accessed 19 June 2018

  26. Song, F., Tan, X., Liu, X., Chen, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn. 47(9), 2825–2838 (2014)

    Article  Google Scholar 

  27. Abiyev, R.H., Altunkaya, K.: Neural network based biometric personal identification using fast iris segmentation. Int. J. Control Autom. Syst. 7(1), 17–23 (2009)

    Article  Google Scholar 

  28. Abiyev, R.H., Abizade, S.: Diagnosing Parkinson’s diseases using fuzzy neural system. Comput. Math. Methods Med. 2016 (2016). Article ID 1267919

    Article  MathSciNet  Google Scholar 

  29. Abiyev, R.H., Kilic, K.: Adaptive Iris Segmentation. Lecture Notes in Computer Sciences. Springer/CS Press, Heidelberg (2009)

    Book  Google Scholar 

  30. Abiyev, R., Altunkaya, K.: Personal iris recognition using neural networks. Int. J. Secur. Appl. 2(2), 41–50 (2008)

    Google Scholar 

  31. Abiyev, R., Altunkaya, K.: Neural Network Based Biometric Personal Identification. Lecture Notes in Computer Sciences. Springer/CS Press, Heidelberg (2007)

    Book  Google Scholar 

  32. Helwan, A., Ozsahin, D.U., Abiyev, R., John, B.: One-year survival prediction of myocardial infarction. (IJACSA). Int. J. Adv. Comput. Sci. Appl. 8, 173–178 (2017)

    Google Scholar 

  33. Idoko, J.B., Abiyev, R.H., Ma’aitah, M.K.S.: Intelligent machine learning algorithms for colour segmentation. WSEAS Trans. Sig. Process. 13, 232–240 (2017)

    Google Scholar 

  34. Idoko, J.B., Abiyev, R., Ma’aitah, M.K.S., Hamit, A.: Integrated artificial intelligence algorithm for skin detection. In: International Conference Applied Mathematics, Computational Science and Systems Engineering (AMCSE 2017), ITM Web of Conferences, vol. 16, p. 02004 (2018)

    Google Scholar 

  35. Ma’aitah, M.K.S., Abiyev, R., Idoko, J.B.: Intelligent classification of liver disorder using fuzzy neural system. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 8, 25–31 (2017)

    Google Scholar 

  36. Helwan, A., Idoko, J.B., Abiyev, R.H.: Machine learning techniques for classification of breast tissue. In: 9th International Conference on Theory and Application of Soft Computing, Computing with Words and Perception, ICSCCW 2017. Procedia Computer Science, vol. 120, pp. 402–410 (2017)

    Article  Google Scholar 

  37. Abiyev, R.H., Arslan, M., Gunsel, I., Cagman, A.: Robot pathfinding using vision based obstacle detection. In: IEEE International Conference on Cybernetics, pp. 1–6 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Applied Artificial Intelligence Research Centre, Department of Computer Engineering, Near East University, Mersin-10, North Cyprus, Turkey

    Murat Arslan, Idoko John Bush & Rahib H. Abiyev

Authors
  1. Murat Arslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Idoko John Bush
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rahib H. Abiyev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murat Arslan .

Editor information

Editors and Affiliations

  1. Joint MBA Program, Azerbaijan State Oil and Industry University, Baku, Azerbaijan

    Rafik A. Aliev

  2. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Janusz Kacprzyk

  3. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  4. Department of Electrical and Computer Engineering, University of Texas at San Antonio, San Antonio, TX, USA

    Mo. Jamshidi

  5. Department of Mechatronics, Near East University, Nicosia TRNC, Turkey

    Fahreddin M. Sadikoglu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Arslan, M., Bush, I.J., Abiyev, R.H. (2019). Head Movement Mouse Control Using Convolutional Neural Network for People with Disabilities. In: Aliev, R., Kacprzyk, J., Pedrycz, W., Jamshidi, M., Sadikoglu, F. (eds) 13th International Conference on Theory and Application of Fuzzy Systems and Soft Computing — ICAFS-2018. ICAFS 2018. Advances in Intelligent Systems and Computing, vol 896. Springer, Cham. https://doi.org/10.1007/978-3-030-04164-9_33

Download citation

Publish with us

Policies and ethics

Search

Navigation