Effect of hand grip actions on object recognition process: a machine learning-based approach for improved motor rehabilitation

Abstract

Brain–computer interface (BCI) is the current trend in technology expansion as it provides an easy interface between human brain and machine. The demand for BCI-based applications is growing tremendously, and efforts are in progress to deploy BCI devices for real-world applications. One of the widely known applications of BCI technology is rehabilitation in which BCI devices can provide various types of assistance to specially abled persons. In this paper, the effect of hand actions on objects is analyzed for motor-related mental task. The proposed approach analyzes electroencephalogram (EEG)-based brain activity which was captured for images shown with different gripping actions on objects. The EEG recordings are first pre-processed, followed by extraction of epochs and frequency bands using discrete wavelet transform; afterward, feature extraction followed by training and classification steps is performed for classifying the grip action into congruent (correct) and incongruent (incorrect) grip categories. The proposed work makes use of average power and relative wavelet energy as discriminating features which are then fed to train an artificial neural network for automatically classifying the incoming EEG patterns into correct or incorrect object hand grips. The performance evaluation of the proposed system is done on real EEG dataset obtained from 14 subjects. Experimental results have shown an accuracy of 75%. Also, to evaluate the effectiveness of our work, a comparison of our work with other state-of-the-art works reported by different authors is presented at the end. The results show the effectiveness of proposed approach and suggest further that the system can be used to analyze and train subjects having motor-related disabilities for perceiving correct or incorrect hand grips on objects.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  1. 1.

    Ramadan RA, Refat S, Elshahed MA, Ali RA (2015) Basics of brain computer interface. Springer, Berlin. https://doi.org/10.1007/978-3-319-10978-7_2

    Google Scholar 

  2. 2.

    Hassanien A, Azar A (2015) Brain computer interfaces: current trends and applications, ISRL, vol 74. Springer, Heidelberg. ISBN 978-3-319-10977-0

    Google Scholar 

  3. 3.

    Sanjay K, Eun YY, Glyn WH (2012) Perceptual and motor-based responses to hand actions on objects: evidence from ERPs. Exp Brain Res 220:153–164

    Article  Google Scholar 

  4. 4.

    Sanjay K, Malcolm R, Glyn WH (2013) Mu rhythm desynchronization reveals motoric influences of hand action on object recognition. Front Hum Neurosci 7:66

    Google Scholar 

  5. 5.

    Alexis OR, Ioannisely BT, Hojjat A, John AB (2014) Combined corticospinal and reticulospinal effects on upper limb muscles. Neurosci Lett 561:30–34. https://doi.org/10.1016/j.neulet.2013.12.043

    Article  Google Scholar 

  6. 6.

    Chaudhari R, Galiyawala HJ (2017) A review on motor imagery signal classification for BCI. Signal Process Int J SPIJ 11:16–34

    Google Scholar 

  7. 7.

    Shedeed HA, Issa MF (2016) Brain-EEG signal classification based on data normalization for controlling a robotic arm. Int J Tomogr Simul 29(1):72–85

    Google Scholar 

  8. 8.

    Srinivasan V, Eswaran C, Sriraam N (2007) Approximate entropy-based epileptic EEG detection using artificial neural networks. IEEE Trans Inf Technol Biomed 11:288–295

    Article  Google Scholar 

  9. 9.

    Tiwari AK, Pachori RB, Kanhangad V, Panigrahi BK (2017) Automated diagnosis of epilepsy using key-point-based local binary pattern of EEG signals. IEEE J Biomed Health Inf 21(4):888–896. https://doi.org/10.1109/JBHI.2016.2589971

    Article  Google Scholar 

  10. 10.

    Acharya UR, Molinari F, Sree SV, Chattopadhyay S, Ng KH, Suri JS (2012) Automated diagnosis of epileptic EEG using entropies. Biomed Signal Proc Control 7:401–408

    Article  Google Scholar 

  11. 11.

    AlSharabi K, Ibrahim S, Djemal R, Alsuwailem A (2016) A DWT-entropy-ANN based architecture for epilepsy diagnosis using EEG signals. In: 2016 2nd international conference on advanced technologies for signal and image processing (ATSIP), pp 288–291

  12. 12.

    Kannathal N, Lim CM, Acharya UR, Sadasivan PK (2005) Entropies for detection of epilepsy in EEG. Comput Methods Progr Biomed 80(3):187–194

    Article  Google Scholar 

  13. 13.

    Umut O, Mahmut H, Mahmut O (2011) EEG signals classification using the K-means clustering and a multilayer perceptron neural network model. Expert Syst Appl 38(10):13475–13481. https://doi.org/10.1016/j.eswa.2011.04.149

    Article  Google Scholar 

  14. 14.

    Ridha D, Khalil A, Sutrisno I, Abdullah A (2017) EEG-Based computer aided diagnosis of autism spectrum disorder using wavelet, entropy, and ANN. In: BioMed research international

  15. 15.

    William JB, Adrienne LT, Helen TF, Charles AN (2011) EEG complexity as a biomarker for autism spectrum disorder risk. In: BMC medicine

  16. 16.

    Chai R, Naik GR, Nguyen TN, Ling SH, Tran Y, Craig A, Nguyen HT (2017) Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system. IEEE J Biomed Health Inf 21:715–724

    Article  Google Scholar 

  17. 17.

    Saha A, Konar A, Nagar AK (2017) EEG analysis for cognitive failure detection in driving using type-2 fuzzy classifiers. IEEE Trans Emerg Top Comput Intell 1:437–453

    Article  Google Scholar 

  18. 18.

    Bahy MME, Hosny M, Mohamed WA, Ibrahim S (2017) EEG signal classification using neural network and support vector machine in brain computer interface. In: Proceedings of the international conference on advanced intelligent systems and informatics 2016. Springer International Publishing, Cham, pp 246–256

  19. 19.

    Craik A, He Y, Contreras-Vidal JL (2019) Deep learning for electroencephalogram (EEG) classification tasks: a review. J Neural Eng 16(3):031001

    Article  Google Scholar 

  20. 20.

    Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J (2019) Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng 16:051001

    Article  Google Scholar 

  21. 21.

    Subasi A (2007) EEG signal classification using wavelet feature extraction and a mixture of expert model. Expert Syst Appl 32:1084–1093

    Article  Google Scholar 

  22. 22.

    Akansu AN, Haddad RA (2001) Wavelet transform, chapter 6. https://doi.org/10.1016/B978-012047141-6/50006-9

  23. 23.

    Kehtarnavaz N (2008) Frequency domain processing, chapter 7. https://doi.org/10.1016/B978-0-12-374490-6.00007-6

  24. 24.

    Vivas EL, García-González A, Figueroa I, Fuentes RQ (2013) Discrete wavelet transform and ANFIS classifier for brain-machine interface based on EEG. In: 2013 6th international conference on human system interactions (HSI), pp 137–144

  25. 25.

    Shannon CE (2001) A mathematical theory of communication. SIGMO BILE Mob Comput Commun Rev 5(1):3–55. https://doi.org/10.1145/584091.584093

    MathSciNet  Article  Google Scholar 

  26. 26.

    Lange G, Low CY, Johar K, Hanapiah FA, Kamaruzaman F (2016) Classification of electroencephalogram data from hand grasp and release movements for BCI controlled prosthesis. Proc Technol 26:374–381. ISSN 2212-0173. https://doi.org/10.1016/j.protcy.2016.08.048

  27. 27.

    Cho J-H, Jeong J-R, Kim D-J, Lee S-W (2020) A novel approach to classify natural grasp actions by estimating muscle activity patterns from EEG signals. In: 2020 8th international winter conference on brain-computer interface (BCI), pp 1–4. IEEE

  28. 28.

    Ramadhan MM, Wijaya SK, Prajitno P (2019) Classification of EEG signals from motor imagery of hand grasp movement based on neural network approach. In: 2019 IEEE international conference on signals and systems (ICSigSys), Bandung, Indonesia, 2019, pp 92–96

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Anju Mishra.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Animal rights

This article does not contain any studies with animals performed by any of the authors.

Informed consent

This article contains methodology that uses the data, which were collected previously in year 2011 by one of the mentioned authors Dr. Sanjay Kumar. Participants have provided written consent prior to participation. This previous study was also approved by the Local Ethics Committee of the University of Birmingham and conformed to the Declaration of Helsinki.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mishra, A., Sharma, S., Kumar, S. et al. Effect of hand grip actions on object recognition process: a machine learning-based approach for improved motor rehabilitation. Neural Comput & Applic (2020). https://doi.org/10.1007/s00521-020-05125-w

Download citation

Keywords

  • Brain–computer interfaces
  • Machine learning
  • EEG
  • Hand action
  • Neural network