Multiclass Classification of Spatially Filtered Motor Imagery EEG Signals Using Convolutional Neural Network for BCI Based Applications

Abstract

Purpose

Brain–Computer Interface (BCI) system offers a new means of communication for those with paralysis or severe neuromuscular disorders. BCI systems based on Motor Imagery (MI) Electroencephalography (EEG) signals enable the user to convert their thoughts into actions without any voluntary muscle movement. Recently, Convolutional neural network (CNN) is used for the classification of MI signals. However, to produce good MI classification, it is necessary to effectively represent the signal as an input image to the CNN and train the deep learning classifier using large training data.

Methods

In this work, EEG signals are acquired over 16 channels and are filtered using a bandpass filter with the frequency range of 1 to 100 Hz. The processed signal is spatially filtered using Common Spatial Pattern (CSP) filter. The spectrograms of the spatially filtered signals are given as input to CNN. A single convolutional layer CNN is designed to classify left hand, right hand, both hands, and feet MI EEG signals. The size of the training data is increased by augmenting the spectrograms of the EEG signals.

Results

The CNN classifier was evaluated using MI signals acquired from twelve healthy subjects. Results show that the proposed method achieved an average classification accuracy of 95.18 ± 2.51% for two-class (left hand and right hand) and 87.37 ± 1.68% for four-class (Left hand, Right hand, Both hands, and Feet) MI.

Conclusion

Thus, the method manifests that this 2D representation of 1D EEG signal along with image augmentation shows a high potential for classification of MI EEG signals using the designed CNN model.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Wolpaw, J. R., & Wolpaw, E. W. (2012). Brain-computer interfaces: something new under the sun. In J. Wolpaw & E. W. Wolpaw (Eds.), Brain-computer interfaces: Principles and practice (pp. 1–14). Oxford: Oxford University Press.

    Google Scholar 

  2. 2.

    Zhang, W., Sun, F., Wu, H., Tan, C., & Ma, Y. (2019). Asynchronous brain-computer interface shared control of robotic grasping. Tsinghua Science and Technology,24(3), 360–370. https://doi.org/10.26599/TST.2018.9010111.

    Article  Google Scholar 

  3. 3.

    Rahman, M. K. M., & Joadder, M. A. M. (2017). A review on the components of EEG-based motor imagery classification with quantitative comparison. Appl Theory Comput Technol,2(2), 1–15.

    Article  Google Scholar 

  4. 4.

    Thomas, J., Maszczyk, T., Sinha, N., Kluge, T., & Dauwels, J. (2017). Deep learning-based classification for brain-computer interfaces. In IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 234–239)

  5. 5.

    Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Muller, K. R. (2007). Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine,25(1), 41–56. https://doi.org/10.1109/MSP.2008.4408441.

    Article  Google Scholar 

  6. 6.

    Lu, H., Eng, H. L., Guan, C., Plataniotis, K. N., & Venetsanopoulos, A. N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering,57(12), 2936–2946. https://doi.org/10.1109/TBME.2010.2082540.

    Article  Google Scholar 

  7. 7.

    Tang, Z., Li, C., & Sun, S. (2017). Single-trial EEG classification of motor imagery using deep convolutional neural networks. Optik,130, 11–18. https://doi.org/10.1016/j.ijleo.2016.10.117.

    Article  Google Scholar 

  8. 8.

    Zhang, J., Yan, C., & Gong, X. (2017, October). Deep convolutional neural network for decoding motor imagery based brain computer interface. In IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) (pp. 1–5). https://doi.org/10.1109/ICSPCC.2017.8242581

  9. 9.

    Tian, G., & Liu, Y. (2018). Study on Classification of Left-Right Hands Motor Imagery EEG Signals Based on CNN. In IEEE International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC) (pp. 324–329). https://doi.org/10.1109/ICCI-CC.2018.8482042

  10. 10.

    Lee, H. K., & Choi, Y. S. (2018). A convolution neural networks scheme for classification of motor imagery EEG based on wavelet time-frequecy image. In IEEE International Conference on Information Networking (ICOIN) (pp. 906–909). https://doi.org/10.1109/ICOIN.2018.8343254

  11. 11.

    Schirrmeister, R., Gemein, L., Eggensperger, K., Hutter, F., & Ball, T. (2017). Deep learning with convolutional neural networks for decoding and visualization of EEG pathology. In IEEE Signal Processing in Medicine and Biology Symposium (SPMB) (pp. 1–7). https://doi.org/10.1109/SPMB.2017.8257015

  12. 12.

    Yang, H., Sakhavi, S., Ang, K. K., & Guan, C. (2015). On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification. In International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 2620–2623). https://doi.org/10.1109/EMBC.2015.7318929

  13. 13.

    Korhan, N., Dokur, Z., & Olmez, T. (2019). Motor imagery based EEG classification by using common spatial patterns and convolutional neural networks. In Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT) (pp. 1–4). https://doi.org/10.1109/EBBT.2019.8741832

  14. 14.

    Başar, E., & Düzgün, A. (2016). The CLAIR model: Extension of Brodmann areas based on brain oscillations and connectivity. International Journal of Psychophysiology,103, 185–198. https://doi.org/10.1016/j.ijpsycho.2015.02.018.

    Article  Google Scholar 

  15. 15.

    Repovs, G. (2010). Dealing with noise in EEG recording and data analysis. Informatica Medica Slovenica,15(1), 18–25.

    Google Scholar 

  16. 16.

    Tabar, Y. R., & Halici, U. (2016). A novel deep learning approach for classification of EEG motor imagery signals. Journal of neural engineering,14(1), 016003. https://doi.org/10.1088/1741-2560/14/1/016003.

    Article  Google Scholar 

  17. 17.

    Uktveris, T., & Jusas, V. (2017). Application of convolutional neural networks to four-class motor imagery classification problem. Information Technology and Control,46(2), 260–273. https://doi.org/10.5755/j01.itc.46.2.17528.

    Article  Google Scholar 

  18. 18.

    Grosse-Wentrup, M., & Buss, M. (2008). Multiclass common spatial patterns and information theoretic feature extraction. IEEE Transactions on Biomedical Engineering,55(8), 1991–2000. https://doi.org/10.1109/TBME.2008.921154.

    Article  Google Scholar 

  19. 19.

    Moffett, S. X., O’Malley, S. M., Man, S., Hong, D., & Martin, J. V. (2017). Dynamics of high frequency brain activity. Scientific Reports,7(1), 1–5. https://doi.org/10.1038/s41598-017-15966-6.

    Article  Google Scholar 

  20. 20.

    Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018). Convolutional neural networks: an overview and application in radiology. Insights into Imaging,9(4), 611–629. https://doi.org/10.1007/s13244-018-0639-9.

    Article  Google Scholar 

  21. 21.

    Fawzi, A., Samulowitz, H., Turaga, D., & Frossard, P. (2016). Adaptive data augmentation for image classification. In IEEE International Conference on Image Processing (ICIP) (pp. 3688–3692). https://doi.org/10.1109/ICIP.2016.7533048

  22. 22.

    Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097–1105). https://doi.org/10.1145/3065386

  23. 23.

    Alaskar, H. M. A. (2018). Deep learning of EMG time–frequency representations for identifying normal and aggressive actions. In International Journal of Computer Science and Network Security (IJCSNS) (pp. 16–25)

  24. 24.

    Hussain, Z., Gimenez, F., Yi, D., & Rubin, D. (2017). Differential data augmentation techniques for medical imaging classification tasks. In AMIA Annual Symposium Proceedings (pp. 979–984)

  25. 25.

    Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017). A survey of deep neural network architectures and their applications. Neurocomputing,234, 11–26. https://doi.org/10.1016/j.neucom.2016.12.038.

    Article  Google Scholar 

  26. 26.

    Sakhavi, S., Guan, C., & Yan, S. (2018). Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems,29(11), 5619–5629. https://doi.org/10.1109/TNNLS.2018.2789927.

    MathSciNet  Article  Google Scholar 

  27. 27.

    Martín-Clemente, R., Olias, J., Thiyam, D. B., Cichocki, A., & Cruces, S. (2018). Information theoretic approaches for motor-imagery BCI systems: Review and experimental comparison. Entropy,20(7), 1–29. https://doi.org/10.3390/e20010007.

    Article  Google Scholar 

  28. 28.

    Chaudhary, S., Taran, S., Bajaj, V., & Sengur, A. (2019). Convolutional neural network based approach towards motor imagery tasks EEG signals classification. IEEE Sensors Journal,19(12), 4494–4500. https://doi.org/10.1109/JSEN.2019.2899645.

    Article  Google Scholar 

  29. 29.

    Pfurtscheller, G., Neuper, C., Flotzinger, D., & Pregenzer, M. (1997). EEG-based discrimination between imagination of right and left hand movement. Electroencephalography and Clinical Neurophysiology,103(6), 642–651. https://doi.org/10.1016/S0013-4694(97)00080-1.

    Article  Google Scholar 

  30. 30.

    Trakoolwilaiwan, T., Behboodi, B., Lee, J., Kim, K., & Choi, J. W. (2017). Convolutional neural network for high-accuracy functional near-infrared spectroscopy in a brain–computer interface: Three-class classification of rest, right-, and left-hand motor execution. Neurophotonics,5(1), 011008. https://doi.org/10.1117/1.NPh.5.1.011008.

    Article  Google Scholar 

Download references

Funding

We wish to inform that this work was supported by the Life Science Research Board (LSRB), Defence Research and Development Organization (DRDO) under the Grants-in-Aid scheme (Grant No. LSRB-291/LS&BD/2017). Nijisha Shajil would like to thank the Department of Science and Technology (DST), India for funding her research through INSPIRE fellowship (IF180459).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sasikala Mohan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the University and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study. No identifiable personal information is disclosed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shajil, N., Mohan, S., Srinivasan, P. et al. Multiclass Classification of Spatially Filtered Motor Imagery EEG Signals Using Convolutional Neural Network for BCI Based Applications. J. Med. Biol. Eng. (2020). https://doi.org/10.1007/s40846-020-00538-3

Download citation

Keywords

  • Brain-computer interface
  • EEG data
  • Motor imagery
  • Convolutional neural network
  • Common spatial pattern
  • Spectrogram