Skip to main content
SpringerLink
Log in

Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10705))

Included in the following conference series:

Abstract

Emotion recognition is the task of recognizing a person’s emotional state. EEG, as a physiological signal, can provide more detailed and complex information for emotion recognition task. Meanwhile, EEG can’t be changed and hidden intentionally makes EEG-based emotion recognition achieve more effective and reliable result. Unfortunately, due to the cost of data collection, most EEG datasets have small number of EEG data. The lack of data makes it difficult to predict the emotion states with the deep models, which requires enough number of training data. In this paper, we propose to use a simple data augmentation method to address the issue of data shortage in EEG-based emotion recognition. In experiments, we explore the performance of emotion recognition with the shallow and deep computational models before and after data augmentation on two standard EEG-based emotion datasets. Our experimental results show that the simple data augmentation method can improve the performance of emotion recognition based on deep models effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 1 (2015)

    Article  Google Scholar 

  2. Duan, R.N., Zhu, J.Y., et al.: Differential entropy feature for EEG-based emotion classification. In: International IEEE/EMBS Conference on Neural Engineering, vol. 8588, pp. 81–84 (2013)

    Google Scholar 

  3. Shi, L.C., Jiao, Y.Y., et al.: Differential entropy feature for EEG-based vigilance estimation. In: International Conference of the IEEE Engineering in Medicine & Biology Society, p. 6627 (2013)

    Google Scholar 

  4. Lin, Y.P., Wang, C.H., et al.: EEG-based emotion recognition in music listening. IEEE Trans. Biomed. Eng. 57(7), 1798 (2010)

    Article  Google Scholar 

  5. Petrantonakis, P.C., Hadjileontiadis, L.J.: A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. IEEE Trans. Inf. Technol. Biomed. 15(5), 737 (2011). A Publication of the IEEE Engineering in Medicine & Biology Society

    Article  Google Scholar 

  6. Wang, X.W., Nie, D., et al.: Emotional state classification from EEG data using machine learning approach. Neurocomputing 129(4), 94–106 (2014)

    Article  Google Scholar 

  7. Heraz, A., Frasson, C.: Predicting the three major dimensions of the learner’s emotions from brainwaves. Int. J. Comput. Sci. 2(3), 187–193 (2007)

    Google Scholar 

  8. Murugappan, M., Ramachandran, N., et al.: Classification of human emotion from EEG using discrete wavelet transform. J. Biomed. Sci. Eng. 03(4), 390–396 (2010)

    Article  Google Scholar 

  9. Brown, L., Grundlehner, B., et al.: Towards wireless emotional valence detection from EEG. In: International Conference of the IEEE Engineering in Medicine & Biology Society, pp. 2188–2191 (2011)

    Google Scholar 

  10. Lecun, Y., Bottou, L., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  11. Krizhevsky, A., Sutskever, I., et al.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  12. Soleymani, M., Esfeden, S.A., et al.: Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans. Affect. Comput. 7(1), 17–28 (2017)

    Article  Google Scholar 

  13. Li, M.G., Zhu, W., et al.: The novel recognition method with optimal wavelet packet and LSTM based recurrent neural network. In: IEEE International Conference on Mechatronics and Automation, pp. 584–589 (2017)

    Google Scholar 

  14. Kumar, S., Sharma, A., et al.: A deep learning approach for motor imagery EEG signal classification. In: Computer Science and Engineering, pp. 34–39 (2017)

    Google Scholar 

  15. Davidson, R.J., Fox, N.A.: Asymmetrical brain activity discriminates between positive and negative affective stimuli in human infants. Science 218(4578), 1235–1237 (1983)

    Article  Google Scholar 

  16. Klimesch, W., Doppelmayr, M., et al.: Induced alpha band power changes in the human EEG and attention. Neurosci. Lett. 244(2), 73 (1998)

    Article  Google Scholar 

  17. Li, M., Lu, B.L.: Emotion classification based on gamma-band EEG. In: International Conference of the IEEE Engineering in Medicine & Biology Society, vol. 2009, pp. 1323–1326 (2009)

    Google Scholar 

  18. Sourina, O., Liu, Y., et al.: Real-time EEG-based emotion recognition for music therapy. J. Multimodal User Interfaces 5(1–2), 27–35 (2011)

    Google Scholar 

  19. Kothe, C.A., Makeig, S.: Estimation of task workload from EEG data: new and current tools and perspectives. In: International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6547–6551 (2011)

    Google Scholar 

  20. Chanel, G., Kierkels, J.J.M., et al.: Short-term emotion assessment in a recall paradigm. Int. J. Hum.-Comput. Stud. 67(8), 607–627 (2009)

    Article  Google Scholar 

  21. Heraz, A., Frasson, C.: Predicting the three major dimensions of the learner’s emotions from brainwaves. Int. J. Comput. Sci. 2(3), 187–193 (2008)

    Google Scholar 

  22. Zheng, W.L., Zhu, J.Y., et al.: Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. Affect. Comput. PP(99), 1 (2016)

    Google Scholar 

  23. Li, K., Li, X., et al.: Affective state recognition from EEG with deep belief networks. In: IEEE International Conference on Bioinformatics and Biomedicine, pp. 305–310 (2013)

    Google Scholar 

  24. Mao, Z.J., Yao, W.X, et al.: EEG-based biometric identification with deep learning. In: International IEEE EMBS Conference on Neural Engineering, pp. 25–28 (2017)

    Google Scholar 

  25. Li, X., Song, D., et al.: Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In: IEEE International Conference on Bioinformatics and Biomedicine, pp. 352–359 (2017)

    Google Scholar 

  26. Fawzi, A., Samulowitz, H., et al.: Adaptive data augmentation for image classification. In: IEEE International Conference on Image Processing, pp. 3688–3692 (2016)

    Google Scholar 

  27. Hadjidimitriou, S.K., Hadjileontiadis, L.J.: Toward an EEG-based recognition of music liking using time-frequency analysis. IEEE Trans. Biomed. Eng. 59(12), 3498 (2012)

    Article  Google Scholar 

  28. Tibdewal, M.N., Tale, S.A.: Multichannel detection of epilepsy using SVM classifier on EEG signal. In: International Conference on Computing Communication Control and Automation, pp. 1–6 (2016)

    Google Scholar 

  29. Chong, Y.S., Mokhtar, N., et al.: Automated classification and removal of EEG artifacts with SVM and wavelet-ICA. IEEE J. Biomed. Health Inform. PP(99), 1 (2017)

    Article  Google Scholar 

  30. He, K., Zhang, X., et al.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2015)

    Google Scholar 

  31. Soleymani, M., Lichtenauer, J., et al.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2014)

    Article  Google Scholar 

  32. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27 (2011)

    Article  Google Scholar 

  33. Vedaldi, A., Lenc, K.: MatConvNet: convolutional neural networks for matlab. In: ACM International Conference on Multimedia, pp. 689–692 (2015)

    Google Scholar 

  34. Zhang, G., Ge, H.: Prediction of xylanase optimal temperature by support vector regression. Electron. J. Biotechnol. 15(1), 7 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61502311, No. 61373122, No. 61620106008), the Natural Science Foundation of Guangdong Province (No. 2016A030310053, No. 2016A030310039), the Special Program for Applied Research on Super Computation of the NSFC-Guangdong Joint Fund (the second phase) (No. U1501501), Shenzhen Emerging Industries of the Strategic Basic Research Project under Grant (No. JCYJ20160226191842793), the Shenzhen high-level overseas talents program, and the Tencent “Rhinoceros Birds” - Scientific Research Foundation for Young Teachers of Shenzhen University (2015, 2016).

Author information

Authors and Affiliations

  1. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518000, People’s Republic of China

    Fang Wang, Sheng-hua Zhong, Jianfeng Peng & Jianmin Jiang

  2. Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong

    Yan Liu

Authors
  1. Fang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheng-hua Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianfeng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianmin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sheng-hua Zhong .

Editor information

Editors and Affiliations

  1. Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria

    Klaus Schoeffmann

  2. Chulalongkorn University, Bangkok, Thailand

    Thanarat H. Chalidabhongse

  3. City University of Hong Kong, Hong Kong, China

    Chong Wah Ngo

  4. Chulalongkorn University, Bangkok, Thailand

    Supavadee Aramvith

  5. Dublin City University, Dublin, Ireland

    Noel E. O’Connor

  6. Gwangju Institute of Science and Technology, Gwangju, Korea (Republic of)

    Yo-Sung Ho

  7. Tampere University of Technology, Tampere, Finland

    Moncef Gabbouj

  8. Rutgers University, Piscataway, New Jersey, USA

    Ahmed Elgammal

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, F., Zhong, Sh., Peng, J., Jiang, J., Liu, Y. (2018). Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10705. Springer, Cham. https://doi.org/10.1007/978-3-319-73600-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-73600-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-73599-3

  • Online ISBN: 978-3-319-73600-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation