Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 17, pp 21967–21994 | Cite as

Sparsity constrained differential evolution enabled feature-channel-sample hybrid selection for daily-life EEG emotion recognition

  • Yixiang Dai
  • Xue Wang
  • Pengbo Zhang
  • Weihang Zhang
  • Junfeng Chen
Article
  • 166 Downloads

Abstract

Electroencephalography (EEG) reflects the activities of human brain and it can represent different emotional states to provide impersonal scientific evidence for daily-life emotional health monitoring. However, traditional multi-channel EEG sensing contains irrelevant or even interferential features, channels or samples, leading to redundant data and hardware complexity. This paper proposes a feature-channel-sample hybrid selection method to improve the channel selection, feature extraction and classification scheme for daily-life EEG emotion recognition. The features and channels are selected in pair with sparsity constrained differential evolution where the feature-channel pairs are optimized synchronously in the global search. Furthermore, the distance evaluation is carried out to remove abnormal samples to improve the emotion recognition accuracy. Therefore, efficient feature vectors for valence-arousal classification can be obtained by a small number of sparsely distributed channels. The experiments are based on the widely-used emotion recognition database DEAP and generate a feature-channel-sample hybrid selection scheme with optimized parameter settings. It can be derived that the proposed method can reduce the EEG channels sharply and maintain a relatively high accuracy compared with the related work. Furthermore, by applying this optimal scheme in practice, the real-scene daily-life EEG emotion recognition experiments are carried out on a sparsity constrained web-enabled system and a 10-fold cross validation is organized to confirm the performance. In conclusion, this paper provides a practical and efficient hardware configuration and feature-channel-sample optimal selection scheme for daily-life EEG emotion recognition.

Keywords

EEG signal processing Emotion recognition Feature selection Channel selection Sparsity constrained differential evolution (SCDE) 

Notes

Acknowledgements

This paper is supported by National Natural Science Foundation of China under Grant #61472216, and by PhD Programs Foundation of Ministry of Education of China under Grant #20120002110067. The authors would like to thank Sander Koelstra, Christian Mühl and other dedicated researchers in the corresponding groups for providing DEAP dataset.

References

  1. 1.
    AlZoubi O, Calvo R A, Stevens R H (2009) Classification of eeg for affect recognition: an adaptive approach. In: AI 2009: advances in artificial intelligence. Springer, pp 52–61Google Scholar
  2. 2.
    Arvaneh M, Guan C, Ang K K, Quek C (2011) Optimizing the channel selection and classification accuracy in eeg-based bci. IEEE Trans Biomed Eng 58 (6):1865–1873CrossRefGoogle Scholar
  3. 3.
    Casson A J, Yates D C, Smith S J, Duncan J S, Rodriguez-Villegas E (2010) Wearable electroencephalography. IEEE Eng Med Biol Mag 29(3):44–56CrossRefGoogle Scholar
  4. 4.
    Chanel G, Kronegg J, Grandjean D, Pun T (2006) Emotion assessment: arousal evaluation using eeg and peripheral physiological signals. In: Multimedia content representation, classification and security, pp 530–537Google Scholar
  5. 5.
    Chanel G, Kierkels J J, Soleymani M, Pun T (2009) Short-term emotion assessment in a recall paradigm. Int J Hum Comput Stud 67(8):607–627CrossRefGoogle Scholar
  6. 6.
    Chanel G, Rebetez C, Bétrancourt M, Pun T (2011) Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Trans Syst Man Cybern Part A Syst Hum 41(6):1052–1063CrossRefGoogle Scholar
  7. 7.
    Chaovalitwongse W A, Pottenger R S, Wang S, Fan Y J, Iasemidis L D (2011) Pattern-and network-based classification techniques for multichannel medical data signals to improve brain diagnosis. IEEE Trans Syst Man Cybern Part A Syst Hum 41 (5):977–988CrossRefGoogle Scholar
  8. 8.
    Chapman B P, Fiscella K, Kawachi I, Duberstein P, Muennig P (2013) Emotion suppression and mortality risk over a 12-year follow-up. J Psychosom Res 75 (4):381–385CrossRefGoogle Scholar
  9. 9.
    Chung S Y, Yoon H J (2012) Affective classification using bayesian classifier and supervised learning. In: 12th international conference on control, automation and systems (ICCAS), 2012. IEEE, pp 1768–1771Google Scholar
  10. 10.
    Colwell K, Ryan D, Throckmorton C, Sellers E, Collins L (2014) Channel selection methods for the p300 speller. J Neurosci Methods 232:6–15CrossRefGoogle Scholar
  11. 11.
    Dai Y, Wang X, Li X, Tan Y (2015) Sparse eeg compressive sensing for web-enabled person identification. Measurement 74:11–20CrossRefGoogle Scholar
  12. 12.
    Dai Y, Wang X, Li X, Zhang P (2015) Reputation-driven multimodal emotion recognition in wearable biosensor network. In: IEEE international instrumentation and measurement technology conference (I2MTC), 2015. IEEE, pp 1747–1752Google Scholar
  13. 13.
    Iacoviello D, Petracca A, Spezialetti M, Placidi G (2015) A real-time classification algorithm for eeg-based bci driven by self-induced emotions. Comput Methods Prog Biomed 122(3):293–303CrossRefGoogle Scholar
  14. 14.
    Jenke R, Peer A, Buss M (2014) Feature extraction and selection for emotion recognition from eeg. IEEE Trans Affect Comput 5(3):327–339CrossRefGoogle Scholar
  15. 15.
    Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014:1–10Google Scholar
  16. 16.
    Kaur B, Singh D, Roy P P (2017) A novel framework of EEG-based user identification by analyzing music-listening behavior. Multimed Tools Appl 76(24):25581–25602Google Scholar
  17. 17.
    Khezri M, Firoozabadi M, Sharafat A R (2015) Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals. Comput Methods Prog Biomed 122(2):149–164CrossRefGoogle Scholar
  18. 18.
    Khosrowabadi R, Quek C, Ang K K, Wahab A (2014) Ernn: a biologically inspired feedforward neural network to discriminate emotion from eeg signal. IEEE Transactions on Neural Networks and Learning Systems 25(3):609–620CrossRefGoogle Scholar
  19. 19.
    Khushaba R N, Al-Ani A, Al-Jumaily A (2011) Feature subset selection using differential evolution and a statistical repair mechanism. Expert Syst Appl 38 (9):11,515–11,526CrossRefGoogle Scholar
  20. 20.
    Kim J C, Clements M A (2015) Multimodal affect classification at various temporal lengths. IEEE Trans Affect Comput 6(4):371–384CrossRefGoogle Scholar
  21. 21.
    Koelstra S, Muhl C, Soleymani M, Lee J S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31CrossRefGoogle Scholar
  22. 22.
    Konstantinidis E I, Frantzidis C A, Pappas C, Bamidis P D (2012) Real time emotion aware applications: a case study employing emotion evocative pictures and neuro-physiological sensing enhanced by graphic processor units. Comput Methods Prog Biomed 107(1):16–27CrossRefGoogle Scholar
  23. 23.
    Lal T N, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Scholkopf B (2004) Support vector channel selection in bci. IEEE Trans Biomed Eng 51(6):1003–1010CrossRefGoogle Scholar
  24. 24.
    Liang S F, Kuo C E, Hu Y H, Pan Y H, Wang Y H (2012) Automatic stage scoring of single-channel sleep eeg by using multiscale entropy and autoregressive models. IEEE Trans Instrum Meas 61(6):1649– 1657CrossRefGoogle Scholar
  25. 25.
    Lin C T, Lin B S, Lin F C, Chang C J (2014) Brain computer interface-based smart living environmental auto-adjustment control system in upnp home networking. IEEE Syst J 8(2):363–370CrossRefGoogle Scholar
  26. 26.
    Lin Y P, Wang C H, Jung T P, Wu T L, Jeng S K, Duann J R, Chen J H (2010) Eeg-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806CrossRefGoogle Scholar
  27. 27.
    Morris J D (1995) Observations: sam: the self-assessment manikin; an efficient cross-cultural measurement of emotional response. J Advert Res 35(6):63–68Google Scholar
  28. 28.
    Picard R W (2016) Automating the recognition of stress and emotion: from lab to real-world impact. IEEE MultiMedia 23(3):3–7CrossRefGoogle Scholar
  29. 29.
    Picot A, Charbonnier S, Caplier A (2012) On-line detection of drowsiness using brain and visual information. IEEE Trans Syst Man Cybern Part A Syst Hum 42(3):764–775CrossRefGoogle Scholar
  30. 30.
    Popescu F, Fazli S, Badower Y, Blankertz B, Müller K R (2007) Single trial classification of motor imagination using 6 dry eeg electrodes. PloS One 2(7):e637CrossRefGoogle Scholar
  31. 31.
    Qaraqe M, Ismail M, Abbasi Q, Serpedin E (2014) Channel selection and feature enhancement for improved epileptic seizure onset detector. In: EAI 4th international conference on wireless mobile communication and healthcare (Mobihealth), 2014. IEEE, pp 258–262Google Scholar
  32. 32.
    Richman L S, Kubzansky L, Maselko J, Kawachi I, Choo P, Bauer M (2005) Positive emotion and health: going beyond the negative. Health Psychol 24 (4):422CrossRefGoogle Scholar
  33. 33.
    Sannelli C, Dickhaus T, Halder S, Hammer E M, Müller K R, Blankertz B (2010) On optimal channel configurations for smr-based brain–computer interfaces. Brain Topogr 23(2):186–193CrossRefGoogle Scholar
  34. 34.
    Shin D, Shin D, Shin D (2017) Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents. Multimed Tools Appl 76(9):11449–11470Google Scholar
  35. 35.
    Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55CrossRefGoogle Scholar
  36. 36.
    Soleymani M, Asghari-Esfeden S, Fu Y, Pantic M (2016) Analysis of eeg signals and facial expressions for continuous emotion detection. IEEE Trans Affect Comput 7(1):17–28CrossRefGoogle Scholar
  37. 37.
    Verma G K, Tiwary U S (2016) Affect representation and recognition in 3d continuous valence–arousal–dominance space. Multimed Tools Appl 1–25Google Scholar
  38. 38.
    Wang X, Wang S (2011) Hierarchical deployment optimization for wireless sensor networks. IEEE Trans Mob Comput 10(7):1028–1041CrossRefGoogle Scholar
  39. 39.
    Wang X, Ding L, Bi D (2010) Reputation-enabled self-modification for target sensing in wireless sensor networks. IEEE Trans Instrum Meas 59(1):171–179CrossRefGoogle Scholar
  40. 40.
    Wen W, Liu G, Cheng N, Wei J, Shangguan P, Huang W (2014) Emotion recognition based on multi-variant correlation of physiological signals. IEEE Trans Affect Comput 5(2):126–140CrossRefGoogle Scholar
  41. 41.
    Xu H, Plataniotis K N (2012) Affect recognition using eeg signal. In: IEEE 14th international workshop on multimedia signal processing (MMSP), 2012. IEEE, pp 299–304Google Scholar
  42. 42.
    Xu P, Tian Y, Chen H, Yao D (2007) Lp norm iterative sparse solution for eeg source localization. IEEE Trans Biomed Eng 54(3):400–409CrossRefGoogle Scholar
  43. 43.
    Yılmaz B, Korkmaz S, Arslan DB, Güngör E, Asyalı MH (2014) Like/dislike analysis using eeg: determination of most discriminative channels and frequencies. Comput Methods Prog Biomed 113(2):705–713CrossRefGoogle Scholar
  44. 44.
    Yin Z, Zhao M, Wang Y, Yang J, Zhang J (2017) Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Programs Biomed 140:93–110Google Scholar
  45. 45.
    Yoon H J, Chung S Y (2013) Eeg-based emotion estimation using bayesian weighted-log-posterior function and perceptron convergence algorithm. Comput Biol Med 43(12):2230–2237CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Yixiang Dai
    • 1
  • Xue Wang
    • 1
  • Pengbo Zhang
    • 1
  • Weihang Zhang
    • 1
  • Junfeng Chen
    • 1
  1. 1.State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision InstrumentTsinghua UniversityBeijingChina

Personalised recommendations