Multiple-Instance Lasso Regularization via Embedded Instance Selection for Emotion Recognition

  • J. Caicedo-AcostaEmail author
  • D. Cárdenas-Peña
  • D. Collazos-Huertas
  • J. I. Padilla-Buritica
  • G. Castaño-Duque
  • G. Castellanos-Dominguez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11486)


Since emotions affect physical and psychologically the health of people, their identification is crucial for understanding human behavior. Despite the several systems developed in this regard, most of them underperform on people with disabilities, their setup is sensitive to noise or non-emotional stimuli. Recent studies consider electroencephalographic (EEG) signals for understanding emotional responses due to reflecting the activity of the central nervous system. However, the non-stationary nature of EEG signals demand elaborated signal processing approaches because not all time instants hold information related to the stimulus-response. This work proposes a temporal analysis approach, termed MILRES, based on the Multi-Instance Learning framework that includes a multiple instance Regularization with LASSO penalty and an Embedded instance Selection. We test MILRES in discriminating two states (high and low) of the valence and arousal emotional dimensions from the DEAP dataset. The proposed approach reaches \(84.4\%\) accuracy and \(79.5\%\) F1-score for valence, and \(81.9\%\) accuracy \(67.9\%\) for arousal. Such results evidence that MILRES outperforms other EEG-based emotion recognition approaches from the state-of-the-art, with the additional benefit of identifying the brain areas involved in processing emotions.


Electroencephalography Emotion recognition Multi-Instance learning Feature selection 



This work is developed within the framework of the research project “programa reconstrucción del tejido social en zonas de pos-conflicto en Colombia del proyecto Fortalecimiento docente desde la alfabetización mediática Informacional y la CTel, como estrategia didáctico-pedagógica y soporte para la recuperación de la confianza del tejido social afectado por el conflicto, Código SIGP 58950” funded by “Fondo Nacional de Financiamiento para la Ciencia, la Tecnología y la Innovación, Fondo Francisco José de Caldas con contrato No 213-2018 con Código 58960”.


  1. 1.
    Vallat-Azouvi, C., Azouvi, P., Le-Bornec, G., Brunet-Gouet, E.: Treatment of social cognition impairments in patients with traumatic brain injury: a critical review. Brain injury 33(1), 87–93 (2019)CrossRefGoogle Scholar
  2. 2.
    Frau-Meigs, D.: Media education. A kit for teachers, students parents and professionals. UNESCO (2007)Google Scholar
  3. 3.
    Picard, R.W., et al.: Affective learning-a manifesto. BT Technol. J. 22(4), 253–269 (2004)CrossRefGoogle Scholar
  4. 4.
    Wang, F., Lv, J., Ying, G., Chen, S., Zhang, C.: Facial expression recognition from image based on hybrid features understanding. J. Vis. Commun. Image Represent. 59, 84–88 (2019)CrossRefGoogle Scholar
  5. 5.
    Bourel, F., Chibelushi, C.C., Low, A.A.: Robust facial expression recognition using a state-based model of spatially-localised facial dynamics. In: Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 113–118. IEEE (2002)Google Scholar
  6. 6.
    Schuller, B., Reiter, S., Muller, R., Al-Hames, M., Lang, M., Rigoll, G.: Speaker independent speech emotion recognition by ensemble classification. In: IEEE International Conference on Multimedia and Expo, ICME 2005, pp. 864–867. IEEE (2005)Google Scholar
  7. 7.
    Purnamasari, P.D., Ratna, A.A.P., Kusumoputro, B.: Development of filtered bispectrum for EEG signal feature extraction in automatic emotion recognition using artificial neural networks. Algorithms 10(2), 63 (2017)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Nasoz, F., Lisetti, C., Alvarez, K., Finkelstein, N.: Emotion recognition from physiological signals for user modeling of affect. In: proceedings of the 9th International Conference on User Model, Pittsburg, USA, pp. 22–26 (2003)Google Scholar
  9. 9.
    Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. IEEE Trans. Inf. Technol. Biomed. 14(2), 186–197 (2010)CrossRefGoogle Scholar
  10. 10.
    Paus, T., Sipila, P., Strafella, A.: Synchronization of neuronal activity in the human primary motor cortex by transcranial magnetic stimulation: an eeg study. J. Neurophysiol. 86(4), 1983–1990 (2001)CrossRefGoogle Scholar
  11. 11.
    Sourina, O., Liu, Y.: A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: Biosignals, pp. 209–214 (2011)Google Scholar
  12. 12.
    Jie, X., Cao, R., Li, L.: Emotion recognition based on the sample entropy of EEG. Bio-med. Mater. Eng. 24(1), 1185–1192 (2014)Google Scholar
  13. 13.
    Kroupi, E., Yazdani, A., Ebrahimi, T.: EEG correlates of different emotional states elicited during watching music videos. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6975, pp. 457–466. Springer, Heidelberg (2011). Scholar
  14. 14.
    Hjorth, B.: EEG analysis based on time domain properties. Electroencephalogr. Clin. Neurophysiol. 29(3), 306–310 (1970)CrossRefGoogle Scholar
  15. 15.
    Wang, X.-W., Nie, D., Lu, B.-L.: EEG-based emotion recognition using frequency domain features and support vector machines. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011. LNCS, vol. 7062, pp. 734–743. Springer, Heidelberg (2011). Scholar
  16. 16.
    Chanel, G., Ansari-Asl, K., Pun, T.: Valence-arousal evaluation using physiological signals in an emotion recall paradigm. In: IEEE International Conference on Systems, Man and Cybernetics, ISIC, pp. 2662–2667. IEEE (2007)Google Scholar
  17. 17.
    Hadjidimitriou, S.K., Hadjileontiadis, L.J.: Toward an EEG-based recognition of music liking using time-frequency analysis. IEEE Trans. Biomed. Eng. 59(12), 3498–3510 (2012)CrossRefGoogle Scholar
  18. 18.
    Mohammadi, Z., Frounchi, J., Amiri, M.: Wavelet-based emotion recognition system using eeg signal. Neural Comput. Appl. 28(8), 1985–1990 (2017)CrossRefGoogle Scholar
  19. 19.
    Alazrai, R., Homoud, R., Alwanni, H., Daoud, M.: EEG-based emotion recognition using quadratic time-frequency distribution. Sensors 18(8), 2739 (2018)CrossRefGoogle Scholar
  20. 20.
    Kaplan, A.Y., Fingelkurts, A.A., Fingelkurts, A.A., Borisov, S.V., Darkhovsky, B.S.: Nonstationary nature of the brain activity as revealed by EEG/MEG: methodological, practical and conceptual challenges. Signal Process. 85(11), 2190–2212 (2005)CrossRefGoogle Scholar
  21. 21.
    Sanei, S., Chambers, J.: Fundamentals of EEG Signal Processing, pp. 35–125. Wiley, Hoboken (2013). Chapter 2CrossRefGoogle Scholar
  22. 22.
    Zhang, X., et al.: Emotion recognition based on electroencephalogram using a multiple instance learning framework. In: Huang, De-Shuang, Jo, Kang-Hyun, Zhang, Xiao-Long (eds.) ICIC 2018. LNCS, vol. 10955, pp. 570–578. Springer, Cham (2018). Scholar
  23. 23.
    Babenko, B., Yang, M.-H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2011)CrossRefGoogle Scholar
  24. 24.
    Zhang, C., Platt, J.C., Viola, P.A.: Multiple instance boosting for object detection. In: Advances in Neural Information Processing Systems, pp. 1417–1424 (2006)Google Scholar
  25. 25.
    Pappas, N., Popescu-Belis, A.: Explicit document modeling through weighted multiple-instance learning. J. Artif. Intell. Res. 58, 591–626 (2017)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)CrossRefGoogle Scholar
  27. 27.
    Lang, P.J.: The emotion probe: studies of motivation and attention. Am. Psychol. 50(5), 372 (1995)CrossRefGoogle Scholar
  28. 28.
    Chen, R.-B., et al.:Multiple-instance logistic regression with lasso penalty. arXiv preprint arXiv:1607.03615 (2016)
  29. 29.
    Chen, P.-Y., Chen, C.-C., Yang, C.-H., Chang, S.-M., Lee, K.-J.: milr: Multiple-instance logistic regression with lasso penalty. R J. 9(1), 446–457 (2017)CrossRefGoogle Scholar
  30. 30.
    Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1 (2010)CrossRefGoogle Scholar
  31. 31.
    Chen, Y., Bi, J., Wang, J.Z.: MILES: multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1931–1947 (2006)CrossRefGoogle Scholar
  32. 32.
    Wang, J., Zucker, J.-D.: Solving multiple-instance problem: a lazy learning approach (2000)Google Scholar
  33. 33.
    Zheng, W.-L., Zhu, J.-Y., Lu, B.-L.: Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. Affect. Comput. (2017)Google Scholar
  34. 34.
    Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H., Yan, B.: Emotion recognition from EEG signals using multidimensional information in EMD domain. BioMed Res. Int. (2017)Google Scholar
  35. 35.
    Bermpohl, F., et al.: Attentional modulation of emotional stimulus processing: an fmri study using emotional expectancy. Hum. Brain Mapp. 27(8), 662–677 (2006)CrossRefGoogle Scholar
  36. 36.
    Rämä, P., Martinkauppi, S., Linnankoski, I., Koivisto, J., Aronen, H.J., Carlson, S.: Working memory of identification of emotional vocal expressions: an fMRI study. Neuroimage 13(6), 1090–1101 (2001)CrossRefGoogle Scholar
  37. 37.
    Deppe, M., Schwindt, W., Kugel, H., Plassmann, H., Kenning, P.: Nonlinear responses within the medial prefrontal cortex reveal when specific implicit information influences economic decision making. J. Neuroimag. 15(2), 171–182 (2005)CrossRefGoogle Scholar
  38. 38.
    Pelletier, M., et al.: Separate neural circuits for primary emotions? Brain activity during self-induced sadness and happiness in professional actors. Neuroreport 14(8), 1111–1116 (2003)CrossRefGoogle Scholar
  39. 39.
    Lane, R.D., Chua, P.M., Dolan, R.J.: Common effects of emotional valence, arousal and attention on neural activation during visual processing of pictures. Neuropsychologia 37(9), 989–997 (1999)CrossRefGoogle Scholar
  40. 40.
    Gillath, O., Bunge, S.A., Shaver, P.R., Wendelken, C., Mikulincer, M.: Attachment-style differences in the ability to suppress negative thoughts: exploring the neural correlates. Neuroimage 28(4), 835–847 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • J. Caicedo-Acosta
    • 1
    Email author
  • D. Cárdenas-Peña
    • 1
  • D. Collazos-Huertas
    • 1
  • J. I. Padilla-Buritica
    • 1
  • G. Castaño-Duque
    • 2
  • G. Castellanos-Dominguez
    • 1
  1. 1.Signal Processing and Recognition GroupUniversidad Nacional de ColombiaManizalesColombia
  2. 2.Universidad Nacional de ColombiaManizalesColombia

Personalised recommendations