Abstract
This paper introduces a distributed multiagent system architecture for multimodal emotion identification, which is based on simultaneous analysis of physiological parameters from wearable devices, human behaviors and activities, and facial micro-expressions. Wearable devices are equipped with electrodermal activity, electrocardiogram, heart rate, and skin temperature sensor agents. Facial expressions are monitored by a vision agent installed at the height of the human’s head. Also, the activity of the user is monitored by a second vision agent mounted overhead. The emotion is refined as a cooperative decision taken at a central agent node denominated “Central Emotion Detection Node” from the local decision offered by the three agent nodes called “Face Expression Analysis Node”, “Behavior Analysis Node” and “Physiological Data Analysis Node”. This way, the emotion identification results are outperformed through an intelligent fuzzy-based decision making technique.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fernández-Caballero, A., Latorre, J.M., Pastor, J.M., Fernández-Sotos, A.: Improvement of the elderly quality of life and care through smart emotion regulation. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 348–355. Springer, Heidelberg (2014)
Castillo, J.C., Fernández-Caballero, A., Castro-González, Á., Salichs, M.A., López, M.T.: A framework for recognizing and regulating emotions in the elderly. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 320–327. Springer, Heidelberg (2014)
Darwin, C.R.: The Expression of the Emotions in Man and Animals. London, John Murray (1872)
Karg, M., Samadani, A., Gorbet, R., Kuhnlenz, K., Hoey, J., Kulic, D.: Body movements for affective expression: a survey of automatic recognition and generation. IEEE Transactions on Affective Computing 4, 341–359 (2013)
Picard, R.W.: Affective computing: challenges. International Journal of Human-Computer Studies 59, 55–64 (2003)
Tao, J., Tan, T.: Affective computing: a review. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 981–995. Springer, Heidelberg (2005)
Petrushin, V.A.: Emotion recognition in speech signal: experimental study, development, and application. In: 6th International Conference of Spoken Language Processing, pp. 222–225 (2000)
Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing 42, 419–427 (2004)
Murugappan, M.: Electromyogram signal based human emotion classification using KNN and LDA. In: 2011 IEEE International Conference on System Engineering and Technology, pp. 106–110 (2011)
Bänziger, T., Grandjean, D., Scherer, K.R.: Emotion recognition from expressions in face, voice, and body: the multimodal emotion recognition test (MERT). Emotion 9, 691–704 (2009)
Picard, R.W.: Emotion research by the people, for the people. Emotion Review 2, 250–254 (2010)
Kukolja, D., Popovic, S., Horvat, M., Kovac, B., Cosic, K.: Comparative analysis of emotion estimation methods based on physiological measurements for real-time applications. International Journal of Human-Computer Studies 72, 717–727 (2014)
Felisberto, F., Fdez.-Riverola, F., Pereira, A.: A ubiquitous and low-cost solution for movement monitoring and accident detection based on sensor fusion. Sensors 14, 8961–8983 (2014)
Fernandez, R.: A Computational Model for the Automatic Recognition of Affect in Speech. Massachusetts Institute of Technology (2004)
Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: a meta-analysis. Psychological Bulletin 111, 256–274 (1992)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 39–58 (2009)
Lozano-Monasor, E., López, M.T., Fernández-Caballero, A., Vigo-Bustos, F.: Facial expression recognition from webcam based on active shape models and support vector machines. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) IWAAL 2014. LNCS, vol. 8868, pp. 147–154. Springer, Heidelberg (2014)
Ekman, P., Friesen, W.V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Tzavaras, A.: Universals and cultural differences in the judgment of facial expressions of emotion. Journal of Personality and Social Psychology 53, 712–717 (1987)
Mauss, I.B., Levenson, R.W., McCarter, L., Wilhelm, F.H., Gross, J.J.: The tie that binds? Coherence among emotion experience, behavior, and physiology. Emotion 5, 175–190 (2005)
Van Galen, G.P., Müller, M.L., Meulenbroek, R.G., Van Gemmert, A.W.: Forearm EMG response activity during motor performance in individuals prone to increased stress reactivity. American Journal of Industrial Medicine 41, 406–419 (2002)
Coombes, S.A., Cauraugh, J.H., Janelle, C.M.: Emotional state and initiating cue alter central and peripheral motor processes. Emotion 7, 275–284 (2007)
Stemmler, G.: The autonomic differentiation of emotions revisited: convergent and discriminant validation. Psychophysiology 26, 617–632 (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Sokolova, M.V., Fernández-Caballero, A., López, M.T., Martínez-Rodrigo, A., Zangróniz, R., Pastor, J.M. (2015). A Distributed Architecture for Multimodal Emotion Identification. In: Bajo, J., et al. Trends in Practical Applications of Agents, Multi-Agent Systems and Sustainability. Advances in Intelligent Systems and Computing, vol 372. Springer, Cham. https://doi.org/10.1007/978-3-319-19629-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-19629-9_14
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19628-2
Online ISBN: 978-3-319-19629-9
eBook Packages: EngineeringEngineering (R0)