Abstract
The capability of perceiving and expressing emotions through different modalities is a key issue for the enhancement of human-agent interaction. In this paper, an architecture for the development of intelligent multimodal affective interfaces is presented. It is based on the integration of Sentic Computing, a new opinion mining and sentiment analysis paradigm based on AI and Semantic Web techniques, with a facial emotional classifier and Maxine, a powerful multimodal animation engine for managing virtual agents and 3D scenarios. One of the main distinguishing features of the system is that it does not simply perform emotional classification in terms of a set of discrete emotional labels but it operates in a novel continuous 2D emotional space, enabling the output of a continuous emotional path that characterizes user’s affective progress over time. Another key factor is the fusion methodology proposed, which is able to fuse any number of unimodal categorical modules, with very different time-scales, output labels and recognition success rates, in a simple and scalable way.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Casell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.): Embodied Conversational Agents. MIT Press, Cambridge (2000)
Isbister, K.: Better game characters by design: A psychological approach, p. 83. Morgan Kaufmann (2006)
Yee, N., Bailenson, J., Urbanek, M., Chang, F., Merget, D.: The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. Cyber Psychology & Behavior 10(1), 115–121 (2007)
Ochs, M., Pelachaud, C., Sadek, D.: An empathic virtual dialog agent to improve human-machine interaction. In: International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 89–96 (2008)
Boukricha, H., Becker, C., Wachsmuth, I.: Simulating empathy for the virtual human Max. In: International Workshop on Emotion and Computing in Conjunction With the German Conference on Artificial Intelligence, pp. 22–27 (2007)
Gilroy, S., Cavazza, M., Niiranen, M., Andre, E., Vogt, T., Urbain, J., Benayoun, M., Seichter, H., Billinghurst, M.: PAD-based multimodal affective fusion. In: ACII, pp. 1–8, 10–12 (2009)
Zeng, Z., Pantic, M., Huang, T.: Emotion Recognition Based on Multimodal Information. In: Affective Information Processing, pp. 241–265 (2009)
Kapoor, A., Burleson, W., Picard, R.: Automatic prediction of frustration. International Journal of Human-Computer Studies 65, 724–736 (2007)
Gunes, H., Piccardi, M., Pantic, M.: From the lab to the real world: Affect recognition using multiple cues and modalities. In: Affective Computing: Focus on Emotion Expression, Synthesis, and Recognition, pp. 185–218 (2008)
Cambria, E., Olsher, D., Kwok, K.: Sentic Activation: A Two-Level Affective Common Sense Reasoning Framework. In: AAAI, Toronto (2012)
Hupont, I., Baldassarri, S., Cerezo, E.: Sensing facial emotions in a continuos 2D affective space. In: IEEE International Conference on System, Man and Cybernetics (2010)
Baldassarri, S., Cerezo, E., Seron, F.: Maxine: a platform for embodied Animated Agents. Computers & Graphics 32(4), 430–437 (2008)
Cambria, E., Hussain, A.: Sentic Computing: Techniques, Tools, and Applications. Springer, Heidelberg (2012)
Cambria, E., Benson, T., Eckl, C., Hussain, A.: Sentic PROMs: Application of sentic computing to the development of a novel unified framework for measuring health-care quality. Expert Systems with Applications 39(12), 10533–10543 (2012)
Cambria, E., Livingstone, A., Hussain, A.: The hourglass of emotions. In: Esposito, A., Vinciarelli, A., Hoffmann, R., Muller, V. (eds.) Cognitive Behavioral Systems. LNCS. Springer, Heidelberg (2012)
Whissell, C.: The dictionary of affect in language. In: Emotion: Theory, Research and Experience, The Measurement of Emotions, vol. 4. Academic, New York (1989)
Picard, R.: Affective Computing. The MIT Press (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hupont, I., Cambria, E., Cerezo, E., Hussain, A., Baldassarri, S. (2012). Sentic Maxine: Multimodal Affective Fusion and Emotional Paths. In: Wang, J., Yen, G.G., Polycarpou, M.M. (eds) Advances in Neural Networks – ISNN 2012. ISNN 2012. Lecture Notes in Computer Science, vol 7368. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31362-2_61
Download citation
DOI: https://doi.org/10.1007/978-3-642-31362-2_61
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31361-5
Online ISBN: 978-3-642-31362-2
eBook Packages: Computer ScienceComputer Science (R0)