Abstract
The advent of whole-body interactive technology has increased the importance of creating systems that take into account body expressions to determine the affective state of the user. In doing so, the role played by the form and motion information needs to be understood. Neuroscience studies have shown that biological motion is recognized by separate pathways in the brain. This paper investigates the contribution of body configuration (form) in the automatic recognition of non-acted affective dynamic expressions in a video game context. Sequences of static postures are automatically extracted from motion capture data and presented to the system which is a combination of an affective posture recognition module and a sequence classification rule to finalize the affective state of each sequence. Our results show that using form information only, the system recognition reaches performances very close to the agreement between observers who viewed the affective expressions as animations containing both form and temporal information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kleinsmith, A., de Silva, P.R., Bianchi-Berthouze, N.: Cross-cultural differences in recognizing affect from body posture. Interacting with Computers 18, 1371–1389 (2006)
Bianchi-Berthouze, N., Kleinsmith, A.: A categorical approach to affective gesture recognition. Connection Science special issue on Epigenetic Robotics - Modeling Cognitive Development in Robotic Systems 15, 259–269 (2003)
Meeren, H., van Heijnsbergen, C., de Gelder, B.: Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the USA 102, 16518–16523 (2005)
Van den Stock, J., Righart, R., de Gelder, B.: Body expressions influence recognition of emotions in the face and voice. Emotion 7, 487–494 (2007)
Bernhardt, D., Robinson, P.: Detecting affect from non-stylised body motions. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 59–70. Springer, Heidelberg (2007)
Kleinsmith, A., Bianchi-Berthouze, N., Steed, A.: Automatic Recognition of Non-Acted Affective Postures. IEEE Transactions on Systems, Man, and Cybernetics Part B (2011)
Kapoor, A., Picard, R.W., Ivanov, Y.: Probabilistic combination of multiple modalities to detect interest. In: 17th Int. Conf. Pattern Recog., pp. 969–972 (2004)
Kapoor, A., Burleson, W., Picard, R.W.: Automatic prediction of frustration. Int. J. Human-Comput. Stud. 65, 724–736 (2007)
Atkinson, A.P., Dittrich, W.H., Gemmell, A.J., Young, A.W.: Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures. Cognition 104, 59–72 (2007)
Omlor, L., Giese, M.A.: Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing 70, 1938–1942 (2007)
Roether, C., Omlor, L., Christensen, A., Giese, M.A.: Critical features for the perception of emotion from gait. Journal of Vision 8, 1–32 (2009)
Hirai, M., Hiraki, K.: The relative importance of spatial versus temporal structure in the perception of biological motion: An event-related potential study. Cognition 99, B15–B29 (2006)
Peelen, M.V., Wiggett, A.J., Downing, P.E.: Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion. Neuron. 49, 815–822 (2006)
McLeod, P., Dittrich, W., Driver, J., Perret, D., Zihl, J.: Preserved and impaired detection of structure from motion by a “motion-blind” patient. Visual Cognition 3, 363–392 (1996)
Giese, M.A., Poggio, T.: Neural mechanisms for the recognition of biological movements. Neuroscience 4, 179–191 (2003)
Vaina, L.M., Lemay, M., Bienfang, D.C., Choi, A.Y., Nakayama, K.: Intact biological motion and structure from motion perception in a patient with impaired motion mechanisms: A case study. Visual Neuroscience 5, 353–369 (1990)
Lange, J., Lappe, M.: The role of spatial and temporal information in biological motion perception. Advances in Cognitive Psychology 3, 419–428 (2007)
Ashraf, A., Lucey, S., Cohn, J., Chen, T., Prkachin, K., Solomon, P.: The painful face: Pain expression recognition using active appearance models. Image and Vision Computing 27, 1788–1796 (2009)
Bianchi-Berthouze, N., Kim, W.W., Patel, D.: Does body movement engage you more in digital game play? and why? In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 102–113. Springer, Heidelberg (2007)
Pasch, M., Bianchi-Berthouze, N., van Dijk, B., Nijholt, A.: Movement-based sports video games: Investigating motivation and gaming experience. Ent. Comp. 9, 169–180 (2009)
Ravaja, N., Saari, T., Turpeinen, M., Laarni, J., Salminen, M., Kivikangas, M.: Spatial presence and emotions during video game playing: Does it matter with whom you play? In: 8th Int. Workshop on Presence, London, pp. 32–333 (2005)
Russell, J.A., Feldman-Barrett, L.: Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. J. Pers. Social Psychol. 76, 805–819 (1999)
Afzal, S., Robinson, P.: Natural affect data – collection and annotation in a learning context, vol. 3, pp. 22–28 (2009) ISBN: 978-1-4419-9625-1
Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., Volpe, G.: Multimodal analysis of expressive gesture in music and dance performances. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 20–39. Springer, Heidelberg (2004)
Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., Driessen, P.F.: Gesture-based affective computing on motion capture data. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 1–7. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kleinsmith, A., Bianchi-Berthouze, N. (2011). Form as a Cue in the Automatic Recognition of Non-acted Affective Body Expressions. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)