Abstract
This paper presents research using full body skeletal movements captured using video-based sensor technology developed by Vicon Motion Systems, to train a machine to identify different human emotions. The Vicon system uses a series of 6 cameras to capture lightweight markers placed on various points of the body in 3D space, and digitizes movement into x, y, and z displacement data. Gestural data from five subjects was collected depicting four emotions: sadness, joy, anger, and fear. Experimental results with different machine learning techniques show that automatic classification of this data ranges from 84% to 92% depending on how it is calculated. In order to put these automatic classification results into perspective a user study on the human perception of the same data was conducted with average classification accuracy of 93%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kapoor, S.M., Picard, R.W.: Towards a Learning Companion that Recognizes Affect. In: Proc. Emotional and Intelligent II: The Tangled Knot of Social Cognition, AAAI Fall Symposium, North Falmouth, MA (November 2001)
Fernandez, R., Picard, R.W.: Modeling Driver’s Speech under Stress. In: Proc. ISCA Workshop on Speech and Emotions, Belfast (2000)
Pantic, M.: Toward an Affect-Sensitve Multimodal Human-Computer Interaction. In: Proc of the IEEE 91(9) (September2003)
Kang, B.S., Han, C.H., Lee, S.T., Youn, D.H., Lee, C.: Speaker dependent emotion recognition using speech signals. In: Proc ICSLP, pp. 383–386 (2000)
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.G.: Emotion recognition in human-computer interaction. IEEE Signal Processing Mag. 18, 32–80 (2001)
Ververidis, D., Kotropoulos, C., Pitas, I.: Automatic Emotional Speech Classification. In: Proc ICASSP, pp. 593–596 (2004)
Schiano, D.J., Ehrlich, S.M., Rahardja, K., Sheridan, K.: Face to interface: Facial affect in human and machine. In: Proc. CHI, pp. 193–200 (2000)
Essa, I., Pentland, A.: Coding analysis interpretation recognition of facial expressions. IEEE Trans. Pattern Anal. Machine Intell. 19, 757–763 (1997)
Blackand, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Conmput. Vis. 25(1), 23–48 (1997)
Chen, L.S., Huang, T.S., Miyasato, T., Nakatsu, R.: Multimodal Human Emotion/Expression Recognition. In: Proc Third International Conference on Automatic Face and Gesture Recognition, Nara, Japan (1998)
De Silva, L.C., Miyasato, T., Nakatsu, R.: Facial emotion recognition using multimodal information. In: Proc FG, pp. 332–335 (2000)
Yoshitomi, Y., Kim, S., Kawano, T., Kitazoe, T.: Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proc. ROMAN, pp. 178–183 (2000)
Picard, R.W.: Towards Computers that Recognize and Respond to User Emotions. IBM System Journal 39, 705–719 (2001)
Picard, R.W., Healey, J.: Affective Wearables. Personal Technologies 1(4), 231–240 (1997)
Pollick, F.E., Paterson, H., Bruderlin, A., Sanford, A.J.: Perceiving affect from arm movement. Cognition 82, B51–B61 (2001)
Vines, M., Wanderley, M., Krumhansl, C., Nuzzo, R., Levirin, D.: Performance Gestures of Musicians: What Structural and Emotional Information do they Convey? In: Gesture-Based Communication in Human-Computer Interaction - 5th International Gesture Workshop, Genova, Italy (2003)
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)
DeMeijer, M.: The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13, 247–268 (1989)
Woolard, A.: Vicon 512 User Manual, Vicon Motion Systems, Tustin CA (January 1999)
Ian, H., Frank, E., Kaufmann, M.: Data Mining: Practical machine learning tools with Java implementations, San Francisco (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., Driessen, P.F. (2005). Gesture-Based Affective Computing on Motion Capture Data. In: Tao, J., Tan, T., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, vol 3784. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573548_1
Download citation
DOI: https://doi.org/10.1007/11573548_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29621-8
Online ISBN: 978-3-540-32273-3
eBook Packages: Computer ScienceComputer Science (R0)