Abstract
Emotions that arise in viewers in response to videos play an essential role in content-based indexing and retrieval. However, the emotional gap between low-level features and high-level semantic meanings is not well understood. This paper proposes a general scheme for video emotion identification using mutual information-based feature selection followed by regression. Continuous arousal and valence values are used to measure video affective content in dimensional arousal-valence space. Firstly, rich audio-visual features are extracted from video clips. The minimum redundancy and maximum relevance feature selection is then used to select most representative feature subsets for arousal and valence modelling. Finally support vector regression is employed to model arousal and valence estimation functions. As evaluated via tenfold cross-validation, the estimation results achieved by our scheme for arousal and valence are: mean absolute error, 0.1358 and 0.1479, variance of absolute error, 0.1074 and 0.1175, respectively. Encouraging results demonstrate the effectiveness of our proposed method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hanjalic, A., Xu, L.Q.: Affective video content representation and modeling. IEEE Transactions on Multimedia, vol. 7, pp. 143–154 (2005)
Kang, H.B.: Affective content detection using HMMs. Proceedings of the 11th ACM International Conference on Multimedia (MM), Berkeley, California, USA, pp. 259–262 (2003)
Xu, M., Jin, J.S., Luo, S., Duan, L.: Hierarchical movie affective content analysis based on arousal and valence features. Proceeding of the 16th ACM International Conference on Multimedia (MM), Vancouver, Canada, pp. 677–680 (2008)
Xu, M., Chia, L.T., Jin, J.: Affective content analysis in comedy and horror videos by audio emotional event detection. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 621–625. IEEE Press, New York (2005)
Zhang, S., Huang, Q., Jiang, S., Gao, W., Tian, Q.: Affective visualization and retrieval for music video. IEEE Transactions on Multimedia, vol. 12, pp. 510–522 (2010)
Zhang, S.L., Tian, Q., Huang, Q.M., Gao, W., Li, S.P.: Utilizing Affective Analysis for Efficient Movie Browsing. 16th IEEE International Conference on Image Processing, Vols 1-6, pp. 1833–1836 (2009)
Arifin, S., Cheung, P.Y.K.: Affective level video segmentation by utilizing the pleasure-arousal-dominance information. IEEE Transactions on Multimedia, vol. 10, pp. 1325–1341 (2008)
Zhang, S., Xu, Y.J., Jia, J., Cai, L.H.: Analysis and Modeling of Affective Audio Visual Speech Based on PAD Emotion Space. 6th International Symposium on Chinese Spoken Language Processing, pp. 281–284 (2008)
Li, D.G., Sethi, I.K., Dimitrova, N., McGee, T.: Classification of general audio data for content-based retrieval. Pattern Recognition Letters 22, pp. 533–544 (2001)
Lu, L., Liu, D., Zhang, H.J.: Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio Speech and Language Processing 14, pp. 5–18 (2006)
Lu, L., Zhang, H.J., Li, S.Z.: Content-based audio classification and segmentation by using support vector machines. Multimedia Systems 8, pp. 482–491 (2003)
Valdez, P., Mehrabian, A.: Effects of Color on Emotions. Journal of Experimental Psychology-General 123, pp. 394–409 (1994)
Wang, H.L., Cheong, L.F.: Affective understanding in film. IEEE Transactions on Circuits and Systems for Video Technology 16, pp. 689–704 (2006)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, pp. 1157–1182 (2003)
Ding, C., Peng, H.: Minimum redundancy feature selection from microarray gene expression data. Journal of Bioinformatics and Computational Biology 3, pp. 185–205 (2005)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, pp. 1226–1238 (2005)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:21–27:27 (2011)
Hsu, C.-W., Chang, C.-C., Lin, C.-J.: A Practical Guide to Support Vector Classification. Department of Computer Science and Information Engineering, National Taiwan University, Taipei (2003)
Cui, Y., Lin, J.S., Zhang, S.L., Luo, S.H., Tian, Q.: Correlation-Based Feature Selection and Regression. Advances in Multimedia Information Processing-PCM, Pt I 6297, pp. 25–35 (2010)
Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.M., Malave, V.L., Mason, R.A., Just, M.A.: Predicting human brain activity associated with the meanings of nouns. Science 320, pp. 1191–1195 (2008)
Hu, X., Deng, F., Li, K., Zhang, T., Chen, H., Jiang, X., Lv, J., Zhu, D., Faraco, C., Zhang, D., Mesbah, A., Han, J., Hua, X., Xie, L., Miller, S., Lei, G., Liu, T.: Bridging low-level features and high-level semantics via fMRI brain imaging for video classification. Proceedings of the International Conference on Multimedia (MM), Firenze, Italy, pp. 451–460 (2010)
Acknowledgements
This work was supported by a CSC-Newcastle scholarship.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media, LLC
About this paper
Cite this paper
Cui, Y. et al. (2013). Mutual Information-Based Emotion Recognition. In: The Era of Interactive Media. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3501-3_39
Download citation
DOI: https://doi.org/10.1007/978-1-4614-3501-3_39
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-3500-6
Online ISBN: 978-1-4614-3501-3
eBook Packages: Computer ScienceComputer Science (R0)