Affective rating ranking based on face images in arousal-valence dimensional space
In dimensional affect recognition, the machine learning methods, which are used to model and predict affect, are mostly classification and regression. However, the annotation in the dimensional affect space usually takes the form of a continuous real value which has an ordinal property. The aforementioned methods do not focus on taking advantage of this important information. Therefore, we propose an affective rating ranking framework for affect recognition based on face images in the valence and arousal dimensional space. Our approach can appropriately use the ordinal information among affective ratings which are generated by discretizing continuous annotations. Specifically, we first train a series of basic cost-sensitive binary classifiers, each of which uses all samples relabeled according to the comparison results between corresponding ratings and a given rank of a binary classifier. We obtain the final affective ratings by aggregating the outputs of binary classifiers. By comparing the experimental results with the baseline and deep learning based classification and regression methods on the benchmarking database of the AVEC 2015 Challenge and the selected subset of SEMAINE database, we find that our ordinal ranking method is effective in both arousal and valence dimensions.
Key wordsOrdinal ranking Dimensional affect recognition Valence Arousal Facial image processing
Unable to display preview. Download preview PDF.
- Baron-Cohen S, 2004. Mind Reading: the Interactive Guide to Emotions. Jessica Kingsley Publishers.Google Scholar
- Li L, Lin H, 2006. Ordinal regression by extended binary classification. Advances in Neural Information Processing Systems, p.865–872.Google Scholar
- Scherer K, 2000. Psychological models of emotion. In: Borod J(Ed.), The Neuropsychology of Emotion. Oxford University Press, New York, USA.Google Scholar
- Scherer K, Schorr A, Johnstone T, 2001. Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press, New York, USA.Google Scholar
- Wöllmer M, Eyben F, Reiter S, et al., 2008. Abandoning emotion classes—towards continuous emotion recognition with modelling of long-range dependencies. Interspeech, p.597–600.Google Scholar
- Wöllmer M, Metallinou A, Eyben F, et al., 2010a. Contextsensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling. Interspeech, p.2362–2365.Google Scholar
- Yu C, Aoki P, Woodruff A, 2004. Detecting user engagement in everyday conversations. 8th Int Conf on Spoken Language Processing, p.1329–1332.Google Scholar