Improving Micro-expression Recognition Accuracy Using Twofold Feature Extraction
Micro-expressions are generated involuntarily on a person’s face and are usually a manifestation of repressed feelings of the person. Micro-expressions are characterised by short duration, involuntariness and low intensity. Because of these characteristics, micro-expressions are difficult to perceive and interpret correctly, and they are profoundly challenging to identify and categorise automatically.
Previous work for micro-expression recognition has used hand-crafted features like LBP-TOP, Gabor filter, HOG and optical flow. Recent work also has demonstrated the possible use of deep learning for micro-expression recognition. This paper is the first work to explore the use of hand-craft feature descriptor and deep feature descriptor for micro-expression recognition task. The aim is to use the hand-craft and deep learning feature descriptor to extract features and integrate them together to construct a large feature vector to describe a video. Through experiments on CASME, CASME II and CASME+2 databases, we demonstrate our proposed method can achieve promising results for micro-expression recognition accuracy with larger training samples.
KeywordsMicro-expression recognition Deep learning Local binary pattern-three orthogonal planes (LBP-TOP) Convolutional neural network (CNN) Small training data Data augmentation
- 3.Happy, S., Routray, A.: Fuzzy histogram of optical flow orientations for micro-expression recognition. IEEE Trans. Affect. Comput. (2017)Google Scholar
- 4.Huang, X., Wang, S.J., Zhao, G., Piteikainen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)Google Scholar
- 5.Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
- 6.Kim, D.H., Baddar, W.J., Ro, Y.M.: Micro-expression recognition with expression-state constrained spatio-temporal feature representations, pp. 382–386. ACM (2016)Google Scholar
- 7.Li, X., et al.: Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans. Affect. Comput. (2017)Google Scholar
- 9.Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. In: BMVC, vol. 1 (2015)Google Scholar
- 10.Patel, D., Hong, X., Zhao, G.: Selective deep features for micro-expression recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2258–2263. IEEE (2016)Google Scholar
- 12.Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Recognising spontaneous facial micro-expressions. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1449–1456. IEEE (2011)Google Scholar
- 14.Takalkar, M.A., Xu, M.: Image based facial micro-expression recognition using deep learning on small datasets. In: 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–7. IEEE (2017)Google Scholar
- 15.Vasconcellos, S.J.L., Salvador-Silva, R., Gauer, V., Gauer, G.J.C.: Psychopathic traits in adolescents and recognition of emotion in facial expressions. Psicologia: Reflexão e. Crítica 27(4), 768–774 (2014)Google Scholar
- 16.Wang, S.J., Yan, W.J., Li, X., Zhao, G., Fu, X.: Micro-expression recognition using dynamic textures on tensor independent color space. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp. 4678–4683. IEEE (2014)Google Scholar