Towards Micro-expression Recognition Through Pyramid of Uniform Temporal Local Binary Pattern Features

  • Taoufik Ben AbdallahEmail author
  • Radhouane Guermazi
  • Mohamed Hammami
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 940)


Compared to macro-expressions, recognizing micro-expres-sions is more challenging due to low intensity and their brief duration. To deal with this issue, the present paper proposes a facial micro-expression recognition approach based on the pyramid of uniform Temporal Local Binary Pattern (PTLBP\(^{u2}\)) features for describing the appearance motion changes in time through video stream. Unlike the majority of approaches that use a high dimensional feature space, the proposed approach is based on a low dimensional space with only 83 features. Compared to the most recent facial micro-expression recognition approaches, our approach proves its effectiveness with an accuracy rate reaching 66.40% on Casme II dataset. A study of the ability of a macro-expression model to recognize micro-expression shows that it is more efficient to recognize certain micro-expressions than others.


Micro-expressions PTLBP\(^{u2}\) Pyramid representation Low-dimensional feature space Random Forests (RF) Macro-expressions 


  1. 1.
    Abdallah, T.B., Guermazi, R., Hammami, M.: Facial-expression recognition based on a low-dimensional temporal feature space. Multimedia Tools Appl. 77(15), 19455–19479 (2018)CrossRefGoogle Scholar
  2. 2.
    Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdisc. Rev. Comput. Stat. 2(4), 443–459 (2010)CrossRefGoogle Scholar
  3. 3.
    Beauchemin, S.S., Barron, J.L.: The computation of optical flow. J. ACM Comput. Surv. 27(3), 433–466 (1995)CrossRefGoogle Scholar
  4. 4.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996). Scholar
  5. 5.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  6. 6.
    Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)CrossRefGoogle Scholar
  7. 7.
    Denisko, D., Hoffman, M.M.: Classification and interaction in random forests. Proc. Nat. Acad. Sci. 115(8), 1690–1692 (2018)CrossRefGoogle Scholar
  8. 8.
    Duan, X., Dai, Q., Wang, X., Wang, Y., Hua, Z.: Recognizing spontaneous micro-expression from eye region. Neurocomputing 217, 27–36 (2016). sI: ALLSHCCrossRefGoogle Scholar
  9. 9.
    Ekman, P.: Telling Lies – Clues to Deceit in the Marketplace, Politics and Marriage 3e (2009)Google Scholar
  10. 10.
    Goshtasby, A.: Image registration by local approximation methods. Image Vis. Comput. 6(4), 255–261 (1988)CrossRefGoogle Scholar
  11. 11.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)CrossRefGoogle Scholar
  12. 12.
    Huang, X., Wang, S.J., Zhao, G., Piteikainen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), ICCVW 2015, pp. 1–9. IEEE Computer Society, Washington, DC (2015)Google Scholar
  13. 13.
    Huang, X., Wang, S., Liu, X., Zhao, G., Feng, X., Pietikäinen, M.: Spontaneous facial micro-expression recognition using discriminative spatiotemporal local binary pattern with an improved integral projection. CoRR abs/1608.02255 (2016).
  14. 14.
    Huang, X., Zhao, G., Hong, X., Zheng, W., Pietikäinen, M.: Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomput. 175(PA), 564–578 (2016)CrossRefGoogle Scholar
  15. 15.
    IMOTIONS - BIOMETRIC RESEARCH PLATFORM: Facial expression analysis: the complete pocket guide (2016).
  16. 16.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)Google Scholar
  17. 17.
    Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput. 13(3), 637–649 (2001)CrossRefGoogle Scholar
  18. 18.
    Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Advances in Neural Information Processing Systems, Cambridge, MA, USA, pp. 777–784 (2004)Google Scholar
  19. 19.
    Liong, S., See, J., Phan, R.C., Wong, K.: Less is more: micro-expression recognition from video using apex frame. J. Sig. Process. Image Commun. 62, 82–92 (2018)CrossRefGoogle Scholar
  20. 20.
    Liu, Y.J., Zhang, J.K., Yan, W.J., Wang, S.J., Zhao, G., Fu, X.: A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans. Affect. Comput. 7(4), 299–310 (2015)CrossRefGoogle Scholar
  21. 21.
    Lu, H., Kpalma, K., Ronsin, J.: Motion descriptors for micro-expression recognition. Sig. Process. Image Commun. 67, 108–117 (2018)CrossRefGoogle Scholar
  22. 22.
    Lu, Z., Luo, Z., Zheng, H., Chen, J., Li, W.: A delaunay-based temporal coding model for micro-expression recognition. In: Jawahar, C., Shan, S. (eds.) Computer Vision - ACCV 2014 Workshops, pp. 698–711. Springer International Publishing, Cham (2015)Google Scholar
  23. 23.
    Oh, Y.H., See, J., Le Ngo, A.C., Phan, R.C.W., Baskaran, V.M.: A survey of automatic facial micro-expression analysis: databases, methods, and challenges. Front. Psychol. 9, 11–28 (2018)CrossRefGoogle Scholar
  24. 24.
    O’Sullivan, M., Frank, M.G., Hurley, C.M., Tiwana, J.: Police lie detection accuracy: the effect of lie scenario. J. Law Hum. Behav. 33(6) (2009)Google Scholar
  25. 25.
    Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: Proceedings of the 13th ACM International Conference on Multimedia (2005)Google Scholar
  26. 26.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Vector Learning. MIT Press (1998)Google Scholar
  27. 27.
    Ruiz-Hernandez, J.A., Pietikäinen, M.: Encoding local binary patterns using the re-parametrization of the second order Gaussian jet. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6 (2013)Google Scholar
  28. 28.
    Verikas, A., Gelzinis, A., Bacauskiene, M.: Mining data with random forests: a survey and results of new tests. J. Pattern Recogn. 44(2), 330–49 (2011)CrossRefGoogle Scholar
  29. 29.
    Wang, S.J., Yan, W.J., Li, X., Zhao, G., Fu, X.: Micro-expression recognition using dynamic textures on tensor independent color space. In: 2014 22nd International Conference on Pattern Recognition, pp. 4678–4683 (2014)Google Scholar
  30. 30.
    Wang, S.J., Chen, H.L., Yan, W.J., Chen, Y.H., Fu, X.: Face recognition and micro-expression recognition based on discriminant tensor subspace analysis plus extreme learning machine. Neural Process. Lett. 39(1), 25–43 (2014)CrossRefGoogle Scholar
  31. 31.
    Wang, Y., See, J., Oh, Y.H., Phan, R.C.W., Rahulamathavan, Y., Ling, H.C., Tan, S.W., Li, X.: Effective recognition of facial micro-expressions with video motion magnification. Multimedia Tools Appl. 76(20), 21665–21690 (2017)CrossRefGoogle Scholar
  32. 32.
    Wang, Y., See, J., Phan, R.C.W., Oh, Y.H.: LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition. In: Cremers, D., Reid, I., Saito, H., Yang, M.H. (eds.) Computer Vision - ACCV 2014, pp. 525–537. Springer International Publishing, Cham (2015)Google Scholar
  33. 33.
    Wolf, L.: Face recognition, geometric vs. appearance-based, pp. 347–352. Springer, Boston (2009)Google Scholar
  34. 34.
    Xu, F., Zhang, J., Wang, J.Z.: Microexpression identification and categorization using a facial dynamics map. IEEE Trans. Affect. Comput. 8(2), 254–267 (2017)CrossRefGoogle Scholar
  35. 35.
    Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., Fu, X.: CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLOS ONE 9(1), 1–8 (2014)Google Scholar
  36. 36.
    Zhang, P., Ben, X., Yan, R., Wu, C., Guo, C.: Micro-expression recognition system. Optik Int. J. Light Electron Opt. 127(3), 1395–1400 (2016)CrossRefGoogle Scholar
  37. 37.
    Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRefGoogle Scholar
  38. 38.
    Zheng, H.: Micro-expression recognition based on 2D Gabor filter and sparse representation. J. Phys. Conf. Ser. 787(1) (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Taoufik Ben Abdallah
    • 1
    Email author
  • Radhouane Guermazi
    • 2
  • Mohamed Hammami
    • 3
  1. 1.Faculty of Economics and Management, MIR@CLUniversity of SfaxSfaxTunisia
  2. 2.Saudi Electronic UniversityRiyadhKingdom of Saudi Arabia
  3. 3.Faculty of Sciences, MIR@CLUniversity of SfaxSfaxTunisia

Personalised recommendations