Facial Expression Recognition Using Directional Gradient Local Ternary Patterns

  • Nahla Nour
  • Serestina ViririEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11909)


Extraction of human emotions from facial expression has attracted significant attention in computer vision community. There are several appearance based techniques like local binary patterns (LBP), local directional patterns (LDP), local ternary patterns (LTP) and gradient local ternary patterns (GLTP). Recently, many investigations have been done to improve these feature extraction techniques. Although GLTP has achieved an improvement in robustness to noise and illumination, it encodes image gradient in four directions and two orientations only. This paper proposes to improve GLTP to directional gradient local ternary patterns (DGLTP) by encoding image gradient on eight directions and four orientations. The eight directional Kirsch mask is used to encode the image gradient followed by dimensionality reduction using linear discriminant analysis (LDA) and AVG, MAX and MIN pooling techniques are compared for fusing facial expression features. The proposed technique was experimented on JAFFE facial expression dataset with support vector machine (SVM). The experimental results show that proposed technique improved accuracy of GLTP.


Local binary pattern Local directional pattern Local ternary pattern Gradient local ternary pattern Directional gradient local ternary pattern 


  1. 1.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003)zbMATHCrossRefGoogle Scholar
  2. 2.
    Tan, D., Nijholt, A.: Brain-computer interfaces and human-computer interaction. In: Tan, D., Nijholt, A. (eds.) Brain-Computer Interfaces, pp. 3–19. Springer, London (2010). Scholar
  3. 3.
    Zavaschi, T.H., Britto Jr., A.S., Oliveira, L.E., Koerich, A.L.: Fusion of feature sets and classifiers for facial expression recognition. Expert Syst. Appl. 40(2), 646–655 (2013)CrossRefGoogle Scholar
  4. 4.
    Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)Google Scholar
  5. 5.
    Hjelmås, E.: Face detection: a survey. Comput. Vis. Image Underst. 83, 236–274 (2001)zbMATHCrossRefGoogle Scholar
  6. 6.
    Sung, K., Poggio, T.: Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)CrossRefGoogle Scholar
  7. 7.
    Viola, P., Jones, M.: Robust real-time object detection. In: International Workshop on Statistical and Computational Theories of Vision Modeling, Learning, Computing, and Sampling (2001) Google Scholar
  8. 8.
    Li, S., Gu, L.: Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition. In: IEEE Conference on Computer Vision and Pattern Recognition Demo Summary (2001)Google Scholar
  9. 9.
    Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proceedings IEEE Conference Computer Vision and Pattern Recognition, pp. 84–91 (1994)Google Scholar
  10. 10.
    Heisele, B., Serre, T., Pontil, M., Poggio, T.: Component-based face detection. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2001)Google Scholar
  11. 11.
    Rowley, H., Baluja, S., Kanade, T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 23–38 (1998). Scholar
  12. 12.
    Yang, M.H., Kriegman, D.J., Ahuja, N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)CrossRefGoogle Scholar
  13. 13.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)CrossRefGoogle Scholar
  14. 14.
    You, H., Rumbe, G.: Comparative study of classification techniques on breast cancer FNA biopsy data. Int. J. Artif. Intell. Interact. Multimedia (2010)Google Scholar
  15. 15.
    Barrena, J.T., Valls, D.P.: Tumor Mass Detection through Gabor Filters and Supervised Pixel-Based Classification in Breast Cancer. University Rovira, Virgil (2014)Google Scholar
  16. 16.
    Refaeilzadeh, P., Tang, L., Liu, H.: Cross-validation. In: Liu, L., Özsu, M.T. (eds.) Encyclopedia of Database Systems, pp. 532–538. Springer, Boston (2009). Scholar
  17. 17.
    Mitchell, T.M.: Machine learning. In: WCB. McGraw-Hill, Boston (1997)Google Scholar
  18. 18.
    Vaidehi, S., Vasuhi, V., et al.: Person authentication using face detection. In: Proceedings of the World Congress on Engineering and Computer Science, pp. 222–224 (2008)Google Scholar
  19. 19.
    Reis, H.T., Andrew Collins, W., Berscheid, E.: The relationship context of human behavior and development. Psychol. Bull. 126(6), 844 (2000)CrossRefGoogle Scholar
  20. 20.
    Suja, P., Thomas, S.M., Tripathi, S., Madan, V.K.: Emotion recognition from images under varying illumination conditions. In: Balas, V.E., Jain, L.C., Kovačević, B. (eds.) Soft Computing Applications. AISC, vol. 357, pp. 913–921. Springer, Cham (2016). Scholar
  21. 21.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 Faces in-the-Wild Challenge: The first facial landmark localization Challenge. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW), IEEE, Sydney (2013)Google Scholar
  22. 22.
    Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local binary patterns and its application to facial image analysis: a survey. IEEE Trans. Syst. Man Cybernet. Part C Appl. Rev. 41(6), 765–781 (2011)CrossRefGoogle Scholar
  23. 23.
    Jabid, T., Kabir, M.H., Chae, O.: Robust facial expression recognition based on local directional pattern. ETRI J. 32(5), 784–794 (2010)CrossRefGoogle Scholar
  24. 24.
    Tan, X., Triggs, B.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans. Image Process. 19(6), 1635–1650 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Bashyal, S., Venayagamoorthy, G.K.: Recognition of facial expressions using Gabor wavelets and learning vector quantization. Eng. Appl. Artif. Intell. 21(7), 1056–1064 (2008)CrossRefGoogle Scholar
  26. 26.
    Singh, S.K., Chauhan, D.S., Vatsa, M., Singh, R.: A robust skin color based face detection algorithm. Tamkang J. Sci. Eng. 6(4), 227–234 (2003)Google Scholar
  27. 27.
    Punitha, A., Kalaiselvigeetha, M.: Texture based emotion recognition from facial expression using support vector machine. Int. J. Comput. Appl. (0975–8887) 80(5) (2013) CrossRefGoogle Scholar
  28. 28.
    Ahmed, F., Hossain, E.: Automated facial expression recognition using gradient-based ternary texture patterns. Chin. J. Eng. 2013, 1–8 (2013)CrossRefGoogle Scholar
  29. 29.
    Yow, K.C., Cipolla, R.: Feature based human face detection. Image Vis. Comput. 15(9), 713–735 (1997)CrossRefGoogle Scholar
  30. 30.
    Hsu, C.-W., Lin, C.-J.: A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Networks 13(2), 415–425 (2002)CrossRefGoogle Scholar
  31. 31.
    Scharr, H.: Optimal operators in digital image processing. Ph.D. thesis (2000)Google Scholar
  32. 32.
    Dhall, A., Goecke, R., Lucey, S., Gedeon, T.: Static facial expression analysis in tough conditions: data, evaluation protocol and benchmark. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 2106–2112. IEEE, November 2011Google Scholar
  33. 33.
    Shrivastava, K., Manda, S., Chavan, P.S., Patil, T.B., Sawant-Patil, S.T.: Conceptual model for proficient automated attendance system based on face recognition and gender classification using haar-cascade, LBPH algorithm along with LDA model. Int. J. Appl. Eng. Res. 13(10), 8075–8080 (2018)Google Scholar
  34. 34.
    Padilla, R., Costa Filho, C.F.F., Costa, M.G.F.: Evaluation of Haar cascade classifiers designed for face detection. World Acad. Sci. Eng. Technol. 64, 362–365 (2012)Google Scholar
  35. 35.
    Ahmed, F., Kabir, M.H.: Directional ternary pattern (DTP) for facial expression recognition. In: 2012 IEEE International Conference on Consumer Electronics (ICCE). IEEE, Las Vegas (2012)Google Scholar
  36. 36.
    Holder, R.P., Tapamo, J.R.: Improved gradient local ternary patterns for facial expression recognition. EURASIP J. Image Video Process. 2017(1), 42 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.College of Computer Science and Information TechnologySudan University of Science and TechnologyKhartoumSudan
  2. 2.School of Mathematics, Statistics and Computer ScienceUniversity of KwaZulu-NatalDurbanSouth Africa

Personalised recommendations