Posed Facial Expression Detection Using Reflection Symmetry and Structural Similarity

  • Harish BhaskarEmail author
  • Davide La Torre
  • Mohammed Al-Mualla
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9164)


In this paper, a method for the detection of posed facial expressions in still images is proposed. The method exploits a combination of geometrical deviations between sets of landmark points together with the difference in quality of visual appearance of patches around these landmark points for accurate and robust detection of posed facial expressions. First, novel descriptors are derived based on the Hausdorff distances between triangulated landmark point sets within a given image satisfying reflective symmetry constraints. Further, the structural similarity of patches around these point sets that are reflection symmetrical is calculated and fused with the geometric features for classification. Experiments using selected examples from publicly available dataset have demonstrated that the proposed method can sufficiently encapsulate the intensity of a facial expression and thus achieve superior accuracy in the separation of posed from spontaneous expressions.


Facial Expression Hausdorff Distance Image Patch Facial Asymmetry Landmark Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Brunet, D.: A Study of the Structural Similarity Image Quality Measure with Applications to Image Processing, Ph.D. thesis, University of Waterloo (2010)Google Scholar
  2. 2.
    Brunet, D., Vrscay, E.R., Wang, Z.: On the mathematical properties of the structural similarity index. IEEE Trans. Image Process. 21(4), 1488–1499 (2012)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Brunet, D., Vrscay, E.R., Wang, Z.: Structural similarity-based approximation of signals and images using orthogonal bases. In: Campilho, A., Kamel, M. (eds.) ICIAR 2010. LNCS, vol. 6111, pp. 11–22. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  4. 4.
    Wainwright, M.J., Schwartz, O., Simoncelli, E.P.: Natural image statistics and divisive normalization: modeling non-linearity and adaptation in cortical neurons. In: Rao, R., Olshausen, B., Lewicki, M. (eds.) Probabilistic Models of the Brain: Perception and Neural Function, pp. 203–222. MIT Press, Cambridge (2002)Google Scholar
  5. 5.
    Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Processing Magazine 26(1), 98–117 (2009)CrossRefGoogle Scholar
  6. 6.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  7. 7.
    Ghazel, M., Freeman, G.H., Vrscay, E.R.: Fractal image denoising. IEEE Trans. Image Process. 12, 1560–1578 (2003)CrossRefGoogle Scholar
  8. 8.
    Barnsley, M.F., Hurd, L.: Fractal Image Compression. A.K. Peters, Wellesley (1993)zbMATHGoogle Scholar
  9. 9.
    Kowalik-Urbaniak, I.A., Torre, D.L., Vrscay, E.R., Wang, Z.: Some “Weberized" L2-based methods of signal/image approximation. In: Campilho, A., Kamel, M. (eds.) ICIAR 2014, Part I. LNCS, vol. 8814, pp. 20–29. Springer, Heidelberg (2014) Google Scholar
  10. 10.
    Kunze, H., La Torre, D., Mendivil, F., Vrscay, E.R.: Fractal-Based Methods in Analysis. Springer, New York (2014)Google Scholar
  11. 11.
    Valstar, M.F., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the International Conference on Multimodal Interfaces (ICMI), vol. 3845 (2007)Google Scholar
  12. 12.
    Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1683–1699 (2007)CrossRefGoogle Scholar
  13. 13.
    Schmidt, K.L., Ambadar, Z., Cohn, J.F., Reed, L.I.: Movement differences between deliberate and spontaneous facial expressions zygomaticus major action in smiling. J. Nonverbal Behav. 30, 3752 (2006)CrossRefGoogle Scholar
  14. 14.
    Dibeklioğlu, H., Salah, A.A., Gevers, T.: Are you really smiling at me? spontaneous versus posed enjoyment smiles. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part III. LNCS, vol. 7574, pp. 525–538. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  15. 15.
    Bhaskar, H., Al-Mualla, M.: Spontaneous vs. posed facial expression analysis using deformable feature models and aggregated classifiers. In: Proceedings of the International Conference on Information FUSION (2013)Google Scholar
  16. 16.
    Valstar, M.F., Pantic, M.: Automatic analysis of brow actions. In: Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI), pp. 162–170 (2006)Google Scholar
  17. 17.
    Tresadern, P.A., Bhaskar, H., Adeshina, S.A., Taylor, J.C., Cootes, T.F.: Combining Local and Global Shape Models for Deformable Object Matching. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 1–12 (2009)Google Scholar
  18. 18.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kande dataset (CK+) : a complete facial expression dataset for action unit and emotion-specified expression. In: Proceedings of the Third IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis (2010)Google Scholar
  19. 19.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 46–53 (2000)Google Scholar
  20. 20.
    Dailey, M.N., Joyce, C., Lyons, M.J., Kamachi, M., Ishi, H., Gyoba, J., Cottrell, G.W.: Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion 10(6), 874–893 (2010)CrossRefGoogle Scholar
  21. 21.
    Aifanti, N., Papachristou, C., Delopoulos, A.: The MUG facial expression database. In: Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) (2010)Google Scholar
  22. 22.
    Fasel, B., Luettin, J.: Recognition of asymmetric facial action unit activities and intensities. In: Proceedings of the International Conference on Pattern Recognition (ICPR) (2000)Google Scholar
  23. 23.
    Mitra, S., Liu, Y.: Local facial asymmetry for expression classification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 889–894 (2004)Google Scholar
  24. 24.
    Liu, Y., Schmidt, K.L., Cohn, J.F., Mitra, S.: Facial asymmetry quantification for expression invariant human identification. J. Comput. Vis. Image Underst. 91(1–2), 138–159 (2003)CrossRefGoogle Scholar
  25. 25.
    Cootes, T.F., Cooper, D., Taylor, C.J., Graham, J.: Active shape models - their training and application. J. Comput. Vis. Image Underst. 61(1), 38–59 (1995)CrossRefGoogle Scholar
  26. 26.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, p. 484. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  27. 27.
    Gao, Y., Rehma, A., Wang, Z.: CW-SSIM based image classification. In: Proceedings of the IEEE International Conference on Image Processing (ICIP), pp. 1249–1252 (2011)Google Scholar
  28. 28.
    Mitra, N.J., Guibas, L., Pauly, M.: Symmetrization. ACM Trans. Graphics (SIGGRAPH) 26(3), 1–8 (2007)CrossRefGoogle Scholar
  29. 29.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. (IJCV) 57(2), 137–154 (2004)CrossRefGoogle Scholar
  30. 30.
    Ester, M., Kriegel, H-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the Second International Conference onKnowledge Discovery and Data Mining (KDD), pp. 226–231. AAAI Press (1996)Google Scholar
  31. 31.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, New York (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Harish Bhaskar
    • 1
    Email author
  • Davide La Torre
    • 2
    • 3
  • Mohammed Al-Mualla
    • 1
  1. 1.Visual Signal Analysis and Processing (VSAP) Research CenterKhalifa University of Science, Technology and ResearchAbu DhabiUAE
  2. 2.Department of Applied Mathematics and SciencesKhalifa University of Science, Technology and ResearchAbu DhabiUAE
  3. 3.Department of Economics, Management and Quantitative MethodsUniversity of MilanMilanItaly

Personalised recommendations