Applied Intelligence

, Volume 49, Issue 7, pp 2659–2671 | Cite as

Sparse modified marginal fisher analysis for facial expression recognition

  • Zhe Wang
  • Li ZhangEmail author
  • Bangjun Wang


Marginal Fisher analysis (MFA) is an efficient method for dimension reduction, which can extract useful discriminant features for image recognition. Since sparse learning can achieve better generalization ability and lessen the amount of computations in recognition tasks, this paper introduces sparsity into MFA and proposes a novel sparse modified MFA (SMMFA) method for facial expression recognition. The goal of SMMFA is to extract discriminative features by using the resulted sparse projection matrix. First, a modified MFA is proposed to find the original projection matrix. Similar to MFA, the modified MFA also defines the intra-class graph and the inter-class graph to describe geometry structure in the same class and local discriminant structure between different classes, respectively. In addition, the modified MFA removes the null space of the total scatter matrix. The sparse solution of SMMFA can be gained by solving the 1 –minimization problem on the original projection matrix using the linearized Bregman iteration. Experimental results show that the proposed SMMFA can effectively extract intrinsic features and has better discriminant power than the state-of-the-art methods.


Facial expression recognition Marginal fisher analysis Sparse learning Linearized Bregman iteration 



This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61373093 and 61572339, by the Soochow Scholar Project, by the Six Talent Peak Project of Jiangsu Province of China, and by the Collaborative Innovation Center of Novel Software Technology and Industrialization.


  1. 1.
    Yuan C, Wu Q, Li P, et al (2018) Expression recognition algorithm based on the relative relationship of the facial landmarks. In: International Congress on Image & Signal Processing, Shanghai, China, pp 1–5Google Scholar
  2. 2.
    Liu X, Kumar BVKV, You J, et al (2017) Adaptive deep metric learning for identity-aware facial expression recognition. In: IEEE Conference on Computer Vision & Pattern Recognition Workshops, Honolulu, Hawaii, pp 522–531Google Scholar
  3. 3.
    Kabir MH, Salekin MS, Uddin MZ, Abdullah-al-Wadud M (2017) Facial expression recognition from depth video with patterns of oriented motion flow. IEEE ACCS 5(99):8880–8889CrossRefGoogle Scholar
  4. 4.
    Vrigkas M, Nikou C, Kakadiaris IA (2016) Exploiting privileged information for facial expression recognition. In: International Conference on Biometrics, Halmstad, Sweden, pp 1–8Google Scholar
  5. 5.
    Wang Z, Ruan Q, An G (2016) Facial expression recognition using sparse local fisher discriminant analysis. Neurocomputing 174:756–766CrossRefGoogle Scholar
  6. 6.
    Ren F, Huang Z (2015) Facial expression recognition based on AAM–SIFT and adaptive regional weighting. IEEE Trans Electr Electron Eng 10(6):713–722CrossRefGoogle Scholar
  7. 7.
    Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial action. Consulting Psychologists Press, Palo AltoGoogle Scholar
  8. 8.
    Amini R, Lisetti C, Ruiz G (2015) HapFACS 3.0: FACS-based facial expression generator for 3D speaking virtual characters. IEEE Trans Affect Comput 6(4):348–360CrossRefGoogle Scholar
  9. 9.
    Hofmann J, Platt T, Ruch W (2017) Laughter and smiling in 16 positive emotions. IEEE Trans Affect Comput 8(4):495–507CrossRefGoogle Scholar
  10. 10.
    Kamarol SKA, Jaward MH, Parkkinen J et al (2016) Spatiotemporal feature extraction for facial expression recognition. IET Image Process 10(7):534–541CrossRefGoogle Scholar
  11. 11.
    Sun Y, Yu J (2017) Facial Expression Recognition by Fusing Gabor and Local Binary Pattern Features. In: International Conference on Multimedia modelling, Reykjavik, Iceland, pp 209–220Google Scholar
  12. 12.
    Puthenputhussery A, Liu Q, Liu C (2017) A sparse representation model using the complete marginal fisher analysis framework and its applications to visual recognition. IEEE Trans Multimedia 19(8):1757–1770CrossRefGoogle Scholar
  13. 13.
    Zheng W, Zong Y, Zhou X, Xin M (2018) Cross-domain color facial expression recognition using transductive transfer subspace learning. IEEE Trans Affect Comput 9(1):21–37CrossRefGoogle Scholar
  14. 14.
    Lin C, Long F, Zhan Y (2018) Facial expression recognition by learning spatiotemporal features with multi-layer independent subspace analysis. In: International Congress on Image & Signal Processing, Shanghai, China, pp 1–6Google Scholar
  15. 15.
    Nikitidis S, Tefas A, Pitas I (2013) Maximum margin discriminant projections for facial expression recognition. In: IEEE International Conference on Signal Processing, Marrakech, Morocco, pp 1–5Google Scholar
  16. 16.
    Jia J, Xu Y, Zhang S, et al (2016) The facial expression recognition method of random forest based on improved PCA extracting feature. In: IEEE International Conference on Signal Processing, Communications and Computing, Hong Kong, China, pp 1–5Google Scholar
  17. 17.
    Bouwmans T, Javed S, Zhang H, Lin Z, Otazo R (2018) On the applications of robust PCA in image and video processing. Proc IEEE 106(8):1427–1457CrossRefGoogle Scholar
  18. 18.
    Imran MA, Miah MSU, Rahman H (2015) Face recognition using eigenfaces. Int J Comput Appl 118(5):12–16Google Scholar
  19. 19.
    Chao L, Ding J, Liu Z (2015) Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection. Signal Process 117(12):1–10CrossRefGoogle Scholar
  20. 20.
    Chen SB, Wang J, Liu CY, Luo B (2017) Two-dimensional discriminant locality preserving projection based on ℓ1-norm maximization. Pattern Recogn Lett 87:147–154CrossRefGoogle Scholar
  21. 21.
    Siddiqi MH, Ali R, Khan AM, Young-Tack Park, Sungyoung Lee (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24(4):1386–1398MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Tian C, Zhang Q, Sun G, et al (2016) Linear discriminant analysis representation and CRC representation for image classification. In: IEEE International Conference on Computer & Communications, Chengdu, China, pp 755–760Google Scholar
  23. 23.
    Shah JH , Sharif M , Yasmin M, Fernandes SL (2017) Facial expressions classification and false label reduction using LDA and threefold SVM. Pattern Recognition Letters. Available online 23 June 2017:
  24. 24.
    Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern 6(3):443–454CrossRefGoogle Scholar
  25. 25.
    Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51CrossRefGoogle Scholar
  26. 26.
    Lu GF, Zou J, Wang Y, Wang Z (2017) L1-norm based null space discriminant analysis. Multim Tools Appl 76(14):15801–15816CrossRefGoogle Scholar
  27. 27.
    Yin J, Jin Z (2012) From NLDA to LDA/GSVD: a modified NLDA algorithm. Neural Comput & Applic 21(7):1575–1583CrossRefGoogle Scholar
  28. 28.
    Chu D, Liao LZ, Ng KP et al (2017) Incremental linear discriminant analysis: a fast algorithm and comparisons. IEEE Trans Neural Networks Learn Syst 26(11):2716–2735MathSciNetCrossRefGoogle Scholar
  29. 29.
    Zhang L, Cobzas D, Wilman AH, Kong L (2018) Significant anatomy detection through sparse classification: a comparative study. IEEE Trans Med Imaging 37(1):128–137CrossRefGoogle Scholar
  30. 30.
    Tibshirani R (1996) Regression shrinkage and selection via the LASSO: a retrospective. J R Stat Soc 58(1):267–288zbMATHGoogle Scholar
  31. 31.
    Qiao T, Li W, Wu B (2014) A new algorithm based on linearized Bregman iteration with generalized inverse for compressed sensing. Circuits Systems & Signal Processing 33(5):1527–1539MathSciNetCrossRefGoogle Scholar
  32. 32.
    Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat 15(2):265–286MathSciNetCrossRefGoogle Scholar
  33. 33.
    Qiao Z, Zhou L, Huang JZ (2009) Sparse linear discriminant analysis with applications to high dimensional low sample size data. IAENG Int J Appl Math 9(1):48–60MathSciNetzbMATHGoogle Scholar
  34. 34.
    Chu D, Liao LZ, Ng MK, Zhang X (2013) Sparse canonical correlation analysis: new formulation and algorithm. IEEE Trans Pattern Anal Mach Intell 35(12):3050–3065CrossRefGoogle Scholar
  35. 35.
    Jeribi A (2015) Spectral graph theory. In: Spectral Theory and Applications of Linear Operators and Block Operator Matrices, Springer, Cham, pp 413–439Google Scholar
  36. 36.
    Cai JF, Osher S, Shen Z (2009) Linearized Bregman iterations for compressed sensing. Math Comput 78(267):1515–1536MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Huang B, Ma S, Goldfarb D (2013) Accelerated linearized Bregman method. J Sci Comput 54(2–3):428–453MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Chen C, Xu G (2016) A new linearized split Bregman iterative algorithm for image reconstruction in sparse-view X-ray computed tomography. Comput Math Appl 71(8):1537–1559MathSciNetCrossRefGoogle Scholar
  39. 39.
    Lyons M, Akamatsu S, Kamachi M, et al (1998) Coding facial expressions with Gabor wavelets. In: Proceedings of the Third IEEE Conference on Face and Gesture Recognition, Nara, Japan, pp 200–205Google Scholar
  40. 40.
    Lee K, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698CrossRefGoogle Scholar
  41. 41.
    Valstar MF, Jiang B, Mehu M, et al (2011) The first facial expression recognition and analysis challenge. In: IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, pp 921–926Google Scholar
  42. 42.
    Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer Science and Technology & Joint International Research Laboratory of Machine Learning and Neuromorphic ComputingSoochow UniversitySuzhouChina
  2. 2.Provincial Key Laboratory for Computer Information Processing TechnologySoochow UniversitySuzhouChina

Personalised recommendations