Effective approach for facial expression recognition using hybrid square-based diagonal pattern geometric model

  • Neha JainEmail author
  • Shishir Kumar
  • Amit Kumar


Facial expressions convey human emotions in an expressive way. The development of an automated system to recognize the facial expressions is the difficult task. Automatic Facial Expression Recognition (FER) is an imperative process that leads to next-generation Human-Machine Interaction (HMI) for clinical practice and behavioral description. Segment detection and the extraction of relevant information from the images are the major issues to design an effective FER system. The creation of the suitable system that addresses these issues is the basic stage to achieve the accurate HMI models. The in-depth information analysis and maximization of labeled database are the real problems in the domain of FER approaches. A novel framework based on Square-Based Diagonal Pattern (SBDP) method on Geometric model called Geometric Appearance Models (GAM) has been presented through this paper that extracts the in-depth detailed of the features. The framework adopts the co-training by using detailed information from RGB-D images. The performance analysis of proposed SBDP-GAM regarding identification rate, sensitivity, accuracy and error rates with the RGB-D images shows the effectiveness diagonal patterns in facial expression identification. The comparative analysis of proposed SBDP-GAM model with the traditional methods regarding the recognition rate, error rate and F-score and on RGB-D images from EURECOMM database states the effectiveness of the proposed method. Moreover, the comparison of proposed SBDP-GAM with existing Support Vector Machine (SVM) regarding the acceptance rate (FAR, FRR, GAR) measures for biographer RGB-D image database proves the effectiveness of SBDP-GAM in FER applications.


Face recognition Feature extraction Image processing Image texture analysis Image sequence analysis 



Foremost, I would like to express my sincere gratitude to Dr. Deepak Kumar Jain, Institute of Automation, Chinese Academy of Sciences, Beijing, China for the continuous support of this research. His patience, motivation, enthusiasm, and immense knowledge helped me in all the time of research project.


  1. 1.
    Baltrušaitis T, Robinson P, Morency LP (2014) Continuous Conditional Neural Fields for Structured Regression. In:ECCV. 593–608Google Scholar
  2. 2.
    Bettadapura V (2012) Face expression recognition and analysis: the state of the art. arXiv:1203.6722Google Scholar
  3. 3.
    Bianco S, Cusano C, Schettini R (2015) Color Constancy Using CNNs. arXiv:1504.04548Google Scholar
  4. 4.
    Chen J, Takiguchi T, Ariki Y (2015) Facial expression recognition with a multithreaded cascade of rotation-invariant HOG. In: ACII. 636–642Google Scholar
  5. 5.
    Cheng Y, Zhao X, Huang K, Tan T (2014) Semi-supervised Learning for RGB-D Object Recognition. In: ICPR. 2377–2382Google Scholar
  6. 6.
    Coates A, Lee H, Ng AY (2010) An analysis of single-layer networks in unsupervised feature learning, Ann Arbor. 1001: 2Google Scholar
  7. 7.
    Fulcher BD, Jones NS (2014) Highly comparative feature-based time-series classification. In: IEEE Trans. KDE. 3026–3037Google Scholar
  8. 8.
    Ghosh D, Ari S (2015) Static Hand Gesture Recognition Using Mixture of Features and SVM Classifier. In: CSNT, 2015Google Scholar
  9. 9.
    Goswami G, Bharadwaj S, Vatsa M, Singh R (2013) On RGB-D face recognition using Kinect. In:BTAS 2013Google Scholar
  10. 10.
    Jiang B, Jia K (2013) Semi-supervised facial expression recognition algorithm on the condition of multi-pose. In: JIHMSD. 138–146Google Scholar
  11. 11.
    Lemaire P, Ardabilian M, Liming C, Daoudi M (2013) Fully automatic 3D facial expressionrecognition using differential mean curvature maps and histograms of oriented gradients. In:FG 2013Google Scholar
  12. 12.
    Liu M, Li S, Shan S, Chen X (2013) Enhancing Expression Recognition in the Wild with Unlabeled Reference Data. In: ACCV. 577–588Google Scholar
  13. 13.
    Madhu M, Amutha R (2012) Face recognition using gray level co-occurrence matrix and snapshot method of the Eigenface. In: IJEIT, 482–488Google Scholar
  14. 14.
    Maghraby AE, Abdalla M, Enany O, El Nahas M (2014) Detect and Analyze Face Parts Information using Viola-Jones and Geometric Approaches. In: IJCA. 23–28Google Scholar
  15. 15.
    Nicolaou MA, Gunes H, Pantic M (2012) Output-associative RVM regression for dimensional and continuous emotion prediction. In: Image and Vision Computing. 186–196Google Scholar
  16. 16.
    Ramirez Rivera A, Rojas C, Oksam C (2013) Local Directional Number Pattern for Face Analysis: Face and Expression Recognition. In: IEEETrans. IP, pp. 1740–1752Google Scholar
  17. 17.
    Reddy P, Reddy BE, Kumar VV (2013) Fuzzy-Based Image Dimensionality Reduction Using Shape Primitives for Efficient Face Recognition. In: JDMT, 2013Google Scholar
  18. 18.
    Romero P, Cid F, Núnez P (2013) A novel real-time facial expression recognition system based on candide-3 reconstruction model. In: WAF 2013Google Scholar
  19. 19.
    Ruan J, Yin J, Chen Q, Chen G (2014) Facial Expression Recognition Based on Gabor Wavelet Transform and Relevance Vector Machine. In: JICS, 2013Google Scholar
  20. 20.
    Rui M, Kose N, Dugelay JL (2014) KinectFaceDB: A Kinect Database for Face Recognition. IEEETrans. SMCS. 1534–1548Google Scholar
  21. 21.
    Sandbach G, Zafeiriou S, Pantic M, Yin L (2012) Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis Comput:683–697Google Scholar
  22. 22.
    Shen F, Xu Y, Liu L, Yang Y, Huang Z, Shen HT (2018) Unsupervised Deep Hashing with Similarity-Adaptive and Discrete Optimization, in IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:
  23. 23.
    Soleymani M, Pantic M, Pun T (2012) Multimodal emotion recognition in response to videos. In: IEEE Trans. AC. 211–223Google Scholar
  24. 24.
    Srivastava S, Joshi N, Gaur M (2014) A Review Paper on Feature Selection Methodologies and Their Applications. In: IJCSNS, pp. 78Google Scholar
  25. 25.
    Wong Y, Harandi MT, Sanderson C (2014) On robust face recognition via sparse coding: the good, the bad and the ugly. IET Biometrics 3:176–189CrossRefGoogle Scholar
  26. 26.
    Wright J, Yang AY , Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation, IEEE transactions on pattern analysis and machine intelligence. 31: 210–227Google Scholar
  27. 27.
    Yang M, Zhang L (2010) Gabor feature based sparse representation for face recognition with gabor occlusion dictionary,” in European conference on computer vision. 448–461Google Scholar
  28. 28.
    Yang M, Zhang L, Yang J, Zhang D (2011) Robust sparse coding for face recognition,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference. 625–632Google Scholar
  29. 29.
    Yang W, Li J, Zheng, Xu (2018) A Nuclear Norm Based Matrix Regression Based Projections Method for Feature Extraction, in IEEE Access, 6:7445–7451. doi:
  30. 30.
    Yongqiang L, Shangfei W, Yongping Z, Qiang J (2013) Simultaneous Facial Feature Tracking and Facial Expression Recognition. In: IEEETrans. IP. 2559–2573Google Scholar
  31. 31.
    Yu Q, Yin Y, Yang G, Ning Y, Li Y (2012) Face and Gait Recognition Based on Semi-supervised Learning. In: Pattern Recognition. 284–291Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringJaypee University of Engineering and TechnologyGunaIndia

Personalised recommendations