Skip to main content
SpringerLink
Log in

Facial expression recognition using bag of distances

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The automatic recognition of facial expressions is critical to applications that are required to recognize human emotions, such as multimodal user interfaces. A novel framework for recognizing facial expressions is presented in this paper. First, distance-based features are introduced and are integrated to yield an improved discriminative power. Second, a bag of distances model is applied to comprehend training images and to construct codebooks automatically. Third, the combined distance-based features are transformed into mid-level features using the trained codebooks. Finally, a support vector machine (SVM) classifier for recognizing facial expressions can be trained. The results of this study show that the proposed approach outperforms the state-of-the-art methods regarding the recognition rate, using a CK+ dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27:1–27:27

    Article  Google Scholar 

  2. Chew S, Rana R, Lucey P, Lucey S, Sridharan S (2012) Sparse temporal representations for facial expression recognition. In: Ho Y-S (ed) Advances in image and video technology. Lecture notes in computer science, vol 7088. Springer, Berlin/Heidelberg, pp 311–322

    Chapter  Google Scholar 

  3. Chew SW, Lucey S, Lucey P, Sridharan S, Cohn JF (2012) Improved facial expression recognition via uni-hyperplane classification. In: 2012 IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp 2554–2561

  4. Cohen I, Sebe N, Chen L, Garg A, Huang TS (2003) Facial expression recognition from video sequences: temporal and static modelling. Comput Vis Image Underst 91(1–2):160–187

    Article  Google Scholar 

  5. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  6. Delaunay B (1934) Sur la sphère vide. Bull Acad Sci USSR 7(6):793–800

    Google Scholar 

  7. De la Torre F, Cohn JF (2011) Guide to visual analysis of humans: looking at people. Chapter Facial expression analysis. Springer

  8. Ekman P, Friesen WV, Hager JC (2002) Facial Action Coding System (FACS): manual. A Human Face, Salt Lake City (USA)

  9. Gu W, Xiang C, Venkatesh YV, Huang D, Lin H (2012) Facial expression recognition using radial encoding of local gabor features and classifier synthesis. Pattern Recogn 45(1):80–91

    Article  Google Scholar 

  10. Ionita MC, Tresadern PA, Cootes TF (2011) Real time feature point tracking with automatic model selection. In: Proc. International Conf. on Computer Vision Workshops (ICCV Workshops), pp 453–460

  11. Jun B, Kim T, Kim D (2011) A compact local binary pattern using maximization of mutual information for face analysis. Pattern Recogn 44(3):532–543

    Article  MathSciNet  MATH  Google Scholar 

  12. Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20:226–239

    Article  Google Scholar 

  13. Kotsia I, Zafeiriou S, Pitas I (2008) Texture and shape information fusion for facial expression and facial action unit recognition. Pattern Recogn 41(3):833–851

    Article  MATH  Google Scholar 

  14. Lee C-C, Huang S-S, Shih C-Y (2010) Facial affect recognition using regularized discriminant analysis-based algorithms. In: EURASIP journal on advances in signal processing, 2010

  15. Li Z, Imai J, Kaneko M (2009) Facial-component-based bag of words and phog descriptor for facial expression recognition. In: IEEE international conference on systems, man and cybernetics, pp 1353–1358

  16. Liao S, Fan W, Chung ACS, Yeung D-Y (2006) Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features. In: Proc. IEEE conf. on image processing, pp 665–668

  17. Licsar A, Sziranyi T, Kovacs L, Pataki B (2009) A folk song retrieval system with a gesture-based interface. IEEE Multimed 16(3):48–59

    Article  Google Scholar 

  18. Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, Bartlett M (2011) The computer expression recognition toolbox (cert). In: 2011 IEEE international conference on automatic face gesture recognition and workshops (FG 2011), pp 298–305

  19. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: Proc. IEEE conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 94–101

  20. Lyons MJ, Budynek J, Akamatsu S (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 21(12):1357–1362

    Article  Google Scholar 

  21. Naika SCL, Jha SS, Das PK, Nair SB (2012) Automatic facial expression recognition using extended ar-lbp. In: Wireless networks and computational intelligence. Communications in computer and information science, vol 292. Springer, Berlin, Heidelberg, pp 244–252

    Google Scholar 

  22. Peng W-T, Chu W-T, Chang C-H, Chou C-N, Huang W-J, Chang W-Y, Hung Y-P (2011) Editing by viewing: automatic home video summarization by viewing behavior analysis. IEEE Trans Multimed 13(3):539–550

    Article  Google Scholar 

  23. Platt J (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifiers. MIT Press, pp 61–74

  24. Raducanu B, Dornaika F (2010) Dynamic facial expression recognition using laplacian eigenmaps-based manifold learning. In: Proc. IEEE Conf. on Robotics and Automation (ICRA), pp 156–161

  25. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

    Article  Google Scholar 

  26. Shin G, Junchul C (2008) Spatio-temporal facial expression recognition using optical flow and hmm. In: Lee R (ed) Software engineering, artificial intelligence, networking and parallel/distributed computing. Studies in computational intelligence, vol 149. Springer, Berlin/Heidelberg, pp 27–38

    Google Scholar 

  27. Su M-C, Hsieh Y-J, Huang D-Y (2007) Facial expression recognition using optical flow without complex feature extraction. WSEAS Trans Comput 6:763–770

    Google Scholar 

  28. Tanchotsrinon C, Phimoltares S, Maneeroj S (2011) Facial expression recognition using graph-based features and artificial neural networks. In: Proc. IEEE conf. on Imaging Systems and Techniques (IST), pp 331–334

  29. Xiao R, Zhao Q, Zhang D, Shi P (2011) Facial expression recognition on multiple manifolds. Pattern Recogn 44(1):107–116

    Article  MATH  Google Scholar 

  30. Yang S, Bhanu B (2012) Understanding discrete facial expressions in video using an emotion avatar image. IEEE Trans Syst Man Cybern B: Cybern 42(4):980–992

    Article  Google Scholar 

  31. Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Article  Google Scholar 

  32. Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928

    Article  Google Scholar 

  33. Zhi R, Ruan Q (2008) Facial expression recognition based on two-dimensional discriminant locality preserving projections. Neurocomputing 71(7–9):1730–1734

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Chung Cheng University, Chia-Yi, Taiwan

    Fu-Song Hsu & Wei-Yang Lin

  2. Department of Multimedia Design, National Taichung University of Science and Technology, Taichung, Taiwan

    Tzu-Wei Tsai

Authors
  1. Fu-Song Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei-Yang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tzu-Wei Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-Yang Lin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hsu, FS., Lin, WY. & Tsai, TW. Facial expression recognition using bag of distances. Multimed Tools Appl 73, 309–326 (2014). https://doi.org/10.1007/s11042-013-1616-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1616-4

Keywords

Search

Navigation