Multimedia Tools and Applications

, Volume 73, Issue 1, pp 309–326 | Cite as

Facial expression recognition using bag of distances

  • Fu-Song Hsu
  • Wei-Yang LinEmail author
  • Tzu-Wei Tsai


The automatic recognition of facial expressions is critical to applications that are required to recognize human emotions, such as multimodal user interfaces. A novel framework for recognizing facial expressions is presented in this paper. First, distance-based features are introduced and are integrated to yield an improved discriminative power. Second, a bag of distances model is applied to comprehend training images and to construct codebooks automatically. Third, the combined distance-based features are transformed into mid-level features using the trained codebooks. Finally, a support vector machine (SVM) classifier for recognizing facial expressions can be trained. The results of this study show that the proposed approach outperforms the state-of-the-art methods regarding the recognition rate, using a CK+ dataset.


Bag of distances Facial expression recognition Facial features 


  1. 1.
    Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27:1–27:27CrossRefGoogle Scholar
  2. 2.
    Chew S, Rana R, Lucey P, Lucey S, Sridharan S (2012) Sparse temporal representations for facial expression recognition. In: Ho Y-S (ed) Advances in image and video technology. Lecture notes in computer science, vol 7088. Springer, Berlin/Heidelberg, pp 311–322CrossRefGoogle Scholar
  3. 3.
    Chew SW, Lucey S, Lucey P, Sridharan S, Cohn JF (2012) Improved facial expression recognition via uni-hyperplane classification. In: 2012 IEEE conference on Computer Vision and Pattern Recognition (CVPR), pp 2554–2561Google Scholar
  4. 4.
    Cohen I, Sebe N, Chen L, Garg A, Huang TS (2003) Facial expression recognition from video sequences: temporal and static modelling. Comput Vis Image Underst 91(1–2):160–187CrossRefGoogle Scholar
  5. 5.
    Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685CrossRefGoogle Scholar
  6. 6.
    Delaunay B (1934) Sur la sphère vide. Bull Acad Sci USSR 7(6):793–800Google Scholar
  7. 7.
    De la Torre F, Cohn JF (2011) Guide to visual analysis of humans: looking at people. Chapter Facial expression analysis. SpringerGoogle Scholar
  8. 8.
    Ekman P, Friesen WV, Hager JC (2002) Facial Action Coding System (FACS): manual. A Human Face, Salt Lake City (USA)Google Scholar
  9. 9.
    Gu W, Xiang C, Venkatesh YV, Huang D, Lin H (2012) Facial expression recognition using radial encoding of local gabor features and classifier synthesis. Pattern Recogn 45(1):80–91CrossRefGoogle Scholar
  10. 10.
    Ionita MC, Tresadern PA, Cootes TF (2011) Real time feature point tracking with automatic model selection. In: Proc. International Conf. on Computer Vision Workshops (ICCV Workshops), pp 453–460Google Scholar
  11. 11.
    Jun B, Kim T, Kim D (2011) A compact local binary pattern using maximization of mutual information for face analysis. Pattern Recogn 44(3):532–543MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20:226–239CrossRefGoogle Scholar
  13. 13.
    Kotsia I, Zafeiriou S, Pitas I (2008) Texture and shape information fusion for facial expression and facial action unit recognition. Pattern Recogn 41(3):833–851CrossRefzbMATHGoogle Scholar
  14. 14.
    Lee C-C, Huang S-S, Shih C-Y (2010) Facial affect recognition using regularized discriminant analysis-based algorithms. In: EURASIP journal on advances in signal processing, 2010Google Scholar
  15. 15.
    Li Z, Imai J, Kaneko M (2009) Facial-component-based bag of words and phog descriptor for facial expression recognition. In: IEEE international conference on systems, man and cybernetics, pp 1353–1358Google Scholar
  16. 16.
    Liao S, Fan W, Chung ACS, Yeung D-Y (2006) Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features. In: Proc. IEEE conf. on image processing, pp 665–668Google Scholar
  17. 17.
    Licsar A, Sziranyi T, Kovacs L, Pataki B (2009) A folk song retrieval system with a gesture-based interface. IEEE Multimed 16(3):48–59CrossRefGoogle Scholar
  18. 18.
    Littlewort G, Whitehill J, Wu T, Fasel I, Frank M, Movellan J, Bartlett M (2011) The computer expression recognition toolbox (cert). In: 2011 IEEE international conference on automatic face gesture recognition and workshops (FG 2011), pp 298–305Google Scholar
  19. 19.
    Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: Proc. IEEE conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 94–101Google Scholar
  20. 20.
    Lyons MJ, Budynek J, Akamatsu S (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 21(12):1357–1362CrossRefGoogle Scholar
  21. 21.
    Naika SCL, Jha SS, Das PK, Nair SB (2012) Automatic facial expression recognition using extended ar-lbp. In: Wireless networks and computational intelligence. Communications in computer and information science, vol 292. Springer, Berlin, Heidelberg, pp 244–252Google Scholar
  22. 22.
    Peng W-T, Chu W-T, Chang C-H, Chou C-N, Huang W-J, Chang W-Y, Hung Y-P (2011) Editing by viewing: automatic home video summarization by viewing behavior analysis. IEEE Trans Multimed 13(3):539–550CrossRefGoogle Scholar
  23. 23.
    Platt J (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifiers. MIT Press, pp 61–74Google Scholar
  24. 24.
    Raducanu B, Dornaika F (2010) Dynamic facial expression recognition using laplacian eigenmaps-based manifold learning. In: Proc. IEEE Conf. on Robotics and Automation (ICRA), pp 156–161Google Scholar
  25. 25.
    Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38CrossRefGoogle Scholar
  26. 26.
    Shin G, Junchul C (2008) Spatio-temporal facial expression recognition using optical flow and hmm. In: Lee R (ed) Software engineering, artificial intelligence, networking and parallel/distributed computing. Studies in computational intelligence, vol 149. Springer, Berlin/Heidelberg, pp 27–38Google Scholar
  27. 27.
    Su M-C, Hsieh Y-J, Huang D-Y (2007) Facial expression recognition using optical flow without complex feature extraction. WSEAS Trans Comput 6:763–770Google Scholar
  28. 28.
    Tanchotsrinon C, Phimoltares S, Maneeroj S (2011) Facial expression recognition using graph-based features and artificial neural networks. In: Proc. IEEE conf. on Imaging Systems and Techniques (IST), pp 331–334Google Scholar
  29. 29.
    Xiao R, Zhao Q, Zhang D, Shi P (2011) Facial expression recognition on multiple manifolds. Pattern Recogn 44(1):107–116CrossRefzbMATHGoogle Scholar
  30. 30.
    Yang S, Bhanu B (2012) Understanding discrete facial expressions in video using an emotion avatar image. IEEE Trans Syst Man Cybern B: Cybern 42(4):980–992CrossRefGoogle Scholar
  31. 31.
    Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58CrossRefGoogle Scholar
  32. 32.
    Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928CrossRefGoogle Scholar
  33. 33.
    Zhi R, Ruan Q (2008) Facial expression recognition based on two-dimensional discriminant locality preserving projections. Neurocomputing 71(7–9):1730–1734CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of Computer Science and Information EngineeringNational Chung Cheng UniversityChia-YiTaiwan
  2. 2.Department of Multimedia DesignNational Taichung University of Science and TechnologyTaichungTaiwan

Personalised recommendations