Multimedia Tools and Applications

, Volume 74, Issue 21, pp 9297–9322 | Cite as

Feast: face and emotion analysis system for smart tablets



Face and emotion recognition is still an open and very challenging problem. This paper presents a system FEAST which is an intelligent control system of Smart Tablets. It involves manipulating user sessions to adapt the working environment to his emotional state. First, a face detection followed by face and emotion recognition is performed, then a profile change is made basing on the obtained results. The face detection is based on skin color and geometric moments and face recognition is done by merging two features spaces, namely, Zernike moments and EAR-LBP. A feature selection technique reducing the parameter space size is applied. The same parameters are used for the emotion recognition.


Feature selection Smart tablet Face detection Emotion recognition 


  1. 1.
    Alastair JG, Darren G, Robert MF, Jon O (2008) Emotion rating from short blog texts. In Proceedings of CHI Florence, ItalyGoogle Scholar
  2. 2.
    Aouatif A, Rziza M, Driss A (2008) SVM-based face recognition using genetic search for frequency-feature subset selection. Image Signal Process Lect Notes Comput Sci 5099:321–328CrossRefGoogle Scholar
  3. 3.
    Ayadi ME, Kamel MS, Karray F (2011) Survey on speech emotion recognition: features, classification schemes, and databases. Pattern Recogn 44:572–587CrossRefMATHGoogle Scholar
  4. 4.
    Black MJ, Yacoob Y (1995) Tracking and recognizing rigid and non-rigid facial motions using local parametric model of image motion. Proc Int Conf Comput Vision IEEE Comput Soc 374–381Google Scholar
  5. 5.
    Burgoon JK, Jensen ML, Meservy TO, Kruse J, Nunamaker JF (2005) Augmenting human identification of emotional states in video. Intelligence Analysis Conference, McCleanGoogle Scholar
  6. 6.
    Camurri A, Lagerlof I, Volpe G (2003) Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int J Hum Comput Stud 59(1–2):213225Google Scholar
  7. 7.
    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698CrossRefGoogle Scholar
  8. 8.
    Carlo S, Rada (2008) Learning to identify emotions in text. In SAC 08, FortalezaGoogle Scholar
  9. 9.
    Cristian XR, Rainer L (2012) A survey on visual adult image recognition. Multimed Tools ApplGoogle Scholar
  10. 10.
    Crowley K, Sliney A, Pitt I, Murphy D (2010) Evaluating a brain-computer interface to categorize human emotional response, advanced learning technologies (ICALT). IEEE Int Conf 276–278Google Scholar
  11. 11.
    Ekman P, Friesen WV (1978) Facial action coding system: a technique for measure-ment of facial movement. Consulting Psychologists Press, Palo AltoGoogle Scholar
  12. 12.
    Erik H, Boon KL (2001) Face Detection. Surv Comput Vision Image Underst 83:236–274CrossRefMATHGoogle Scholar
  13. 13.
    Frank YS, Chao-Fa C (2004) Automatic extraction of head and face boundaries and facial features. Inf Sci 158:117–130CrossRefGoogle Scholar
  14. 14.
    Gunes H, Piccardi M (2006) A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior. Proc ICPR Int Conf Pattern Recogn 1:1148–1153Google Scholar
  15. 15.
    Gyu-Tae P, Bien Z (2000) Neural network-based fuzzy observer with application to facial analysis. Pattern Recogn Lett 21(1):93105Google Scholar
  16. 16.
    Haapalainen E, Kim S, Forlizzi JF, Dey AK (2010) Psycho-physiological measures for assessing cognitive load. Proc ACM Int Conf Ubiquit Comput ACM 301–310Google Scholar
  17. 17.
    Han CC, Liao HYM, Chung Yu K, Chen LH (1997) Fast face detection via morphology-based pre-processing. In: Lecture notes in computer science 1311. Springer, Berlin, p 469476Google Scholar
  18. 18.
    Hong P, Siyu X, Lizuo J, Liangzheng X (2011) Efficient face recognition fusing Dy-namic morphological quotient image with local binary pattern. IWANN Part II LNCS 6692:228–235Google Scholar
  19. 19.
    Hua Y, Yihua C, Hang S, Yawen F (2012) The large-scale crowd analysis based on sparse spatial-temporal local binary pattern. Mutimed Tools ApplGoogle Scholar
  20. 20.
    Jae-Kyung K, Won-Sung S, YangSun L (2012) Advanced knowledge sharing strategies based on learning style similarity for smart education. Comput Appl Biotechnol Multimed Ubiquit City 353:141–148CrossRefGoogle Scholar
  21. 21.
    James. J, Anurag R, Jestin J (2012) Architecture for secure tablet integration in automotive network. Proceedings of the FISITAGoogle Scholar
  22. 22.
    Javad H, Karim F, Madjid A (2003) An efficient human face recognition system using pseudo Zernike moment invariant and radial basis function neural network. Int J Pattern Recogn 17:41–62CrossRefGoogle Scholar
  23. 23.
    Jen-Da S, Shyi-Ming C (2007) Feature subset selection based on fuzzy entropy measures for handling classification problems. Appl Intell 28:69–82Google Scholar
  24. 24.
    Jingru WM, Guangzheng WY (1999) Knowledge-based edge detection and feature extraction of human-face organs. Pattern Recogn Artif Intell 12(3):340346Google Scholar
  25. 25.
    Kapur A, Kapur A, Babul NV, Tzanetakis G, Driessen PF (2005) Gesture-based af-fective computing on motion capture data. In: ACII 17Google Scholar
  26. 26.
    Karin S, Loannis P (1997) A fully automated approach to facial feature detection and tracking. Int Work Audio and Video Biom Person Authentification (AVBPA)Google Scholar
  27. 27.
    Lai JH, Yuen PC, FENG GC (2001) Face recognition using holistic fourier invariant features. Pattern Recogn 34(1):95109CrossRefGoogle Scholar
  28. 28.
    Lanitis A, Taylor CJ, Cootes TF (1995) An automatic face identification system using flexible appearance models. Image Vision Comput 13(5):393–401CrossRefGoogle Scholar
  29. 29.
    Le Hoai B, Nguyen AT (2005) Using rough Set in feature selection and reduction in face recognition problem. Adv Knowl Discov Data Min Lect Notes Comput Sci 3518:226–233CrossRefGoogle Scholar
  30. 30.
    Le HT, Nguyen DTN, Tran SH (2011) A facial expression classification system integrating canny principal component analysis and artificial neural network. Int J Mach Learn Comput 1(4)Google Scholar
  31. 31.
    Leonardis A, Bischof H (2000) Robust recognition using Eigen images. Comput Vision Image Underst 78(1):99118CrossRefGoogle Scholar
  32. 32.
    Mase K (1991) Recognition of facial expression from optical flow. IEICE Trans 74(10):34743483Google Scholar
  33. 33.
    Michal K (2010) Energy-based blob analysis for improving precision of skin segmentation. Multimed Tools Appl 49:463–481CrossRefGoogle Scholar
  34. 34.
    Nefian AV, Monson H, Hayes III (1998) Face detection and recognition using hidden Markov models. ICIP 1:141–145Google Scholar
  35. 35.
    New T, Foo S, DeSilva L (2003) Speech emotion recognition using hidden Markov models. Speech Comm 41:603623Google Scholar
  36. 36.
    Niemann H (1990) Pattern analysis and understanding. Springer, VerlagCrossRefMATHGoogle Scholar
  37. 37.
    Nikolaidis A, Pitas I (2000) Facial feature extraction and pose determination. Pattern Recogn 33(5):17831791Google Scholar
  38. 38.
    Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distribution. Pattern Recogn 29(1):51–59CrossRefGoogle Scholar
  39. 39.
    Osuna E, Freund R, Girosi F (1997)Training support vector machines, an application to face detection. Proc IEEE Conf Comput Vision Pattern Recogn 130–136Google Scholar
  40. 40.
    Pei-zhi C, Shui-li CA (2010) New face recognition algorithm based on DCT and LBPGoogle Scholar
  41. 41.
    Pfurtscheller G, Neuper C (2001) Motor imagery and direct brain-computer communi-cation. Proc IEEE 89:1123–1134CrossRefGoogle Scholar
  42. 42.
    Pitas I, Tsekeridou S (1998) Facial feature extraction in frontal views using biometric analogies. In: Proc of EUSIPCO 315–318Google Scholar
  43. 43.
    Pitas I, Venetsanopoulos AN (1990) Non linear digital filters: Principles and Applica-tions. Kluwer Academic publisherGoogle Scholar
  44. 44.
    Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):2338CrossRefGoogle Scholar
  45. 45.
    Ryu YS, Oh SY (2001) Automatic extraction of Eye and mouth fields from a face Im-age using Eigen features and multilayer perceptirons. Pattern Recogn 34(8):24592466Google Scholar
  46. 46.
    Satyanadh G, Vijayan A (2007) Selection for improved face recognition in multisensor images. Signals Comm Technol 109–120Google Scholar
  47. 47.
    Schneiderman H, Kenade T (1998) Probabilistic modeling of local appearance and spatial relationships for object recognition. Proc. IEEE Conf. Comput Vision Pattern Recogn 4551Google Scholar
  48. 48.
    Shenghua B, Shengliang X, Li Z, Rong Yan, Zhong S, Dingyi H, Yong Y (2012) Mining social emotions from affective text. In: IEEE Transactions on Knowledge and Data Engineering 24(9)Google Scholar
  49. 49.
    Shizhi C, YingLi T, Qingshan L, Dimitris NM (2011) Recognizing expressions from face and body gesture by temporal normalized motion and appearance features. Comput Vision Pattern Recogn Work (CVPRW)Google Scholar
  50. 50.
    Sung KK (1996) Learning and example selection for object and pattern detection. PhD thesis, Massachusetts Institute of TechnologyGoogle Scholar
  51. 51.
    Taskeed J, Md. Hasanul K, Oksam C (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5)Google Scholar
  52. 52.
    Tefas A, Kotropoulos C, Pitas I (2001) Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication. IEEE Trans Pattern Anal Mach Intell 23(7):735746CrossRefGoogle Scholar
  53. 53.
    Tian YL, Kanade T, Cohn J (2000) Recognizing lower face action units for facial expression analysis. Proc IEEE Int Conf Autom Face Gesture Recogn 484–490Google Scholar
  54. 54.
    Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):7186CrossRefGoogle Scholar
  55. 55.
    Venugopal KR, Patnaik LM (2012) Automatic facial expression. Recog AR-LBP ICIP CCIS 292:244–252Google Scholar
  56. 56.
    Wan-zeng K, Shan-an Z (2007) Multi-face detection based on down sampling and modi-fied subtractive clustering for color images. J Zhejiang Univ Sci A 8(1):7278Google Scholar
  57. 57.
    Xin-Mao H, Yi C (2013) Using homography relationship for auto-calibration in mobile smart-project device system. Multimed Tools Appl. doi: 10.1007/s11042-013-1783-3 Google Scholar
  58. 58.
    Yacoob Y, Davis L (1994) Computing spatio-temporal representations of human faces. Computer Vision and Pattern Recognition. Proceedings CVPR ’94, IEEE Computer Society Conference 70–75Google Scholar
  59. 59.
    Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):3458Google Scholar
  60. 60.
    Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: A survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58CrossRefGoogle Scholar
  61. 61.
    Yazhou L, Hongxun Y, Wen G, Debin Z (2005) Illumination invariant feature selection for face recognition, advances in multimedia information processing—PCM. Lect Notes Comput Sci 3768:946–957CrossRefGoogle Scholar
  62. 62.
    Yilmaz A, G¨okmen M (2001) Eigenhill vs. Eigenface and Eigenedge. Pattern Recogn 34(1):181–184CrossRefGoogle Scholar
  63. 63.
    Yu-Tzu L, Ruei-Yan L, Yu-Chih L, Greg C (2012) Real-time eye-gaze estimation using a low-resolution webcam. Multimedia Tools Appl 65(3):543–568Google Scholar
  64. 64.
    Zhiping S, Xi L, Qingyong L, Qing H (2012) Extracting discriminative features for CBIR. Multimed Tools Appl 61:263–279CrossRefGoogle Scholar
  65. 65.
    Zhou N, Lipo W (2009) Class-dependant feature selection for face recognition. AdvNeuro-Inf Process Lect Notes Comput Sci 5507:551–558CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.REGIM: Research Groups on Intelligent MachinesUniversity of Sfax, National Engineering School of Sfax (ENIS)SfaxTunisia
  2. 2.Department of Electronics, Faculty of EngineeringBadji-Mokhtar Annaba UniversityAnnabaAlgeria
  3. 3.Faculty of Science and Technique, Department of InformatiqueMohamed Cherif Messaadia University of Souk AhrasSouk AhrasAlgeria
  4. 4.Department of informatique, Faculty of EngineeringBadji Mokhtar-Annaba UniversityAnnabaAlgeria

Personalised recommendations