Skip to main content
SpringerLink
Log in

RGB-D Sensor for Facial Expression Recognition in AAL Context

  • Conference paper
  • First Online:
Sensors and Microsystems (AISEM 2017)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 457))

Included in the following conference series:

  • 1033 Accesses

Abstract

This paper investigates the use of a commercial and low-cost RGB-D sensor for real-time facial expression recognition in Ambient Assisted Living Context. Since head poses and light conditions could be very different in domestic environments, the methodology used was designed to handle such situations. The implemented framework is able to classify four different categories of facial expressions: (1) happy, (2) sad, (3) fear/surprise, and (4) disgust/anger. The classification is obtained through an hybrid-based approach, by combining appearance and geometric features. The HOG feature descriptor and a group of Action Units compose the feature vector that is given as input, in the classification step, to a group of Support Vector Machines. The robustness of the approach is highlighted by the results obtained: the average accuracy for fear/surprise is the lowest with 85.2%, while happy is the facial expression better recognized (93.6%). Sad and disgust/anger are the facial expression confused the most.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), IEEE, pp. 1–4 (2008)

    Google Scholar 

  2. Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)

    Article  Google Scholar 

  3. Zheng, W.: Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Trans. Affect. Comput. 5(1), 71–85 (2014)

    Article  Google Scholar 

  4. Cament, L.A., Galdames, F.J., Bowyer, K.W., Perez, C.A.: Face recognition under pose variation with local Gabor features enhanced by active shape and statistical models. Pattern Recogn. 48(11), 3371–3384 (2015)

    Article  Google Scholar 

  5. Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)

    Article  Google Scholar 

  6. Malawski, F., Kwolek, B., Sako, S.: Using kinect for facial expression recognition under varying poses and illumination. In: International Conference on Active Media Technology, pp. 395–406, Springer International Publishing (2014)

    Google Scholar 

  7. Andò, B., Siciliano, P., Marletta, V., Monteriù, A.: Ambient Assisted Living. (2015)

    Google Scholar 

  8. Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)

    Article  Google Scholar 

  9. Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int J Signal Process., Image Process. Pattern Recognit. 8(1), 9–22 (2015)

    Google Scholar 

  10. Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recognit. 42(7), 1340–1350 (2009)

    Article  MATH  Google Scholar 

  11. Chen, Y., Hua, C., Bai, R.: Regression-based active appearance model initialization for facial feature tracking with missing frames. Pattern Recognit. Lett. 38, 113–119 (2014)

    Article  Google Scholar 

  12. Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010)

    Article  Google Scholar 

  13. Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)

    Article  Google Scholar 

  14. Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6): (2007)

    Google Scholar 

  15. Dahmane, M., Meunier, J.: Emotion recognition using dynamic grid-based HoG features. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 884–888, IEEE (2011)

    Google Scholar 

  16. Jack, R.E., Garrod, O.G., Schyns, P.G.: Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24(2), 187–192 (2014)

    Article  Google Scholar 

  17. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)

    Article  Google Scholar 

  18. Face Tracking: https://msdn.microsoft.com/en-us/library/jj130970.aspx

  19. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)

    Article  Google Scholar 

  20. Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013)

    Google Scholar 

  21. Dalal, N., & Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893, IEEE (2005)

    Google Scholar 

  22. Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22 (1978)

    Google Scholar 

  23. Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Neurocomputing, pp. 41–50. Springer, Berlin (1990)

    Google Scholar 

  24. Hsu, C.W., Chang, C.C., Lin, C.J.: A Practical Guide to Support Vector Classification. (2003)

    Google Scholar 

  25. Aly, S., Trubanova, A., Abbott, L., White, S., Youssef, A.: VT-KFER: a Kinect-based RGBD+ time dataset for spontaneous and non-spontaneous facial expression recognition. In: 2015 International Conference on Biometrics (ICB), pp. 90–97. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Research Council of Italy, Institute for Microelectronics and Microsystems, Lecce, Italy

    Andrea Caroppo, Alessandro Leone & Pietro Siciliano

Authors
  1. Andrea Caroppo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Leone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pietro Siciliano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Caroppo .

Editor information

Editors and Affiliations

  1. IMM-CNR, Lecce, Italy

    Alessandro Leone

  2. IMM-CNR, Lecce, Italy

    Angiola Forleo

  3. IMM-CNR, Lecce, Italy

    Luca Francioso

  4. IMM-CNR, Lecce, Italy

    Simona Capone

  5. IMM-CNR, Lecce, Italy

    Pietro Siciliano

  6. University of Rome Tor Vergata, Roma, Italy

    Corrado Di Natale

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Cite this paper

Caroppo, A., Leone, A., Siciliano, P. (2018). RGB-D Sensor for Facial Expression Recognition in AAL Context. In: Leone, A., Forleo, A., Francioso, L., Capone, S., Siciliano, P., Di Natale, C. (eds) Sensors and Microsystems. AISEM 2017. Lecture Notes in Electrical Engineering, vol 457. Springer, Cham. https://doi.org/10.1007/978-3-319-66802-4_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66802-4_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66801-7

  • Online ISBN: 978-3-319-66802-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation