Skip to main content
SpringerLink
Log in

Automated Processing and Classification of Face Images for Human–Computer Interaction Applications

  • Chapter
Intelligent Interactive Systems in Knowledge-Based Environments

Part of the book series: Studies in Computational Intelligence ((SCI,volume 104))

Automated face detection and facial expression classification of images arises in the design of human–computer interaction and multimedia interactive service systems as a difficult, yet crucial, pattern recognition problem. Towards this goal, we have been building NEU-FACES, a novel system for processing multiple camera images of computer user faces to determine their affective state. In this chapter, we present an empirical study that we conducted to specify related design requirements, study statistically the expression recognition performance of humans, and identify quantitative facial features of high expression discrimination and classification power.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. Klein, R.W. Picard, and J. Riseberg. Support for Human Emotional Needs in Human–Computer Interaction, CHI’97 Workshop on Human Needs and Social Responsibility, 1997

    Google Scholar 

  2. M. Csikszentmihalyi. Flow: The Psychology of Optimal Experience, Harper and Row, New York, 1994

    Google Scholar 

  3. D. Goleman. Emotional Intelligence, Bantam Books, New York, 1995

    Google Scholar 

  4. P. Salovey and J.D. Mayer. Emotional Intelligence. Imagination, Cognition and Personality 9(3), 185–211 (1990)

    Google Scholar 

  5. A. Ortony, G.L. Clore, and A. Collins. The Cognitive Structure of Emotions, Cambridge University Press, 1988

    Google Scholar 

  6. H.W. Marsh and R.J. Shavelson. Self-concept: Its multifaceted, Hierarchical Structure. Educational Psychologist 20, 107–204 (1985)

    Article  Google Scholar 

  7. M. Rosenberg. Conceiving the Self, Basic Books, New York, 1979

    Google Scholar 

  8. I.-O. Stathopoulou and G.A. Tsihrintzis. A new neural network-based method for face detection in images and applications in bioinformatics. Proceedings of the 6th International Workshop on Mathematical Methods in Scattering Theory and Biomedical Engineering, September 17–21, 2003

    Google Scholar 

  9. I.-O. Stathopoulou and G.A. Tsihrintzis. A neural network-based facial analysis system. 5th International Workshop on Image Analysis for Multimedia Interactive Services, Lisboa, Portugal, April 21–23, 2004

    Google Scholar 

  10. I.-O. Stathopoulou and G.A. Tsihrintzis. An improved neural network-based face detection and facial expression classification system. IEEE International Conference on Systems, Man, and Cybernetics 2004, The Hague, The Netherlands, October 10–13, 2004

    Google Scholar 

  11. I.-O. Stathopoulou and G.A. Tsihrintzis. Pre-processing and expression classification in low quality face images. 5th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services, Smolenice, Slovak Republic, June 29–July 2, 2005

    Google Scholar 

  12. I.-O. Stathopoulou and G.A. Tsihrintzis. Evaluation of the discrimination power of features extracted from 2-D and 3-D facial images for facial expression analysis. 13th European Signal Processing Conference, Antalya, Turkey, September 4–8, 2005

    Google Scholar 

  13. I.-O. Stathopoulou and G.A. Tsihrintzis. Detection and expression classification systems for face images (FADECS), 2005 IEEE Workshop on Signal Processing Systems (SiPS’05), Athens, Greece, November 2–4, 2005

    Google Scholar 

  14. I.-O. Stathopoulou and G.A. Tsihrintzis. An accurate method for eye detection and feature extraction in face color images, IWSSIP-2006. 13th International Conference on Systems, Signals and Image Processing, Budapest, Hungary, September 21–23, 2006

    Google Scholar 

  15. I.-O. Stathopoulou and G.A. Tsihrintzis. Facial expression classification: specifying requirements for an automated system. 10th International Conference on Knowledge-Based & Intelligent Information & Engineering Systems, Bournemouth, United Kingdom, October 9–11, 2006

    Google Scholar 

  16. A.J. Colmenarez and T.S. Huang. Face detection with information-based maximum discrimination. Computer Vision and Pattern Recognition 782–787 (1997)

    Google Scholar 

  17. G. Yang and T.S. Huang. Human face detection in a complex background. Pattern Recognition 27(1), 53–63 (1994)

    Article  Google Scholar 

  18. S.Y. Lee, Y.K. Ham, and R.H. Park. Recognition of human front faces using knowledge-based feature extraction and neuro-fuzzy algorithm. Pattern Recognition 29(11), 1863–1876 (1996)

    Article  Google Scholar 

  19. T.K. Leung, M.C. Burl, and P. Perona. Finding faces in cluttered scenes using random labeled graph matching. Fifth International Conference on Computer Vision, IEEE Computer Society Press, Cambridge, MA, pp. 637–644, 1995

    Google Scholar 

  20. H.A. Rowley, S. Baluja, and T. Kanade. Rotation invariant neural network-based face detection. CMU-CS-97-201, 1997

    Google Scholar 

  21. H.A. Rowley, S. Baluja, and T. Kanade. Neural Network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1) (1998)

    Google Scholar 

  22. P. Juell and R. Marsh. A hierarchical neural network for human face detection. Pattern Recognition 29(5), 781–787 (1996)

    Article  Google Scholar 

  23. C. Lin and K. Fan. Triangle-based approach to the detection of human face. Pattern Recognition 34, 1271–1284 (2001)

    Article  MATH  Google Scholar 

  24. K.K. Sung and T. Poggio. Example-based learning for view-based human face detection. Proceedings on Image Understanding Workshop, Monterey, CA, pp. 843–850, 1994

    Google Scholar 

  25. P. Sinha. Qualitative representations for recognition. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York, LNCS 2525, pp. 249–262, 2002

    Google Scholar 

  26. M.–H. Yang and N. Ahuja. Face Detection and Gesture Recognition for Human–Computer Interaction. Kluwer Academic Publishers, Dordrecht, The Netherlands, 2001

    Google Scholar 

  27. Gender Classification (Databases). http://ise0.stanford.edu/class/ee368a_proj00/project15/intro.html; http://ise0.stanford.edu/class/ee368a_proj00/project15/append_a.html

  28. C. Fowlkes, S. Belongie, and J. Malik. Efficient Spatiotemporal Grouping Using the Nystr om Method CVPR, Hawaii, December 2001

    Google Scholar 

  29. S. Belongie. Notes on Clustering Pointsets with Normalized Cuts, 2000

    Google Scholar 

  30. J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8) (2000)

    Google Scholar 

  31. Image Segmentation using the Nystrom Method. http://rick.ucsd.edu/bleong/

  32. A.M. Martinez and R. Benavente. The AR face database. CVC Tech. Report #24, 1998. http://cobweb.ecn.purdue.edu/aleix/aleix_face_DB.html

  33. M.J. Lyons, J. Budynek, and S. Akamatsu. Automatic classification of single facial images. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(12), 1357–1362 (1999) http://www.kasrl.org/jaffe.html

    Google Scholar 

  34. The Yale Database: http://cvc.yale.edu/projects/yalefaces/yalefaces.html

  35. T. Kanade, J.F. Cohn, and Y. Tian. Comprehensive database for facial expression analysis. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000. http://vasc.ri.cmu.edu/idb/html/face/facial_expression/

  36. P. Ekman and W. Friesen. Unmasking the face: A Guide to Recognizing Emotions from Facial Expressions, Prentice Hall, Englewood Cliffs, NJ, 1975. http://www.paulekman.com/

  37. M. Pantic, M.F. Valstar, R. Rademaker, and L. Maat. Web-based Database for Facial Expression Analysis. Proc. IEEE Int’l Conf. Multimedia and Expo (ICME’05), Amsterdam, The Netherlands, July 2005. http://www.mmifacedb.com/

  38. J. Ahlberg. A system for face localization and facial feature extraction. Tech. Rep. LITH-ISY-R-2172, Linkoping University, 1999

    Google Scholar 

  39. J. Huang and H. Wechsler. Eye detection using optimal wavelet packets and RBFs. International Journal of Pattern Recognition Artificial Intelligence (IJPRAI) 13(6), 1009–1026 (1999)

    Google Scholar 

  40. K.M. Lam and H. Yan. Locating and extracting the covered eye in human face image. Pattern Recognition 29(5), 771–779 (1996)

    Article  MathSciNet  Google Scholar 

  41. L. Yin and A. Basu. Realistic animation using extended adaptive mesh for model based coding. In: Proc. Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 315–318, 1999

    Google Scholar 

  42. V. Vezhnevets and A. Degtiareva. Robust and accurate eye contour extraction. In: Proc. Graphicon, pp. 81–84, 2003

    Google Scholar 

  43. M. Ladws, J.C. Vorbruggen, J. Buhmann, and J. Lange. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions Computational 42, 570–582 (1993)

    Google Scholar 

  44. T.S. Lee. Image representation using 2D Gabor wavelets. IEEE Transactions PAMI 18(10), 959–971 (1996)

    Google Scholar 

  45. G.C. Feng and P.C. Yuen. Variance projection function and its application to eye detection for human face recognition. Patter Recognition Letters 19, 899–906 (1998)

    Article  Google Scholar 

  46. C. Feng and P.C. Yuen. Multi-cues eye detection on gray intensity image. Pattern Recognition 34(5), 1033–1046 (2001)

    Article  MATH  Google Scholar 

  47. T. Goto, W.-S. Lee, and N. Magnenat-Thalmann. Facial feature extraction for quick 3D face modeling. Signal Processing: Image Communication 17, 243–259 (2002)

    Article  Google Scholar 

  48. M. Porat and Y.Y. Zeevi. The generalized Gabor scheme of image representation in biological and machine vision. IEEE Transactions PAMI 10(4), 452–468 (1988)

    MATH  Google Scholar 

  49. K. Ville, J.-K. Kamarainen, and H. Kalviainen. Simple Gabor feature space for invariant object recognition. Patter Recognition Letters 25, 311–318 (2004)

    Article  Google Scholar 

  50. A. Yulle, P. Hallinan, and D. Cohen. Feature extraction from faces using deformable templates. International Journal of Computation Vision 8(2), 99–111 (1992)

    Article  Google Scholar 

  51. N. Zheng, W. Song, and W. Li. Image coding based on flexible contour model. Machine Graphics Vision 1(8), 83–94 (1999)

    Google Scholar 

  52. Z.-H. Zhou and X. Geng. Projection functions for eye detection. Pattern Recognition 37(5), 1049–1056 (2004)

    Article  MATH  Google Scholar 

  53. M.M. Fleck, D.A. Forsyth, and C. Bregler. Finding Naked People, 1996 European Conference on Computer Vision, Vol. II, pp. 592–602, 1996

    Google Scholar 

  54. M. Virvou and E. Alepis. Mobile educational features in authoring tools for personalised tutoring. Journal Computers and Education 44(1), 53–68 (2005)

    Article  Google Scholar 

  55. M. Virvou and G. Katsionis. Relating error diagnosis and performance characteristics for affect perception and empathy in an educational software application. Proceedings of the 10th International Conference on Human Computer Interaction (HCII) 2003, June 22–27 2003, Crete, Greece

    Google Scholar 

  56. M. Virvou and E. Alepis. Creating tutoring characters through a Web-based authoring tool for educational software. Proceedings of the IEEE International Conference on Systems Man & Cybernetics, 2003, Washington D.C., USA

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, University of Piraeus, Piraeus, 185 34, Greece

    Ioanna-Ourania Stathopoulou & George A. Tsihrintzis

Authors
  1. Ioanna-Ourania Stathopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George A. Tsihrintzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Piraeus, Karaoli & Dimitriou Street 80, 185 34, Piraeus, Greece

    Maria Virvou

  2. School of Electrical & Information Engineering, University of South Australia KES Centre, Mawson Lakes Campus, Adelaide, SA, 5095, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Stathopoulou, IO., Tsihrintzis, G.A. (2008). Automated Processing and Classification of Face Images for Human–Computer Interaction Applications. In: Virvou, M., Jain, L.C. (eds) Intelligent Interactive Systems in Knowledge-Based Environments. Studies in Computational Intelligence, vol 104. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77471-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-77471-6_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-77470-9

  • Online ISBN: 978-3-540-77471-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation