A Multi-Agent System for Face Detection and Annotation

  • Rajiv Khosla
  • Ishwar K. Sethi
  • Ernesto Damiani
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 582)


The purpose of this chapter is to describe the application of the problem solving ontology component of HCVM in the area of face detection and annotation. computer-based aspects of an embedded feature-based multi-agent face detection and annotation system are described in this chapter. Unlike similar feature-based systems that start from searching for facial organs in the images and group them to find faces, the system solves the problem using the problem solving agents of the HCVM in a top-down fashion. The embedded mtilti-agent face detection system makes hypotheses of face locations and seeks evidences to verify them. The five problem solving agents follow a coarse grain to fine grain methodology and work on various features of color images. The coarse to fine top-down methodology is more akin to human perceptions than a bottom-up approach. The higher level problem solving agents like decomposition work as background knowledge for the control and decision agents resulting in improvement of the accuracy and speed of detection. It is also distinct from many other top-down systems in that no re-scaling of the images is needed in the searching process. To improve the detection rate, a unique iterative region-partitioning algorithm is developed in this multi-agent system. The problem solving agents model aspects related to skin-tone region segmentation, noise reduction, candidate face regions location and facial feature extraction/face detection, and annotation. The detection algorithm is invariant to scale and rotation to some degree. The performance of the multi-agent system has been tested with a large number of images and the results obtained thus far are very encouraging.


Facial Feature Face Detection Annotation System Image Annotation Skin Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Dai, Ying et al, (1998) “Study of facial expression recognition using the Hopfield model”, DSP 90–7, pp. 37–42 (in Japanese)Google Scholar
  2. Dai, Ying et al, (1996) “Face-Texture Model Based on SGLD and its Application in Face Detection in a Color Scene”, Pattern Recognition, Vol. 29, No. 6, p1007–1017.CrossRefGoogle Scholar
  3. Gonzalez, Rafael C., Woods, Richard E., (1992) Digital Image Processing,Addison-Wesley Publishing Company.Google Scholar
  4. Govindaraju, V., Sher, D. B., Srihari, R.K., and Srihari, S. N., (1989), “Locating human faces in newspaper photographs”, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, June.Google Scholar
  5. Jeng, Shi-Hong et al, (1998) “Facial Feature Detection Using Geometrical Face Model: an Efficient Approach”, Pattern Recognition, Vol. 31, No. 3, p273–282.MathSciNetCrossRefGoogle Scholar
  6. Kapfer, M. et al, (1997) “Detection of human faces in color image sequences with arbitrary motions for very low bit-rate videophone coding”, Pattern Recognition Letters 18, p1503–1518.Google Scholar
  7. Leung, T. K. et al, (1995) “Finding Faces in Cluttered Scenes using Random Labeled Graph Matching”, Fifth Intl. Conf. On Computer Vision, Cambridge, MA, June.Google Scholar
  8. Moghaddam, Baback et al, (1998)“A Subspace Method for Maximum Likelihood Target Detection”, IEEE International Conference on Image Processing,Washington DC, October. RowleyGoogle Scholar
  9. Henry A., et al, (1998), “Neural Network-Based Face Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 1, January, pp23–37Google Scholar
  10. Saber, Eli and Tekalp, A. Murat, (1998) “Frontal-view Face Detection and Facial Feature Extraction using Color, Shape and Symmetry Based Cost Functions”, Pattern Recognition Letters,V0019 N8, June, p669–680Google Scholar
  11. Sirohey, Saad Ahmed,(1993) “Human Face Segmentation and Identification”, Master’s Thesis,Center for Automation Research, University of Maryland, MD, Nov.Google Scholar
  12. Sung, Kah-Kay et al, (1998) “Example-Based Learning for View-Based Human Face Detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 1, January Sung, Kah-Kay, “Learning and Example Selection for Object and Pattern Detection”, Ph.D. dissertation, Artificial Intelligence Lab., Massachusetts Institute of TechnologyGoogle Scholar
  13. Yang, Guanzheng et al, (1994) “Human Face Detection in a Complex Background”, Pattern Recognition, Vol. 27, No. 1, pp 53–63.CrossRefGoogle Scholar
  14. Yow, Kin Choong, et al, (1997) “Feature-based human face detection”, Image and Vision Computing 15 713–735CrossRefGoogle Scholar
  15. Zhang, Y. J. et al, “Automatic Face Segmentation Using Color Cues for Coding Typical Videophone Scenes”, SPIE Vol. 3024 Google Scholar

Copyright information

© Springer Science+Business Media New York 2000

Authors and Affiliations

  • Rajiv Khosla
  • Ishwar K. Sethi
  • Ernesto Damiani

There are no affiliations available

Personalised recommendations