Experiments in Using Visual and Textual Clues for Image Hunting on the Web

  • Yuksel Alp Aslandogan
  • Clement T. Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1929)


In this paper we describe our experiences with Diogenes, a web-based search agent for finding person images. Diogenes1 implements different ways of combining visual and textual information for identifying person images. The sources of visual information are a face detection and a face recognition module. The textual information is obtained by analyzing the HTML structure and full text of web pages. Four different ways of combining these pieces of information are evaluated experimentally: (1) Face detection followed by text/HTML analysis, (2) face detection followed by face recognition, a linear combination of (1) and (2) and finally, a Dempster-Shafer combination of (1) and (2). We also compare the performance of Diogenes to those of research prototype and commercial image search engines. We report the results of a set of experimental retrievals for 20 persons examining over 30,000 URLs. In these retrievals Diogenes had the best average precision among the search engines evaluated including WebSEEk, AltaVista, Lycos and Ditto.


Face Recognition Facial Image Image Retrieval Average Precision Face Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Y. AlpAslandogan, Charles Thier, Clement T.Yu, Jun Zou, and Naphtali Rishe. Using Semantic Contents and WordNet(TM) in Image Retrieval. In Proceedings of ACM SIGIR Conference, Philadelphia, PA, 1997.Google Scholar
  2. 2.
    Y. Alp Aslandogan and Clement Yu. Multiple Evidence Combination in Image retrieval: Diogenes Searches for People on the Web. In Proceedings of ACM SIGIR 2000, Athens, Greece, July 2000.Google Scholar
  3. 3.
    Theo Gevers and Arnold W. M. Smeulders. PicToSeek: A Content-Based Image Search System for the World Wide Web. In Proceedings of SPIE Visual 97, 1997.Google Scholar
  4. 4.
    Joemon M. Jose, Jonathan Furner, and David J. Harper. Spatial Querying for Image Retrieval: A User Oriented Evaluation. In ACM SIGIR, pages 2320–240,1998.Google Scholar
  5. 5.
    Michael S. Lew, Kim Lempinen, and Nies Huijsmans. Webcrawling Using Sketches. In Proceedings of SPIE Visual 97, pages 77–84, 1997.Google Scholar
  6. 6.
    Sougata Mukherjea, Kyoji Hirata, and Yoshinori Hara. AMORE: A World Wide Web Image Retrieval Engine. World Wide Web, 2(3):115–132, 1999.CrossRefGoogle Scholar
  7. 7.
    Olaf Munkelt, Oliver Kaufmann, and Wolfgang Eckstein. Content-based Image Retrieval in theWorld WideWeb: AWeb Agent for Fetching Portraits. In Proceedings of SPIE Vol. 3022, pages 408–416, 1997.CrossRefGoogle Scholar
  8. 8.
    Henry A. Rowley, Shumeet Baluja, and Takeo Kanade. Neural Network-Based Face Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38, Jan 1998.CrossRefGoogle Scholar
  9. 9.
    Salton, G. Automatic Text Processing. Addison Wesley Mass., 1989.Google Scholar
  10. 10.
    Glenn Shafer. A Mathematical Theory of Evidence. Princeton University Press, 1976.Google Scholar
  11. 11.
    J. R. Smith and S. F. Chang. Visually Searching the Web for Content. IEEE Multimedia, 4(3):12–20, July-September 1997.CrossRefGoogle Scholar
  12. 12.
    Michael J. Swain, Charles Frankel, and Vassilis Athitsos. WebSeer: An Image Search Engine for the World Wide Web. Technical Report TR-96-14, University of Chicago, Department of Computer Science, July 1996.Google Scholar
  13. 13.
    Leonid Taycher, Marco LaCascia, and Stan Sclaroff. Image Digestion and Relevance Feedback in the ImageRover WWW Search Engine. In Proceedings of SPIE Visual 97, 1997.Google Scholar
  14. 14.
    M. Turk and A. Pentland. Eigenfaces for Recognition. Cognitive Neuroscience, 3(1):71–86, 1991.CrossRefGoogle Scholar
  15. 15.
    Clement T. Yu and Weiyi Meng. Principles of Database Query Processing for Advanced Applications. Data Management Systems. Morgan Kaufmann, 1998.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Yuksel Alp Aslandogan
    • 1
  • Clement T. Yu
    • 1
  1. 1.Department of EECSUniversity of Illinois at ChicagoChicagoUSA

Personalised recommendations