Knowledge Agents on the Web

  • Yariv Aridor
  • David Carmel
  • Ronny Lempel
  • Aya Soffer
  • Yoelle S. Maarek
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1860)


This paper introduces and evaluates a new paradigm, called Knowledge Agents, that incorporates agent technology into the process of domainspecific Web search. An agent is situated between the user and a search engine. It specializes in a specific domain by extracting characteristic information from search results. Domains are thus user-defined and can be of any granularity and specialty. This information is saved in a knowledge base and used in future searches. Queries are refined by the agent based on its domain-specific knowledge and the refined queries are sent to general purpose search engines. The search results are ranked based on the agent’s domain specific knowledge, thus filtering out pages which match the query but are irrelevant to the domain. A topological search of the Web for additional relevant sites is conducted from a domain-specific perspective. The combination of a broad search of the entire Web with domain-specific textual and topological scoring of results, enables the knowledge agent to find the most relevant documents for a given query within a domain of interest. The knowledge acquired by the agent is continuously updated and persistently stored thus users can benefit from search results of others in common domains.


Search Engine Information Retrieval Relevant Site Movie Review Knowledge Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ben-Shaul, I., Herscovici, M., Jacovi, M., Maarek, Y.S., Pelleg, D., Shtalhaim, M., Soroka, V., Ur, S.: Adding support for dynamic and focused search with fetuccino. In: Proceedings of the Eighth International WWW Conference, pp. 575–587. Elsevier, Amsterdam (1999)Google Scholar
  2. 2.
    CampSearch. The search engine for camps,
  3. 3.
    IBM Almaden Research Center. Clever,
  4. 4.
    Chakrabarti, S., Dom, B., Gibson, D., Kleinberg, J., Kumar, S.R., Raghavan, P., Rajagopalan, S., Tomkins, A.: Mining the web’s link structure. IEEE Computer 32(8), 60–67 (1999)Google Scholar
  5. 5.
    Chakrabarti, S., Dom, B., ven den Berg, M.: Focused crawling: A new approach to topic-specific web resource discovery. In: Proceedings of the Eighth International WWW Conference, pp. 545–562. Elsevier, Amsterdam (1999)Google Scholar
  6. 6.
    Excite Inc. Excite search,
  7. 7.
    Google Inc. Google search engine,
  8. 8.
    Yahoo Inc. Yahoo!,
  9. 9.
    IBM Jcentral. Search the web for java,
  10. 10.
    Kleinberg, J.M.: Authoritaive sources in a hyperlinked environment. In: Proceedings ofthe Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, January 1998, vol. 25-27, pp. 668–677 (1998)Google Scholar
  11. 11.
    Lempel, R.: Finding authoritative sites on the WWW (and other hyperlinked media) by analyzing the web’s link-structure. Master’s thesis, Technion, Israel Institute of Technology (July 1999)Google Scholar
  12. 12.
    Maarek, Y., Smadja, F.: Full text indexing based on lexical relations, an application: Software libraries. In: Belkin, N., van Rijsbergen, C. (eds.) Proceedings of SIGIR 1989, pp. 198–206. ACM press, Cambridge (1989)CrossRefGoogle Scholar
  13. 13.
    Manber, U., Bigot, P.A.: The search broker. In: The First Usenix Symposium on Internet Technologies and Systems, Monterey CA, December 1997, pp. 231–240 (1997)Google Scholar
  14. 14.
    MRQE. Movie review query engine,
  15. 15.
    Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. Computer Series. McGraw-Hill, New York (1983)zbMATHGoogle Scholar
  16. 16.
    Search Engine Watch. Search engine watch,
  17. 17.
    Xu, J., Croft, W.B.: Query expansion using local and global document analysis. In: Proceedings of the 19th annual international ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 4–11 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Yariv Aridor
    • 1
  • David Carmel
    • 1
  • Ronny Lempel
    • 2
  • Aya Soffer
    • 1
  • Yoelle S. Maarek
    • 1
  1. 1.IBM Haifa Research LaboratoryMATAMHaifaIsrael
  2. 2.Computer Science DepartmentTechnionHaifaIsrael

Personalised recommendations