Focused Crawling Using Fictitious Play
A new probabilistic approach for focused crawling in hierarchically ordered information repositories is presented in this paper. The model is suitable for searching the World Wide Web and it is based on the fictitious play model from the theory of learning in games. The leading idea of the play is that players (software agents) are competing of the resources so that the search focuses on areas where relevant information can be found more likely. The model is basically a coordination model of the agent population but it is also possible to plug different features into the model, e.g. features for the user’s relevance feedback or semantic links between documents. Additionally, the method is highly scalable and the efficient parallel implementation of the method is possible.
Unable to display preview. Download preview PDF.
- 1.Brin, S. and Page, L.: The anatomy of a large-scale hypertextual Web search engine. In: Proceedings of the 7th International World Wide Web Conference (WWW7), Brisbane, Australia (1998)Google Scholar
- 2.De Bra, P., Houben, G., Kornatzky, Y., Post, R.: Information Retrieval in Distributed Hypertexts. In: Proceedings of the 4th Recherche d’Information Assistee par Ordinateur Conference (RIAO’94), New York, NY (1994)Google Scholar
- 3.CLEVER Searching. Network Document. Referenced 8.3.2002. Available: http://www.almaden.ibm.com/cs/k53/clever.html
- 4.Albert, R.: Statistical Mechanics of Complex Networks. A Dissertation. Notre Dame University (2001)Google Scholar
- 5.Fudenberg, D. and Levine, D. K.: The Theory of Learning in Games. Economic Learning and Social Evolution Series. The MIT Press (1999)Google Scholar