Abstract
Distributed heterogeneous search environments are an emerging phenomenon in Web search, in which topic-specific search engines provide search services, and metasearchers distribute user’s queries to only the most suitable search engines. Previous research has explored the performance of such environments from the user’s perspective (e.g., improved quality of search results). We focus instead on performance from the search service provider’s point of view (e.g, income from queries processed vs. resources used to answer them). We analyse a scenario in which individual search engines compete for queries by choosing which documents to index. We propose the COUGAR algorithm that specialised search engines can use to decide which documents to index on each particular topic. COUGAR is based on a game-theoretic analysis of heterogeneous search environments, and uses reinforcement learning techniques to exploit the sub-optimal behaviour of its competitors.
This research was supported by grant SFI/01/F.1/C015 from Science Foundation Ireland, and grant N00014-03-1-0274 from the US Office of Naval Research.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Gravano, L., Garcia-Molina, H.: GlOSS: Text-source discovery over the Internet. ACM Trans. on Database Systems 24, 229–264 (1999)
Rubinstein, A.: Modelling Bounded Rationality. The MIT Press, Cambridge (1997)
Khoussainov, R., Kushmerick, N.: Performance management in competitive distributedWeb search. In: Proc. of the IEEE/WIC Intl. Conf. on Web Intelligence (2003) (to appear)
Risvik, K., Michelsen, R.: Search engines and web dynamics. Computer Networks 39 (2002)
van Rijsbergen, C.J.: Information Retrieval, 2nd edn. Butterworths (1979)
Chakrabarti, S., van den Berg, M., Dom, B.: Focused crawling: A new approach to topicspecific Web resource discovery. In: Proc. of the 8th WWW Conf. (1999)
Osborne, M., Rubinstein, A.: A Course in Game Theory. The MIT Press, Cambridge (1999)
Conitzer, V., Sandholm, T.: Complexity results about Nash equilibria. Technical Report CMU-CS-02-135, Carnegie Mellon University (2002)
Robinson, J.: An iterative method of solving a game. Annals of Mathematics 54 (1951)
Carmel, D., Markovitch, S.: Learning models of intelligent agents. In: Proc. of the 13th National Conf. on AI (1996)
Peshkin, L.: Reinforcement Learning by Policy Search. PhD thesis, MIT (2002)
Stonebraker, M., Devine, R., Kornacker, M., Litwin, W., Pfeffer, A., Sah, A., Staelin, C.: An economic paradigm for query processing and data migration in Mariposa. In: Proc. of the 3rd Intl. Conf. on Parallel and Distributed Information Systems, pp. 58–67 (1994)
Greenwald, A., Kephart, J., Tesauro, G.: Strategic pricebot dynamics. In: Proc. of the 1st ACM Conf. on Electronic Commerce, pp. 58–67 (1999)
Greenwald, A., Kephart, J.: Shopbots and pricebots. In: Proc. of the 16th Intl. Joint Conf. on AI, pp. 506–511 (1999)
Bowling, M., Veloso, M.: Rational and convergent learning in stochastic games. In: Proc. of the 17th Intl. Joint Conf. on AI (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Khoussainov, R., Kushmerick, N. (2003). Optimising Performance of Competing Search Engines in Heterogeneous Web Environments. In: Lavrač, N., Gamberger, D., Blockeel, H., Todorovski, L. (eds) Machine Learning: ECML 2003. ECML 2003. Lecture Notes in Computer Science(), vol 2837. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39857-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-39857-8_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20121-2
Online ISBN: 978-3-540-39857-8
eBook Packages: Springer Book Archive