Optimising Performance of Competing Search Engines in Heterogeneous Web Environments

  • Rinat Khoussainov
  • Nicholas Kushmerick
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2837)


Distributed heterogeneous search environments are an emerging phenomenon in Web search, in which topic-specific search engines provide search services, and metasearchers distribute user’s queries to only the most suitable search engines. Previous research has explored the performance of such environments from the user’s perspective (e.g., improved quality of search results). We focus instead on performance from the search service provider’s point of view (e.g, income from queries processed vs. resources used to answer them). We analyse a scenario in which individual search engines compete for queries by choosing which documents to index. We propose the COUGAR algorithm that specialised search engines can use to decide which documents to index on each particular topic. COUGAR is based on a game-theoretic analysis of heterogeneous search environments, and uses reinforcement learning techniques to exploit the sub-optimal behaviour of its competitors.


Nash Equilibrium Search Engine User Query Index Size Sample Trial 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Gravano, L., Garcia-Molina, H.: GlOSS: Text-source discovery over the Internet. ACM Trans. on Database Systems 24, 229–264 (1999)CrossRefGoogle Scholar
  2. 2.
    Rubinstein, A.: Modelling Bounded Rationality. The MIT Press, Cambridge (1997)Google Scholar
  3. 3.
    Khoussainov, R., Kushmerick, N.: Performance management in competitive distributedWeb search. In: Proc. of the IEEE/WIC Intl. Conf. on Web Intelligence (2003) (to appear)Google Scholar
  4. 4.
    Risvik, K., Michelsen, R.: Search engines and web dynamics. Computer Networks 39 (2002)Google Scholar
  5. 5.
    van Rijsbergen, C.J.: Information Retrieval, 2nd edn. Butterworths (1979)Google Scholar
  6. 6.
    Chakrabarti, S., van den Berg, M., Dom, B.: Focused crawling: A new approach to topicspecific Web resource discovery. In: Proc. of the 8th WWW Conf. (1999) Google Scholar
  7. 7.
    Osborne, M., Rubinstein, A.: A Course in Game Theory. The MIT Press, Cambridge (1999)Google Scholar
  8. 8.
    Conitzer, V., Sandholm, T.: Complexity results about Nash equilibria. Technical Report CMU-CS-02-135, Carnegie Mellon University (2002)Google Scholar
  9. 9.
    Robinson, J.: An iterative method of solving a game. Annals of Mathematics 54 (1951)Google Scholar
  10. 10.
    Carmel, D., Markovitch, S.: Learning models of intelligent agents. In: Proc. of the 13th National Conf. on AI (1996) Google Scholar
  11. 11.
    Peshkin, L.: Reinforcement Learning by Policy Search. PhD thesis, MIT (2002)Google Scholar
  12. 12.
    Stonebraker, M., Devine, R., Kornacker, M., Litwin, W., Pfeffer, A., Sah, A., Staelin, C.: An economic paradigm for query processing and data migration in Mariposa. In: Proc. of the 3rd Intl. Conf. on Parallel and Distributed Information Systems, pp. 58–67 (1994)Google Scholar
  13. 13.
    Greenwald, A., Kephart, J., Tesauro, G.: Strategic pricebot dynamics. In: Proc. of the 1st ACM Conf. on Electronic Commerce, pp. 58–67 (1999)Google Scholar
  14. 14.
    Greenwald, A., Kephart, J.: Shopbots and pricebots. In: Proc. of the 16th Intl. Joint Conf. on AI, pp. 506–511 (1999)Google Scholar
  15. 15.
    Bowling, M., Veloso, M.: Rational and convergent learning in stochastic games. In: Proc. of the 17th Intl. Joint Conf. on AI (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Rinat Khoussainov
    • 1
  • Nicholas Kushmerick
    • 1
  1. 1.Department of Computer ScienceUniversity College DublinDublin 4Ireland

Personalised recommendations