A Three Level Search Engine Index Based in Query Log Distribution

  • Ricardo Baeza-Yates
  • Felipe Saint-Jean
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2857)


Queries to a search engine follow a power-law distribution, which is far from uniform. Hence, it is natural to adapt a search engine index to the query distribution. In this paper we present a three level memory organization for a search engine inverted file index that includes main and secondary memory, as well as precomputed answers, such that the use of main memory and the answer time are significantly improved. We include experimental results as well as an analytical model.


Main Memory Answer Time Secondary Memory Inverted List Query Word 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, p. 513. ACM Press/Addison-Wesley, England (1999), Google Scholar
  2. 2.
    Beeferman, D., Berger, A.: Agglomerative clustering of a search engine query log. In: Proceedings on the 2000 Conference on Knowledge Discovery and Data Mining, Boston, MA, August 2000, pp. 407–416 (2000)Google Scholar
  3. 3.
    Markatos, E.P.: On Caching Search Engine Query Results. In: Proceedings of the 5th International Web Caching and Content Delivery Workshop (May 2000)Google Scholar
  4. 4.
    Saraiva, P.C., de Moura, E.S., Ziviani, N., Meira, W., Fonseca, R., Ribeiro-Neto, B.: Rank-preserving two-level caching for scalable search engines. In: Proceedings of the 24th annual international ACM SIGIR on Research and development in information retrieval, New Orleans, LA, September 2001, pp. 51–58 (2001)Google Scholar
  5. 5.
    Silverstein, C., Henzinger, M., Marais, H., Moricz, M.: Analysis of a Very Large AltaVista Query Log. SIGIR Forum 33(3) (1999)Google Scholar
  6. 6.
    Spink, A., Jansen, B.J., Wolfram, D., Saracevic, T.: From E-Sex to E-Commerce: Web Search Changes. IEEE Computer 35(3), 107–109 (2002)Google Scholar
  7. 7.
    Xie, Y., O’Hallaron, D.: Locality in Search Engine Queries and Its Implications for Caching. In: Infocom 2002 (2002)Google Scholar
  8. 8.
    Zhang, D., Dong, Y.: A Novel Web Usage Mining Approach For Search Engine. Computer Networks (2003)Google Scholar
  9. 9.
    Zipf, G.: Selective Studies and the Principle of Relative Frequency in Language. Harvard University Press, Cambridge (1932)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Ricardo Baeza-Yates
    • 1
  • Felipe Saint-Jean
    • 1
  1. 1.Center for Web Research, Department of Computer ScienceUniversidad de Chile, Blanco EncaladaSantiagoChile

Personalised recommendations