Advertisement

Research of Distributed Index Based on Lucene

  • Zhuang ChenEmail author
  • Chonglai Zhu
  • Wei Cheng
  • Qiulin Song
  • Shibang Cai
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 139)

Abstract

Inverted index is the mainstay technology for full-text retrieval, however, there exist some problems such as low efficiency of index construction, updating and high Maintenance cost. In order to improve the retrieval performance of large scale indexing, the strategy of distributed indexing is proposed in this paper. This paper gives a detailed description of building index structures using Map Reduce, and compares the performance of building index and query between distributed search system and central search system. Experimental results show that the proposed scheme greatly reduces the time of index construction and query time.

Keywords

Inverted index Search engine Distributed index 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Brin, S., Page, L.: The anatomy of a large scale hypertextual web search engine. In: 7th International WWW Conference (1998)Google Scholar
  2. 2.
    Justin, Z., Moffat, A.: Inverted Files for Text Search Engines. ACM Computing Surveys 38(2), 1–56 (2006)Google Scholar
  3. 3.
    Dean, J., Ghemawat, S.: MapReduce: Simplied Data Processingon Large Clusters. In: Proc. of OSDI 2004, pp. 137–150 (2004)Google Scholar
  4. 4.
    Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large scale peer-to-peer systems. In: Proc. IFIP/ACM Middleware, Heidelberg, Germany (November 2001)Google Scholar
  5. 5.
    Clarke, I., Sandberg, O., Wiley, B., Hong, T.: Freenet: A distributed anonymous information storage and retrieval system. In: Proc. of the ICSI Workshop on Design Issues in Anonymity and Unobservability, Berkeley, CA (June 2000)Google Scholar
  6. 6.
    The Apache Jakarta Project: Lucene, http://lucene.apache.org/
  7. 7.
    Apache Hadoop. Hadoop [EB/OL]. [2009203206], http://hadoop.apache.Org/
  8. 8.
    Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Computing Surveys 38(2), Article 6 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Zhuang Chen
    • 1
    Email author
  • Chonglai Zhu
    • 1
  • Wei Cheng
    • 1
  • Qiulin Song
    • 1
  • Shibang Cai
    • 1
  1. 1.Institute of Computer Science and EngineeringChongqing University of TechnologyChongQingChina

Personalised recommendations