Advertisement

Parallel Information Retrieval on an SCI-Based PC-NOW

  • Sang-Hwa Chung
  • Hyuk-Chul Kwon
  • Kwang Ryel Ryu
  • Han-Kook Jang
  • Jin-Hyuk Kim
  • Cham-Ah Choi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1800)

Abstract

This paper presents an efficient parallel information retrieval (IR) system which provides fast information service for the Internet users on low-cost high-performance PC-NOW environment. The IR system is implemented on a PC cluster based on the Scalable Coherent Interface (SCI), a powerful interconnecting mechanism for both shared memory models and message passing models. In the IR system, the inverted-index file (IIF) is partitioned into pieces using a greedy declustering algorithm and distributed to the cluster nodes to be stored on each node’s hard disk. For each incoming user’s query with multiple terms, terms are sent to the corresponding nodes which contain the relevant pieces of the IIF to be evaluated in parallel. According to the experiments, the IR system outperforms an MPI-based IR system using Fast Ethernet as an interconnect. Speed- up of up to 4.0 was obtained with an 8-node cluster in processing each query on a 500,000-document IIF.

Keywords

Relevance Feedback Query Term Processing Node Disk Access Distribute Shared Memory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    IEEE, “MYRINET: A GIGABIT PER SECOND LOCAL AREA NETWORK”, IEEE-Micro, Vol.15, No.1, February 1995, pp.29–36.CrossRefGoogle Scholar
  2. 2.
    “Active Messages: a Mechanism for Integrated Communication and Computation”, Thorsten von Eicken and David Culler, et. al., 1992.Google Scholar
  3. 3.
    “Fast Messages (FM): Efficient, Portable Communication for Workstation Clusters and Massively-Parallel Processors”, IEEE Concurrency, vol. 5, No. 2, April-June 1997, pp. 60–73. (Pakin, Karamcheti & Chien)Google Scholar
  4. 4.
    “U-Net: A User-Level Network Interface for Parallel and Distributed Computing”, Anindya Basu, Vineet Buch, Werner Vogels, Thorsten von Eicken, Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP), Copper Mountain, Colorado, December 3–6, 1995.Google Scholar
  5. 5.
  6. 6.
    “NUMA-Q: An SCI based Enterprise Server”, http://www.sequent.com/products/highend_srv/sci_wp1.html
  7. 7.
    “SCI Interconnect Chipset and Adapter: Building Large Scale Enterprise Servers with Pentium Pro SHV Nodes”, http://www.dg.com/about/html/sci_interconnect_chipset_and_a.html
  8. 8.
    S.H. Park, H.C. Kwon, “An Improved Relevance Feedback for Korean Information Retrieval System”, Proc. of the 16th IASTED International Conf. Applied Informatics, IASTED/ACTA Press, pp.65–68, Garmisch-Partenkirchen, Germany, February 23–25, 1998Google Scholar
  9. 9.
    Salton, G. and Buckley, C., “Improving retrieval performance by relevance feedback”, American Society for Information Science, 41,4, pp. 288–297, 1990.CrossRefGoogle Scholar
  10. 10.
  11. 11.
    “A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard”, http://www-unix.mcs.anl.gov/mpi/mpich/docs.html

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Sang-Hwa Chung
    • 1
  • Hyuk-Chul Kwon
    • 1
  • Kwang Ryel Ryu
    • 1
  • Han-Kook Jang
    • 1
  • Jin-Hyuk Kim
    • 1
  • Cham-Ah Choi
    • 1
  1. 1.Division of Computer Science and EngineeringPusan National UniversityPusanKorea

Personalised recommendations