Abstract
We propose a parallel MPI search engine that is capable of automatically switching between asynchronous message passing and bulk-synchronous message passing modes of operation. When the observed query traffic is small or moderate the standard multiple managers/workers thread based model of message passing is applied for processing the queries. However, when the query traffic increases a round-robin based approach is applied in order to prevent from unstable behavior coming from queries demanding the use of a large amount of resources in computation, communication and disk accesses. This is achieved by (i) a suitable object-oriented multi-threaded MPI software design and (ii) an “atomic” organization of the query processing which allows the use of a novel control strategy that decides the proper mode of operation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Badue, C., Baeza-Yates, R., Ribeiro, B., Ziviani, N.: Distributed query processing using partitioned inverted files. In: Eighth Symposium on String Processing and Information Retrieval (SPIRE 2001), pp. 10–20 (2001)
Cacheda, F., Plachouras, V., Ounis, I.: Performance analysis of distributed architectures to index one terabyte of text. In: McDonald, S., Tait, J. (eds.) Proc. ECIR European Conf. on IR Research, Sunderland, UK, pp. 395–408 (2004)
Jeong, B.S., Omiecinski, E.: Inverted file partitioning schemes in multiple disk systems. IEEE Transactions on Parallel and Distributed Systems 16, 142–153 (1995)
MacFarlane, A.A., McCann, J.A., Robertson, S.E.: Parallel search using partitioned inverted files. In: 7th International Symposium on String Processing and Information Retrieval, pp. 209–220. IEEE CS Press, Los Alamitos (2000)
Moffat, W., Webber, J., Zobel, B.-Y.R.: A pipelined architecture for distributed text query evaluation. Information Retrieval (2006)
Persin, M., Zobel, J., Sacks-Davis, R.: Filtered document retrival with frequency-sorted indexes. Journal of the American Society for Information Science 47(10), 749–764 (1996)
Ribeiro-Neto, B.A., Barbosa, R.A.: Query performance for tightly coupled distributed digital libraries. In: Third ACM Conference on Digital Libraries, pp. 182–190. ACM Press, New York (1998)
Stanfill, C.: Partitioned posting files: a parallel inverted file structure for information retrieval. In: 13th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Brussels, Belgium, pp. 413–428. ACM Press, New York (1990)
Valiant, L.: A bridging model for parallel computation. Comm. ACM 33, 103–111 (1990)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Marin, M., Costa, V.G. (2007). (Sync|Async) + MPI Search Engines. In: Cappello, F., Herault, T., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2007. Lecture Notes in Computer Science, vol 4757. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75416-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-75416-9_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75415-2
Online ISBN: 978-3-540-75416-9
eBook Packages: Computer ScienceComputer Science (R0)