Abstract
In-place and merge-based index maintenance are the two main competing strategies for on-line index construction in dynamic information retrieval systems based on inverted lists. Motivated by recent results for both strategies, we investigate possible combinations of in-place and merge-based index maintenance. We present a hybrid approach in which long posting lists are updated in-place, while short lists are updated using a merge strategy. Our experimental results show that this hybrid approach achieves better indexing performance than either method (in-place, merge-based) alone.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Büttcher, S., Clarke, C.L.A.: Indexing Time vs. Query Time Trade-offs in Dynamic Information Retrieval Systems. In: Proc. of the 14th ACM Conf. on Information and Knowledge Management (CIKM 2005), Bremen, Germany (2005)
Büttcher, S., Clarke, C.L.A.: Memory Management Strategies for Single-Pass Index Construction in Text Retrieval Systems. UW-TR-CS-2005-32. Technical report, University of Waterloo, Canada (October 2005)
Clarke, C., Craswell, N., Soboroff, I.: The TREC Terabyte Retrieval Track. SIGIR Forum 39(1), 25 (2005)
Cutting, D.R., Pedersen, J.O.: Optimization for Dynamic Inverted Index Maintenance. In: Proceedings of the 13th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1990), pp. 405–411. ACM Press, New York (1990)
Heinz, S., Zobel, J.: Efficient Single-Pass Index Construction for Text Databases. Journal of the American Society for Information Science and Technology 54(8), 713–729 (2003)
Lester, N., Moffat, A., Zobel, J.: Fast On-Line Index Construction by Geometric Partitioning. In: Proceedings of the 14th ACM Conference on Information and Knowledge Management (CIKM 2005), Bremen, Germany (2005)
Lester, N., Zobel, J., Williams, H.E.: In-Place versus Re-Build versus Re- Merge: Index Maintenance Strategies for Text Retrieval Systems. In: CRPIT 26: Proceedings of the 27th Conference on Australasian Computer Science, pp. 15–23. Australian Computer Society, Inc. (2004)
Moffat, A.: Economical Inversion of Large Text Files. Computing Systems 5(2), 125–139 (1992)
Moffat, A., Bell, T.C.: In-Situ Generation of Compressed Inverted Files. Journal of the American Society of Information Science 46(7), 537–550 (1995)
Shieh, W.-Y., Chung, C.-P.: A Statistics-Based Approach to Incrementally Update Inverted Files. Inf. Processing and Management 41(2), 275–288 (2005)
Shoens, K.A., Tomasic, A., Garc´ıa-Molina, H.: SyntheticWorkload Performance Analysis of Incremental Updates. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1994), pp. 329–338 (1994)
Tomasic, A., García-Molina, H., Shoens, K.: Incremental Updates of Inverted Lists for Text Document Retrieval. In: Proceedings of the 1994 ACM SIGMOD Conference, pp. 289–300. ACM Press, New York (1994)
Zobel, J., Heinz, S., Williams, H.E.: In-Memory Hash Tables for Accumulating Text Vocabularies. Information Processing Letters 80(6) (2001)
Zobel, J., Moffat, A., Ramamohanarao, K.: Inverted Files versus Signature Files for Text Indexing. ACM Trans. on Database Systems 23(4), 453–490 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Büttcher, S., Clarke, C.L.A. (2006). A Hybrid Approach to Index Maintenance in Dynamic Text Retrieval Systems. In: Lalmas, M., MacFarlane, A., Rüger, S., Tombros, A., Tsikrika, T., Yavlinsky, A. (eds) Advances in Information Retrieval. ECIR 2006. Lecture Notes in Computer Science, vol 3936. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11735106_21
Download citation
DOI: https://doi.org/10.1007/11735106_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33347-0
Online ISBN: 978-3-540-33348-7
eBook Packages: Computer ScienceComputer Science (R0)