Abstract
The aim of query-based sampling is to obtain a sufficient, representative sample of an underlying (text) collection. Current measures for assessing sample quality are too coarse grain to be informative. This paper outlines a measure of finer granularity based on probabilistic topic models of text. The assumption we make is that a representative sample should capture the broad themes of the underlying text collection. If these themes are not captured, then resource selection will be affected in terms of performance, coverage and reliability. For example, resource selection algorithms that require extrapolation from a small sample of indexed documents to determine which collections are most likely to hold relevant documents may be affected by samples which do not reflect the topical density of a collection. To address this issue we propose to measure the relative entropy between topics obtained in a sample with respect to the complete collection. Topics are both modelled from the collection and inferred in the sample using latent Dirichlet allocation. The paper outlines an analysis and evaluation of this methodology across a number of collections and sampling algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Callan, J.P.: Distributed information retrieval. In: Advances in information retrieval, pp. 127–150. Kluwer Academic Publishers, Dordrecht (2000)
Si, L., Callan, J.P.: Relevant document distribution estimation method for resource selection. In: ACM SIGIR 2003, Ontario Canada, pp. 298–305 (2003)
Callan, J.P., Connell, M.: Query-based sampling of text databases. ACM Transactions of Information Systems 19, 97–130 (2001)
Craswell, N., Bailey, P., Hawking, D.: Server selection on the world wide web. In: DL 2000: Proceedings of the fifth ACM conference on Digital libraries, pp. 37–46 (2000)
Gravano, L., Ipeirotis, P.G., Sahami, M.: Qprober: A system for automatic classification of hidden-web databases. ACM Trans. Inf. Syst. 21, 1–41 (2003)
Ipeirotis, P.G., Agichtein, E., Jain, P., Gravano, L.: To search or to crawl?: towards a query optimizer for text-centric tasks. In: SIGMOD 2006: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pp. 265–276 (2006)
Shokouhi, M., Scholer, F., Zobel, J.: Sample sizes for query probing in uncooperative distributed information retrieval. In: Zhou, X., Li, J., Shen, H.T., Kitsuregawa, M., Zhang, Y. (eds.) APWeb 2006. LNCS, vol. 3841, pp. 63–75. Springer, Heidelberg (2006)
Hawking, D., Thomas, P.: Server selection methods in hybrid portal search. In: ACM SIGIR 2005, Brazil, pp. 75–82 (2005)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Baillie, M., Azzopardi, L., Crestani, F.: Adaptive query-based sampling of distributed collections. In: Crestani, F., Ferragina, P., Sanderson, M. (eds.) SPIRE 2006. LNCS, vol. 4209, pp. 316–328. Springer, Heidelberg (2006)
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)
Ipeirotis, P.G., Ntoulas, A., Cho, J., Gravano, L.: Modeling and managing content changes in text databases. In: ICDE 2005, pp. 606–617 (2005)
Shokouhi, M., Baillie, M., Azzopardi, L.: Updating collection representations for federated search. In: SIGIR 2007, Amsterdam, Netherlands, pp. 511–518 (2007)
Thomas, P., Hawking, D.: Evaluating sampling methods for uncooperative collections. In: SIGIR 2007, Amsterdam, Netherlands, pp. 503–510 (2007)
Azzopardi, L., Vinay, V.: Accessibility in information retrieval. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol. 4956, pp. 482–489. Springer, Heidelberg (2008)
Bar-Yossef, Z., Gurevich, M.: Random sampling from a search engine’s index. In: ACM WWW 2006, pp. 367–376 (2006)
Griffiths, T.L., Steyvers, M.: Finding scientific topics. Proceedings of the National Academy of Science 101, 5228–5235 (2004)
Manning, C.D., Raghavan, P., Schutze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)
Hofmann, T.: Probabilistic latent semantic indexing. In: ACM SIGIR 1999, Berkley, USA, pp. 50–57 (1999)
Wei, X., Croft, W.B.: Lda-based document models for ad-hoc retrieval. In: ACM SIGIR 2006, Seattle, USA, pp. 178–185 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Baillie, M., Carman, M.J., Crestani, F. (2009). A Topic-Based Measure of Resource Description Quality for Distributed Information Retrieval. In: Boughanem, M., Berrut, C., Mothe, J., Soule-Dupuy, C. (eds) Advances in Information Retrieval. ECIR 2009. Lecture Notes in Computer Science, vol 5478. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00958-7_43
Download citation
DOI: https://doi.org/10.1007/978-3-642-00958-7_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00957-0
Online ISBN: 978-3-642-00958-7
eBook Packages: Computer ScienceComputer Science (R0)