Abstract
Reaching high precision and recall rates in the results of term-based queries on text collections is becoming more and more crucial, as long as the amount of available documents increases and their quality tends to decrease. In particular, retrieval techniques based on the strict correspondence between terms in the query and terms in the documents miss important and relevant documents where it just happens that the terms selected by their authors are slightly different than those used by the final user that issues the query. Our proposal is to explicitly consider term co-occurrences when building the vector space. Indeed, the presence in a document of different but related terms to those in the query should strengthen the confidence that the document is relevant as well. Missing a query term in a document, but finding several terms strictly related to it, should equally support the hypothesis that the document is actually relevant. The computational perspective that embeds such a relatedness consists in matrix operations that capture direct or indirect term co-occurrence in the collection. We propose two different approaches to enforce such a perspective, and run preliminary experiments on a prototypical implementation, suggesting that this technique is potentially profitable.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Buzydlowski, J.W., White, H.D., Lin, X.: Term co-occurrence analysis as an interface for digital libraries. In: Proceedings of the Joint Conference on Digital Libraries (2001)
Ide, N., Véronis, J.: Word sense disambiguation: The state of the art. Computational Linguistics 24, 1–40 (1998)
Karpys, G., Han, E.: Concept indexing: A fast dimensionality reduction algorithm with applications to document retrieval and categorization. Technical Report 00-016, Minnesota University Minneapolis (March 2000)
Kontostathis, A., Pottenger, W.M.: Detecting patterns in the lsi term-term matrix. In: Proceedings of the ICDM 2002 Workshop on Foundations of Data Mining and Discovery (2002)
Kontostathis, A., Pottenger, W.M.: A framework for understanding latent semantic indexing (lsi) performance. Inf. Process. Manage. 42(1), 56–73 (2006)
Krovetz, R.: More than one sense per discourse. In: NEC Princeton NJ Labs., Research Memorandum (1998)
Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review 104, 111–140 (1997)
Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis. Discourse Processes 25, 259–284 (1998)
Magnini, B., Cavaglià, G.: Integrating subject field codes into wordnet. In: Proceedings of LREC 2000, Second International Conference on Language Resources and Evaluation, pp. 1413–1418 (2000)
Miller, G.A.: Wordnet: A lexical database for English. Communications of the ACM 38(11), 39–41 (1995)
O’Brien, G.W.: Information management tools for updating an svd-encoded indexing scheme. Technical Report CS-94-258, University of Tennessee, Knoxville (October 1994)
Peat, H.J., Willet, P.: The limitations of term co-occurrence data for query expansion in document retrieval systems. Journal of the American Society for Information Science 42(5), 378–383 (1991)
Rapp, R.: The computation of word associations: Comparing syntagmatic and paradigmatic approaches. In: Proceedings of the 29th International Conference on Computational Linguistics (2002)
Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)
Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Communications of the ACM 18(11), 613–620 (1975)
Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34, 1–47 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ferilli, S., Biba, M., Basile, T.M.A., Esposito, F. (2010). Using Explicit Word Co-occurrences to Improve Term-Based Text Retrieval. In: Agosti, M., Esposito, F., Thanos, C. (eds) Digital Libraries. IRCDL 2010. Communications in Computer and Information Science, vol 91. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15850-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-15850-6_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15849-0
Online ISBN: 978-3-642-15850-6
eBook Packages: Computer ScienceComputer Science (R0)