Skip to main content
SpringerLink
Log in

An Empirical Study of Query Expansion and Cluster-Based Retrieval in Language Modeling Approach

  • Conference paper
Information Retrieval Technology (AIRS 2005)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 3689))

Included in the following conference series:

Abstract

In information retrieval, the word mismatch problem is a critical issue. To resolve the problem, several techniques have been developed, such as query expansion, cluster-based retrieval, and dimensionality reduction. Of these techniques, this paper performs an empirical study on query expansion and cluster-based retrieval. We examine the effect of using parsimony in query expansion and the effect of clustering algorithms in cluster-based retrieval. In addition, query expansion and cluster-based retrieval are compared, and their combinations are evaluated in terms of retrieval performance. By performing experimentation on seven test collections of NTCIR and TREC, we conclude that 1) query expansion using parsimony is well performed, 2) cluster-based retrieval by agglomerative clustering is better than that by partitioning clustering, and 3) query expansion is generally more effective than cluster-based retrieval in resolving the word-mismatch problem, and finally 4) their combinations are effective when each method significantly improves baseline performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Berger, A., Lafferty, J.: Information Retrieval as Statistical Translation. In: Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 222–229 (1999)

    Google Scholar 

  2. Gao, J., Nie, J., Wu, G., Cao, G.: Dependence Language Model for Information Retrieval. In: Proceedings of 27nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 170–177 (2004)

    Google Scholar 

  3. Hiemstra, D., Robertson, S., Zaragoza, H.: Parsimonious Language Models for Information Retrieval. In: Proceedings of 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 178–185 (2004)

    Google Scholar 

  4. Hiemstra, D.: Term Specific Smoothing for Language Modeling Approach to Information Retrieval: The Importance of a Query Term. In: Proceedings of 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 35–41 (2002)

    Google Scholar 

  5. Hiemstra, D.: Using Language Models for Information Retrieval. In PhD Thesis, University of Twente (2001)

    Google Scholar 

  6. Hoffman, T.: Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning 42(1-2), 177–196 (2001)

    Article  Google Scholar 

  7. Kamvar, S., Klein, D., Manning, C.: Interpreting and Extending Classical Agglomerative Clustering Algorithms using a Model-based Approach. In: Proceedings of 19th International Conference on Machine Learning, pp. 283–290 (2002)

    Google Scholar 

  8. Kurland, O., Lee, L.: Corpus Structure, Language Models, and Ad hoc Information Retrieval. In: Proceedings of 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 194–201 (2004)

    Google Scholar 

  9. Lafferty, J., Zhai, C.: Document Language Models, Query Models, and Risk Minimization for Information Retrieval. In: Proceedings of 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 111–119 (2001)

    Google Scholar 

  10. Lavrenko, V., Choquette, M., Croft, W.: Cross-Lingual Relevance Model. In: Proceedings of 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 175–182 (2002)

    Google Scholar 

  11. Lavrenko, V., Croft, B.: Relevance-based Language Models. In: Proceedings of 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 120–127 (2001)

    Google Scholar 

  12. Liu, X.: Cluster-Based Retrieval Using Language Models. In: Proceedings of 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 186–193 (2004)

    Google Scholar 

  13. Lee, J., Cho, H., Park, H.: n-Gram-based Indexing for Korean Text Retrieval. Information Processing & Management 35(4), 427–441 (1999)

    Article  MathSciNet  Google Scholar 

  14. Miller, D., Leek, T., Schwartz, R.: A Hidden Markov Model Information Retrieval System. In: Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 214–221 (1999)

    Google Scholar 

  15. Na, S., Kang, I., Kang, S., Lee, J.: Estimation of Query Model from Parsimonious Translation Model. In: Myaeng, S.-H., Zhou, M., Wong, K.-F., Zhang, H.-J. (eds.) AIRS 2004. LNCS, vol. 3411, pp. 239–250. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  16. Ponte, A., Croft, J.: A Language Modeling Approach to Information Retrieval. In: Proceedings of 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 275–281 (1998)

    Google Scholar 

  17. Ponte, A.: A Language Modeling Approach to Information Retrieval. PhD thesis, University of Massachusetts (1998)

    Google Scholar 

  18. Robertson, S., Sparck Jones, K.: Relevance Weighting of Search Terms. Journal of the American Society for Information Science 27(3), 143–160 (1979)

    Google Scholar 

  19. Song, F., Croft, W.: A General Language Model for Information Retrieval. In: Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 279–280 (1999)

    Google Scholar 

  20. Zhai, C., Lafferty, J.: Model-based Feedback in the Language Modeling Approach to Information Retrieval. In: Proceedings of the 10th International Conference on Information and Knowledge Management, pp. 430–410 (2002)

    Google Scholar 

  21. Zhai, C., Lafferty, J.: A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval. In: Proceedings of 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 334–342 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Electrical and Computer Engineering, POSTECH, AITrc, Republic of Korea

    Seung-Hoon Na, In-Su Kang, Ji-Eun Roh & Jong-Hyeok Lee

Authors
  1. Seung-Hoon Na
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. In-Su Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji-Eun Roh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jong-Hyeok Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Pohang University of Science and Technology, San 31, Hyoja-dong, Nam-gu, 790-784, Pohang, Korea

    Gary Geunbae Lee

  2. Computer and Communication Media Research, NEC Corp., Miyazaki 4-1-1, Miyamae-ku, 216-8555, Kawasaki, Japan

    Akio Yamada

  3. Human-Computer Communications Laboratory, Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong

    Helen Meng

  4. School of Engineering, Information and Communications University, 119, Munjiro, Yuseong-gu, 305-732, Daejeon, Korea

    Sung Hyon Myaeng

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Na, SH., Kang, IS., Roh, JE., Lee, JH. (2005). An Empirical Study of Query Expansion and Cluster-Based Retrieval in Language Modeling Approach. In: Lee, G.G., Yamada, A., Meng, H., Myaeng, S.H. (eds) Information Retrieval Technology. AIRS 2005. Lecture Notes in Computer Science, vol 3689. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11562382_21

Download citation

  • DOI: https://doi.org/10.1007/11562382_21

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-29186-2

  • Online ISBN: 978-3-540-32001-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation