Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Language Models

  • Djoerd Hiemstra
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_923

Synonyms

Generative models

Definition

A language model assigns a probability to a piece of unseen text, based on some training data. For example, a language model based on a big English newspaper archive is expected to assign a higher probability to “a bit of text” than to “aw pit tov tags,” because the words in the former phrase (or word pairs or word triples if so-called N-gram models are used) occur more frequently in the data than the words in the latter phrase. For information retrieval, typical usage is to build a language model for each document. At search time, the top ranked document is the one whose language model assigns the highest probability to the query.

Historical Background

The term language models originates from probabilistic models of language generation developed for automatic speech recognition systems in the early 1980s [9]. Speech recognition systems use a language model to complement the results of the acoustic modelwhich models the relation between words (or...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Allan J, Aslam J, Belkin N, Buckley C, Callan J, Croft B, Dumais S, Fuhr N, Harman D, Harper DJ, Hiemstra D, Hofmann T, Hovy E, Kraaij W, Lafferty J, Lavrenko V, Lewis D, Liddy L, Manmatha R, McCallum A, Ponte J, Prager J, Radev D, Resnik P, Robertson S, Rosenfeld R, Roukos S, Sanderson M, Schwartz R, Singhal A, Smeaton A, Turtle H, Voorhees E, Weischedel E, Xu J, Zhai CX, editors. Challenges in information retrieval and language modeling. SIGIR Forum. 2003;37(1):31–47.Google Scholar
  2. 2.
    Balog K, Azzopardi L, Rijke M. Formal models for expert finding in enterprise corpora. In: Proceedings of 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2006. p. 43–50.Google Scholar
  3. 3.
    Basharin GP, Langville AN, Naumov VA. The life and work of A.A. Markov. Linear Algebra Appl. 2004;386(1):3–26.MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Berger A, Lafferty J. Information retrieval as statistical translation. In: Proceedings of 22nd ACM Conference on Research and Development in Information Retrieval; 1999. p. 222–9.Google Scholar
  5. 5.
    Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Machine Learn Res. 2003;3(5):993–1022.zbMATHGoogle Scholar
  6. 6.
    Hiemstra D, Jong F. Disambiguation strategies for cross-language information retrieval. Lecture notes in computer science. In: Proceedings of the 3rd European Conference on Research and Advanced Technology for Digital Libraries; 1999. p. 274–93.CrossRefGoogle Scholar
  7. 7.
    Hiemstra D, Kraaij W. Twenty-one at TREC-7: ad-hoc and cross-language track. In: Proceedings of 7th Text Retrieval Conference; 1998. p. 227–38.Google Scholar
  8. 8.
    Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1999. p. 50–57.Google Scholar
  9. 9.
    Jelinek F. Statistical methods for speech recognition. Cambridge, MA: MIT Press; 1997.Google Scholar
  10. 10.
    Jin H, Schwartz R, Sista S, Walls F. Topic tracking for radio, TV broadcast and newswire. In: Proceedings of DARPA Broadcast News Workshop; 1999.Google Scholar
  11. 11.
    Kraaij W, Westerveld T, Hiemstra D. The importance of prior probabilities for entry page search. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2002. p. 27–34.Google Scholar
  12. 12.
    Kraft DH, Bruce Croft W, Harper DJ, Zobel J. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2001.Google Scholar
  13. 13.
    Lavrenko V, Croft WB. Relevance models in information retrieval. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Kluwer: Dordecht; 2003. p. 11–56.zbMATHCrossRefGoogle Scholar
  14. 14.
    Miller DRH, Leek T, Schwartz RM. A hidden Markov model information retrieval system. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1999. p. 214–21.Google Scholar
  15. 15.
    Ponte JM, Bruce CW. A language modeling approach to information retrieval. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1998. p. 275–81.Google Scholar
  16. 16.
    Schwartz RM, Sista S, Leek T. Unsupervised topic discovery. In: Proceedings of Language Models for Information Retrieval Workshop; 2001.Google Scholar
  17. 17.
    Shannon CE. A mathematical theory of communication. Bell Syst Tech J. 1948;27(379–423):623–56.MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Spitters M, Kraaij W. Language models for topic tracking. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Dordecht: Kluwer; 2003. p. 95–124.zbMATHGoogle Scholar
  19. 19.
    Xu J, Weischedel R. A probabilistic approach to term translation for cross-lingual retrieval. In: Bruce Croft W, Lafferty J, editors. Language modeling for information retrieval. Dordecht: Kluwer; 2003. p. 125–40.zbMATHCrossRefGoogle Scholar
  20. 20.
    Zhai C, Lafferty J. Model-based feedback in the language modeling approach to information retrieval. In: Proceedings of ACM International Conference on Information and Knowledge Management; 2001. p. 403–10.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TwenteEnschedeThe Netherlands

Section editors and affiliations

  • Giambattista Amati
    • 1
  1. 1.Fondazione Ugo BordoniRomeItaly