Skip to main content
SpringerLink
Log in

Document Length Normalization

  • Living reference work entry
  • First Online:
Encyclopedia of Database Systems

  • 74 Accesses

Synonyms

Term frequency normalization; Lengthnormalization

Definition

Document length normalization adjusts the term frequency or the relevance score in order to normalize the effect of document length on the document ranking.

Key Points

The reasons for employing a document length normalization method in an IR system are quite subtle. In general, the effect observed on the ranking by the presence of many lengthy documents in a collection is to favor their retrieval with respect to shorter documents.

Singhal, Buckley and Mitra gave the following two reasons for adopting a length normalization in the vector space model [4]:

  1. 1.

    The same term usually occurs repeatedly in long documents.

  2. 2.

    The vocabulary of a long document is usually large.

In 1994, Robertson and Walker also studied the effect of document length in the context of the probabilistic model. They observed that:

Some documents may simply cover more material than others, […], a long document covers a similar scope to a short...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Recommended Reading

  1. Amati G. Probabilistic models for information retrieval based on divergence from randomness [PhD thesis]. University of Glasgow: Department of Computing Science; 2003.

    Google Scholar 

  2. Robertson SE, Walker S. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1994. p. 232–41.

    Google Scholar 

  3. Robertson SE, Walker S, Jones S, Hancock-Beaulieu M. Okapi at trec-3. In: Proceedings of the 3rd Text Retrieval Conference; 1994.

    Google Scholar 

  4. Singhal A, Buckley C, Mitra M. Pivoted document length normalization. In: Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1996. p. 21–9.

    Google Scholar 

  5. Zhai C, Lafferty J. A study of smoothing methods for language models applied to ad hoc information retrieval. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 2001. p. 334–42.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Glasgow, Glasgow, UK

    Ben He

Authors
  1. Ben He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben He .

Editor information

Editors and Affiliations

  1. Georgia Institute of Technology College of Computing, Atlanta, Georgia, USA

    Ling Liu

  2. University of Waterloo School of Computer Science, Waterloo, Ontario, Canada

    M. Tamer Özsu

Section Editor information

  1. Fondazione Ugo Bordoni, Via B. Castiglione, 59, Rome, Italy

    Giambattista Amati

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Science+Business Media LLC

About this entry

Cite this entry

He, B. (2016). Document Length Normalization. In: Liu, L., Özsu, M. (eds) Encyclopedia of Database Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4899-7993-3_934-2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4899-7993-3_934-2

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, New York, NY

  • Online ISBN: 978-1-4899-7993-3

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation