TF*IDF

Synonyms

Term frequency by inverse document frequency

Definition

A weighting function that depends on the term frequency (TF) in a given document calculated with its relative collection frequency (IDF). This weighting function is calculated as follows [1]. Assuming that term j occurs in at least one document d(dj ≠ 0), the inverse document frequency (IDF) would be

$$ {\mathrm{Log}}_2\left(N/{d}_j\right)+1={\mathrm{log}}_2N-{\mathrm{log}}_2{d}_j $$

The ratio dj/N is the fraction of documents in the collection that contain the term. The term frequency-inverse document frequency weight (TF*IDF) of term j in document i is defined by multiplying the term frequency by the inverse document frequency:

$$ {W}_{ij}={f_{\mathrm{ij}}}^{\ast}\left[{\mathrm{log}}_2N-{\mathrm{log}}_2{d}_j\right] $$

Where

• N: number of documents in the collection

• d_j: number of documents containing term j

• f_ij: frequency of term j in document i

• W_ij: is the weight of term j in document i

The use of the logarithm in the...