Query Performance Prediction Using Joint Inverse Document Frequency of Multiple Terms

Abstract

In an information retrieval system, predicting query performance, for keyword based queries is important in giving early feedback to the user which can result in an improved query which in turn results in a better query result. There exists clarity score based and ranking robustness score based techniques to solve this problem. Both these, eventhough shows good performance, suffers from high computational time needs and are post-retrieval methods. In contrast to this, there do exist several pre-retrieval parameters which can judge the query without executing it. Pre-retrieval parameters based on distribution of information in query terms, which basically depends on inverse document frequency (idf) of query terms, are shown to be good predictors. Among these, the standard-deviation of idf values of query terms is known to be better. This paper generalizes this and proposes to use joint idf for a set of terms together, than using each term’s idf individually. Empirical studies are done using some standard data sets. The parameters based on the proposed method are shown to be better than the previous method which is nothing but a special case of the proposed method.