Construction of Query Concepts in a Document Space Based on Data Mining Techniques

Abstract

We discuss the issue of reformulating the user’s initial query to improve the retrieval performance. We propose to refine the query by adding a set of query concepts that are meant to precisely denote the user’s information need. To extract the most probable query concepts, we first extract a set of features from each document using summarization, and classify the extracted features into a set of predefined categories from Yahoo!. Finally, we cluster these features into primitive (basic) concepts. For a new query, we select its most associated primitive concepts and generate all possible interpretations of the query. The most probable interpretations are chosen as query concepts and are added to the initial query during the reformulation process. Our experiments are performed on the TREC 8 collection. The experimental evaluation shows that our query concept approach is as good as current query reformulation approaches, while being particularly effective for poorly performing queries. We also show that various data mining techniques could be helpful to generate the primitive concepts more effectively.