A Useful Method for Producing Competitive Ad Hoc Task Results

Abstract

This paper reports the final results of our experiments involving the 2009 INEX Ad-Hoc Track and describes the methodology upon which our current, 2010 experiments are built. In 2009, our INEX investigations centered on indentifying a methodology for producing what we have referred to as improved focused elements—i.e., elements which when evaluated are competitive with others in the upper ranges of the official rankings. Our 2009 INEX paper [5] describes our approach to producing such elements, which is based on the combination of traditional document retrieval (to identify the document set of interest to the query) with our method of dynamic element retrieval (to generate and retrieve the elements of the document(s) so identified) and the subsequent application of a specific focusing technique for the Focused and Relevant-in-Context tasks (to select the focused elements). The system is based on the Vector Space Model [10]; basic functions are performed using the Smart experimental retrieval system [9]. In this paper, we report the final results of these experiments, applied to the INEX 2009 Thorough, Focused and Relevant-in-Context tasks. Results show that this approach produces highly ranked results for all three of these Ad Hoc tasks. Significance tests, applied to these results as compared to the top-ranked runs, show in which cases statistically significant results are obtained. Our 2010 work is ongoing at present.