Skip to main content
SpringerLink
Log in
Book cover

New Directions in Cognitive Information Retrieval pp 61–80Cite as

Integration Approaches to Relevance

  • Chapter

Part of the book series: The Information Retrieval Series ((INRE,volume 19))

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aalbersberg, I. J. (1992). Incremental Relevance Feedback. Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in IR, Copenhagen, 11–22.

    Google Scholar 

  • Barry, C. L. and Schamber, L. (1998). Users’ Criteria for Relevance Evaluation: A Cross-situational Comparison. Information Processing and Management. 34(2/3), 219–236.

    Article  Google Scholar 

  • Beaulieu, M. and Jones, S. (1998). Interactive Searching and Interface Issues in the Okapi Best Match Retrieval System. Interacting with Computers, 10(3), 237–248.

    Article  Google Scholar 

  • Borlund, P. (2003a). The IIR Evaluation Model: a Framework for Evaluation of Interactive IR Systems. Information Research, 8(3), Paper No. 152. Available at: http://informationr.net/ir/8-3/paper152.html

    Google Scholar 

  • Borlund, P. (2003b). The Concept of Relevance in IR. Journal of the American Society for Information Science and Technology, 54(10), 913–925.

    Article  Google Scholar 

  • Campbell, I. (2000). Interactive Evaluation of the Ostensive Model, Using a New Test-Collection of Images with Multiple Relevance Assessments. Journal of Information Retrieval, 2(1), 89–114.

    Article  Google Scholar 

  • Choi, Y. and Rasumussen, E. M. (2002). Users’ Relevance Criteria in Image Retrieval in American History. Information Processing and Management, 38(5), 695–726.

    Article  MATH  Google Scholar 

  • De Vries, A. P., Kazai, G., and Lalmas, M. (2004). Tolerance to Irrelevance: A User-effort Oriented Evaluation of Retrieval Systems without Predefined Retrieval Unit. In Proceedings of RIAO 2004 (Recherche d’Information Assistée par Ordinateur) (pp. 163–173). Avignon.

    Google Scholar 

  • Eisenberg, M. and Barry, C. (1988). Order Effects: a Study of the Possible Influence of Presentation Order on User Judgements of Document Relevance. Journal of the American Society of Information Science, 39(5), 293–300.

    Article  Google Scholar 

  • Eisenberg, M. and Hu, X. (1987). Dichotomous Relevance Judgments and the Evaluation of Information Systems. In Proceedings of the American Society for Information Science 50th Annual Meeting, (pp. 66–69).

    Google Scholar 

  • Florance, V. and Marchionini, G. (1995). Information Processing in the Context of Medical Care. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in IR (pp. 158–163). Seattle.

    Google Scholar 

  • Greisdorf, H. (2003). Relevance Thresholds: A Multi-Stage Predictive Model of How Users Evaluate Information. Information Processing and Management, 39(3), 403–423.

    Article  MATH  Google Scholar 

  • Hersh, W. and Turpin, A. (2001). Why Batch and User Evaluations Do Not Give the Same Results. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in IR (pp. 225–231). New Orleans.

    Google Scholar 

  • Harter, S. P. (1996). Variations in Relevance Assessments and the Measurement of Retrieval Effectiveness. Journal of the American Society for Information Science and Technology, 47(1), 37–49.

    Article  Google Scholar 

  • Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedle, J. T. (2004). Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems, 22(1), 5–53.

    Article  Google Scholar 

  • Huang, M.-H. and Wang H.-Y. (2004). The Influence of Document Presentation Order and Number of Documents Judged on Users’ Judgements of Relevance. Journal of the American Society for Information Science and Technology, 55(11), 970–979.

    Article  MathSciNet  Google Scholar 

  • Ingwersen, P. (2002). Information Retrieval Interaction. Taylor Graham.

    Google Scholar 

  • Janes, J. (1991). The Binary Nature of Continuous Relevance Judgments: A Study of User’s Perceptions. Journal of the American Society for Information Science and Technology, 42(10), 754–756.

    Article  Google Scholar 

  • Järvelin, K. and Kekäläinen, J. (2000). IR Evaluation Methods for Retrieving Highly Relevant Documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development IR (pp. 41–48). Athens.

    Google Scholar 

  • Katter, R. V. (1968). The Influence of Scale Form on Relevance Judgments. Information Storage and Retrieval, 4, 1–11.

    Article  Google Scholar 

  • Kazai, G., Lalmas, M., and deVries, A. P. (2004). The Overlap Problem in Content-Oriented XML Retrieval. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 72–79). Sheffield.

    Google Scholar 

  • Kelly, D. and Belkin, N. J. (2004). Display time as implicit feedback: Understanding task effects. In Proceedings of the 27th Annual ACM International Conference on Research and Development in Information Retrieval (pp. 377–384). Sheffield.

    Google Scholar 

  • Lesk, M. E. and Salton, G. (1969). Relevance Assessments and Retrieval System Evaluation. Information Storage and Retrieval, 4, 343–359.

    Article  Google Scholar 

  • McLaughlin, K. L. and Sonnenwald, D. (2002). User Perspectives on Relevance Criteria: A Comparison Among Relevant, Partially Relevant and Not-Relevant Judgements. Journal of the American Society for Information Science and Technology, 53(5), 327–342.

    Article  Google Scholar 

  • Markkula, M., Tico, M., Sepponen, B., Nirkkonen, K., and Sormunen, E. (2001). A Test Collection for the Evaluation of Content-Based Image Retrieval Algorithms—A User and Task-Based Approach. Information Retrieval, 4(3/4), 275–293.

    Article  MATH  Google Scholar 

  • Maron, M. E. (1964). Mechanized Documentation: The Logic Behind a Probabilistic Interpretation. Statistical Association Methods For Mechanized Documentation. National Bureau of StandardsMiscellaneous Publications, 269, 9–13.

    Google Scholar 

  • Mizzaro, S. (1997). Relevance: The Whole History. Journal of the American Society for Information Science and Technology, 48(9), 810–832.

    Article  Google Scholar 

  • Mizzaro, S. (1999). Measuring the Agreement Among Relevance Judges. In Proceedings of the Final Mira Workshop (Mira’ 99). British Computer Society Electronic Workshops in Computing series. Glasgow. 1–13. Available at: http://ewic.bcs.org/conferences/1999/mira99/

    Google Scholar 

  • Oard, D. W., Mayfield, J., Kharevych, L., Strassel, S., Soergel, D., Doermann, D., Huang, X., Murray, G. C., Wang, J., Ramabhadran, B., Franz, M., and Gustman, S. (2004). Building an IR Test Collection for Spontaneous Conversational Speech. In Proceedings of the 27th Annual International Conference on Research and Development in Information Retrieval (pp. 41–48). Sheffield.

    Google Scholar 

  • Oddy, R. N. (1981). Laboratory Tests: Automatic Systems. In: Information Retrieval Experiment. In: K. Spark Jones (Editor). Butterworths, 156–179.

    Google Scholar 

  • Reid, J. (2000). A Task-Oriented Non-interactive Evaluation Methodology for IR Systems. Information Retrieval, 2(1), 115–129.

    Article  Google Scholar 

  • Ruthven, I. (2001). Abduction, Explanation and Relevance Feedback. Unpublished doctoral dissertation. University of Glasgow.

    Google Scholar 

  • Ruthven, I., Lalmas, M., and van Rijsbergen, C. J. (2002). Ranking Expansion Terms Using Partial and Ostensive Evidence. In Proceedings of the 4th International Conference on Conceptions of Library and Information Science (CoLIS 4): Emerging Frameworks (pp. 199–220). Seattle.

    Google Scholar 

  • Saracevic, T. (1975). Relevance: A Review of and a Framework for the Thinking on the Notion in Information Science. Journal of the American Society for Information Science, 26(6), 321–343.

    Google Scholar 

  • Saracevic, T. (1996). Relevance Reconsidered. In Proceedings of the Second Conference on Conceptions of Library and Information Science. Information Science: Integration in Perspectives. Copenhagen, 201–218.

    Google Scholar 

  • Salojärvi, J., Kojo, I., Simola, J., and Kaski, S. (2003). Can Relevance be Inferred From Eye Movements in IR? In Proceedings of the Workshop on Self-Organizing Maps (WSOM’03) (pp. 261–266). Hibikino.

    Google Scholar 

  • Sormunen, E. (2002). Liberal Relevance Criteria of TREC—Counting on Negligible Documents? In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in IR (pp. 324–330). Tampere.

    Google Scholar 

  • Spink, A. (1996). Study of Interactive Feedback During Mediated Information Retrieval. Journal of the American Society for Information Science, 47(8), 603–609.

    Article  Google Scholar 

  • Spink, A., Greisdorf, H., and Bateman, J. (1998). From Highly Relevant to Not Relevant: Examining Different Regions of Relevance. Information Processing and Management, 34(5), 599–621.

    Article  Google Scholar 

  • Sweeney, S. and Crestani, F. (2003). Supporting Searching on Small Screen Devices Using Summarization. In Proceedings of Mobile HCI 2003 International Workshop on Mobile and Ubiquitous Information Access. Lecture Notes in Computer Science, Vol. 2954 (pp. 187–201). Udine.

    Google Scholar 

  • Tombros, A. and Crestani, F. (2000). Users’ Perception of Relevance of Spoken Documents. Journal of the American Society for Information Science, 51(9), 929–939.

    Article  Google Scholar 

  • Tombros, A. Ruthven, I., and Jose, J. M. (2005). How Users Assess Web Pages for Information-seeking. Journal of the American Society for Information Science and Technology, 56(4), 327–344.

    Article  Google Scholar 

  • Tiamiyu, M. A. and Ajiferuke, I. Y. (1988). A Total Relevance and Document Interaction Effects Model for the Evaluation of IR Processes. Information Processing and Management, 24(4), 391–404.

    Article  Google Scholar 

  • Vakkari, P. (2000). Cognition and Changes of Search Terms and Tactics During Task Performance: a Longitudinal Study. In Proceedings of RIAO 2004 (Recherche d’Information Assistée par Ordinateur) (pp. 894–907). Paris.

    Google Scholar 

  • Vakkari, P. and Sormunen, E. (2004). The Influence of Relevance Levels on the Effectiveness of Interactive IR. Journal of the American Society for Information Science and Technology, 55(11), 963–969.

    Article  Google Scholar 

  • Voorhees, E. M. (2000). Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. Information Processing and Management, 36, 697–716.

    Article  Google Scholar 

  • Voorhess, E. M. (2001). Evaluation by Highly Relevant Documents. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 74–82). New Orleans.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Strathclyde, UK

    Ian Ruthven

Authors
  1. Ian Ruthven
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Information Technology, Queensland University of Technology, Brisbane, Australia

    Amanda Spink

  2. Graduate School of Library and Information Studies, McGill University, Montreal, Quebec, Canada

    Charles Cole

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer

About this chapter

Cite this chapter

Ruthven, I. (2005). Integration Approaches to Relevance. In: Spink, A., Cole, C. (eds) New Directions in Cognitive Information Retrieval. The Information Retrieval Series, vol 19. Springer, Dordrecht . https://doi.org/10.1007/1-4020-4014-8_4

Download citation

  • DOI: https://doi.org/10.1007/1-4020-4014-8_4

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-4013-9

  • Online ISBN: 978-1-4020-4014-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation