Advertisement

Evaluation of Information Access Technologies at the NTCIR Workshop

  • Noriko Kando
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3237)

Abstract

This paper introduces the NTCIR Workshop, a series of evaluation workshops that are designed to enhance research in information access technologies, such as information retrieval, cross-lingual information retrieval, text summarization, question answering and text mining, by providing infrastructure for large-scale evaluations. A brief history, the test collections, and recent progress after the previous CLEF  Workshop are described, highlighting the differences from CLEF. To conclude, some thoughts on future directions are suggested.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Kando, N.: NTCIR Workshop: Japanese- and Chinese-English cross-lingual information retrieval and Evaluation. multi-grade relevance judgments. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 24–33. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  3. 3.
    Kando, N.: CLIR System Evaluation at the Second NTCIR Workshop. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 371–388. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Kando, N.: CLIR at NTCIR Workshop 3; Cross-language and cross-genre retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 485–504. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Kando, N., Nozue, T. (eds.): NTCIR Workshop 1: Proceedings of the First NTCIR Workshop on Research in Japanese Text Retrieval and Term Recognition, Tokyo Japan, August 30–September 1, 1999. NII, Tokyo (1999) (ISBN 4-924600-77-6), http://research.nii.ac.jp/
  6. 6.
    Eguchi, K., Kando, N., Adachi, J. (eds.): NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese and Japanese Text Retrieval and Text Summarization. Tokyo Japan, June 2000–March 2001. NII, Tokyo (2001) (ISBN 4-924600-96-2), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings2/,
  7. 7.
    Oyama, K., Ishida, E., Kando, N. (eds.): NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003) (ISBN-4-86049-016-9), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/
  8. 8.
    Kuriyama, K., Yoshioka, M., Kando, N.: Construction of a large scale test collection NTCIR-2: The effect of additional interactive search and cross-lingual pooling. IPSJ Transactions on Databases, 43(SIG2 (TOD13)), 48–59 (2002) (in Japanese)Google Scholar
  9. 9.
    Iwayama, M., Fujii, A., Kando, N., Takano, A.: Overview of the patent retrieval task at NTCIR-3. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-OV-PATENT-IwayamaM.pdf
  10. 10.
    Eguchi, K., Oyama, K., Ishida, E., Kando, N., Kuriyama, K.: Overview of web retrieval task at the Third NTCIR Workshop. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-OV-WEB-EguchiK.pdf
  11. 11.
    Buckley, C.: trec_eval IR evaluation package. Available from, ftp://ftp.cs.cornell.edu/pub/smart/
  12. 12.
    Jarvelin, K., Kekalainen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Athens Greec, July 2000, pp. 41–48. ACM Press, New York (2000)Google Scholar
  13. 13.
    Voorhees, E.M.: Evaluation by highly relevant documents. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), New Orleans, September 2001, pp. 74–82. ACM Press, New York (2001)CrossRefGoogle Scholar
  14. 14.
    Iwayama, M., Fujii, A., Kando, N., Marukawa, K.: Empirical study on retrieval models for different document genres: Patents and newspaper articles. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), Toronto, Canada, July 2003, pp. 251–258. ACM Press, New York (2003)CrossRefGoogle Scholar
  15. 15.
    Eguchi, K., Oyama, K., Ishida, E., Kando, N., Kuriyama, K.: Evaluation methods for web retrieval tasks considering hyperlink structure. IEICE Transaction on Information and Systems E86-D(9), 1804–1813 (2003)Google Scholar
  16. 16.
    Itoh, H., Mano, H., Ogawa, Y.: Term distillation for cross-DB retrieval. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-PATENT-ItohH.pdf Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Noriko Kando
    • 1
  1. 1.National Institute of Informatics (NII)TokyoJapan

Personalised recommendations