Skip to main content
SpringerLink
Log in
Book cover

Workshop of the Cross-Language Evaluation Forum for European Languages

CLEF 2003: Comparative Evaluation of Multilingual Information Access Systems pp 29–43Cite as

Evaluation of Information Access Technologies at the NTCIR Workshop

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3237))

Abstract

This paper introduces the NTCIR Workshop, a series of evaluation workshops that are designed to enhance research in information access technologies, such as information retrieval, cross-lingual information retrieval, text summarization, question answering and text mining, by providing infrastructure for large-scale evaluations. A brief history, the test collections, and recent progress after the previous CLEF  Workshop are described, highlighting the differences from CLEF. To conclude, some thoughts on future directions are suggested.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. NTCIR Project, http://research.nii.ac.jp/ntcir/

  2. Kando, N.: NTCIR Workshop: Japanese- and Chinese-English cross-lingual information retrieval and Evaluation. multi-grade relevance judgments. In: Peters, C. (ed.) CLEF 2000. LNCS, vol. 2069, pp. 24–33. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  3. Kando, N.: CLIR System Evaluation at the Second NTCIR Workshop. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 371–388. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  4. Kando, N.: CLIR at NTCIR Workshop 3; Cross-language and cross-genre retrieval. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 485–504. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  5. Kando, N., Nozue, T. (eds.): NTCIR Workshop 1: Proceedings of the First NTCIR Workshop on Research in Japanese Text Retrieval and Term Recognition, Tokyo Japan, August 30–September 1, 1999. NII, Tokyo (1999) (ISBN 4-924600-77-6), http://research.nii.ac.jp/

  6. Eguchi, K., Kando, N., Adachi, J. (eds.): NTCIR Workshop 2: Proceedings of the Second NTCIR Workshop on Research in Chinese and Japanese Text Retrieval and Text Summarization. Tokyo Japan, June 2000–March 2001. NII, Tokyo (2001) (ISBN 4-924600-96-2), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings2/ ,

  7. Oyama, K., Ishida, E., Kando, N. (eds.): NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003) (ISBN-4-86049-016-9), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/

  8. Kuriyama, K., Yoshioka, M., Kando, N.: Construction of a large scale test collection NTCIR-2: The effect of additional interactive search and cross-lingual pooling. IPSJ Transactions on Databases, 43(SIG2 (TOD13)), 48–59 (2002) (in Japanese)

    Google Scholar 

  9. Iwayama, M., Fujii, A., Kando, N., Takano, A.: Overview of the patent retrieval task at NTCIR-3. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-OV-PATENT-IwayamaM.pdf

  10. Eguchi, K., Oyama, K., Ishida, E., Kando, N., Kuriyama, K.: Overview of web retrieval task at the Third NTCIR Workshop. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-OV-WEB-EguchiK.pdf

  11. Buckley, C.: trec_eval IR evaluation package. Available from, ftp://ftp.cs.cornell.edu/pub/smart/

  12. Jarvelin, K., Kekalainen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Athens Greec, July 2000, pp. 41–48. ACM Press, New York (2000)

    Google Scholar 

  13. Voorhees, E.M.: Evaluation by highly relevant documents. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), New Orleans, September 2001, pp. 74–82. ACM Press, New York (2001)

    Chapter  Google Scholar 

  14. Iwayama, M., Fujii, A., Kando, N., Marukawa, K.: Empirical study on retrieval models for different document genres: Patents and newspaper articles. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), Toronto, Canada, July 2003, pp. 251–258. ACM Press, New York (2003)

    Chapter  Google Scholar 

  15. Eguchi, K., Oyama, K., Ishida, E., Kando, N., Kuriyama, K.: Evaluation methods for web retrieval tasks considering hyperlink structure. IEICE Transaction on Information and Systems E86-D(9), 1804–1813 (2003)

    Google Scholar 

  16. Itoh, H., Mano, H., Ogawa, Y.: Term distillation for cross-DB retrieval. In: NTCIR Workshop 3: Proceedings of the Third NTCIR Workshop on Research in Information Retrieval, Question Answering and Summarization, Tokyo Japan, October 2001–October 2002. NII, Tokyo (2003), http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-PATENT-ItohH.pdf

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Informatics (NII), Tokyo, 101-8430, Japan

    Noriko Kando

Authors
  1. Noriko Kando
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. ISTI-CNR, Area di Ricerca, Pisa, Italy

    Carol Peters

  2. No Affiliations,  

    Julio Gonzalo  & Martin Braschler  & 

  3. German Institute for International and Security Affairs, Stiftung Wissenschaft und Politik (SWP), Ludwigkirchplatz 3-4, 10719, Berlin, Germany

    Michael Kluck

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kando, N. (2004). Evaluation of Information Access Technologies at the NTCIR Workshop. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds) Comparative Evaluation of Multilingual Information Access Systems. CLEF 2003. Lecture Notes in Computer Science, vol 3237. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30222-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-30222-3_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-24017-4

  • Online ISBN: 978-3-540-30222-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation