Abstract
Information Retrieval system evaluation campaigns produce valuable scientific data, which should be preserved carefully so that they can be available for further studies. A complete record should be maintained of all analyses and interpretations in order to ensure that they are reusable in attempts to replicate particular results or in new research and so that they can be referred to or cited at any time.
In this paper, we describe the data curation approach for the scientific data produced by evaluation campaigns. The medium/long-term aim is to create a large-scale Digital Library System (DLS) of scientific data which supports services for the creation, interpretation and use of multidisciplinary and multilingual digital content.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cleverdon, C.W.: The Cranfield Tests on Index Languages Devices. In: Readings in Information Retrieval, pp. 47–60. Morgan Kaufmann Publisher, Inc, San Francisco, California, USA (1997)
Ackoff, R.L.: From Data to Wisdom. Journal of Applied Systems Analysis 16, 3–9 (1989)
Zeleny, M.: Management Support Systems: Towards Integrated Knowledge Management. Human Systems Management 7, 59–70 (1987)
Abiteboul, S., et al.: The Lowell Database Research Self-Assessment. Communications of the ACM (CACM) 48, 111–118 (2005)
Ioannidis, Y., et al.: Digital library information-technology infrastructures. International Journal on Digital Libraries 5, 266–274 (2005)
Agosti, M., Di Nunzio, G.M., Ferro, N.: A Data Curation Approach to Support In-depth Evaluation Studies. In: MLIA 2006. Proc. International Workshop on New Directions in Multilingual Information Access, pp. 65–68, [last visited 2007, March 23] (2006), http://ucdata.berkeley.edu/sigir2006-mlia.htm
Agosti, M., Di Nunzio, G.M., Ferro, N.: Scientific Data of an Evaluation Campaign: Do We Properly Deal With Them? In: CLEF 2006. LNCS, vol. 4730, pp. 11–20. Springer, Heidelberg (2007)
National Science Board: Long-Lived Digital Data Collections: Enabling Research and Education in the 21st Century (NSB-05-40). National Science Foundation (NSF). [last visited 2007, March 23] (2005), http://www.nsf.gov/pubs/2005/nsb0540/
European Commission Information Society and Media: i2010: Digital Libraries. [last visited 2007, March 23] (2006), http://europa.eu.int/information_society/activities/digital_libraries/doc/brochures/dl_brochure_2006.pdf
Working Group on Data for Science: FROM DATA TO WISDOM: Pathways to Successful Data Management for Australian Science. Report to Minister’s Science, Engineering and Innovation Council (PMSEIC), [last visited 2007, March 23] (2006), http://www.dest.gov.au/sectors/science_innovation/publications_resources/profiles/Presentation_Data_for_Science.htm
Lord, P., Macdonald, A.: e-Science Curation Report. Data curation for e-Science in the UK: an audit to establish requirements for future curation and provision. The JISC Committee for the Support of Research (JCSR). [last visited 2007, March 23] (2003), http://www.jisc.ac.uk/uploaded_documents/e-ScienceReportFinal.pdf
Anderson, W.L.: Some Challenges and Issues in Managing, and Preserving Access To, Long-Lived Collections of Digital Scientific and Technical Data. Data Science Journal 3, 191–202 (2004)
Harman, D., Buckley, C.: The NRRC Reliable Information Access (RIA) Workshop. In: SIGIR 2004. Proc. 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 528–529. ACM Press, New York (2004)
Berners-Lee, T., Fielding, R., Irvine, U.C., Masinter, L.: Uniform Resource Identifiers (URI): Generic Syntax. RFC 2396 (1998)
Paskin, N., (ed.): The DOI Handbook – Edition 4.4.1. International DOI Foundation (IDF). [last visited 2007, August 30] (2006), http://dx.doi.org/10.1000/186
NISO: ANSI/NISO Z39.88 - 2004 – The OpenURL Framework for Context-Sensitive Services. National Information Standards Organization (NISO). [last visited 2007, March 23] (2005), http://www.niso.org/standards/standard_detail.cfm?std_id=783
Brase, J.: Using Digital Library Techniques – Registration of Scientific Primary Data. In: Heery, R., Lyon, L. (eds.) ECDL 2004. LNCS, vol. 3232, pp. 488–494. Springer, Heidelberg (2004)
Paskin, N.: Digital Object Identifiers for Scientific Data. Data Science Journal 4, 12–20 (2005)
Hull, D.: Using Statistical Testing in the Evaluation of Retrieval Experiments. In: SIGIR 1993. Proc. 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 329–338. ACM Press, New York (1993)
Fuhr, N., Hansen, P., Micsik, A., Sølvberg, I.: Digital Libraries: A Generic Classification Scheme. In: Constantopoulos, P., Sølvberg, I.T. (eds.) ECDL 2001. LNCS, vol. 2163, pp. 187–199. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Agosti, M., Di Nunzio, G.M., Ferro, N. (2007). The Importance of Scientific Data Curation for Evaluation Campaigns. In: Thanos, C., Borri, F., Candela, L. (eds) Digital Libraries: Research and Development. DELOS 2007. Lecture Notes in Computer Science, vol 4877. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77088-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-77088-6_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77087-9
Online ISBN: 978-3-540-77088-6
eBook Packages: Computer ScienceComputer Science (R0)