Abstract
Evaluation of digital libraries assesses their effectiveness, quality and overall impact. In this paper we propose to use the Daffodil system as an experimental framework for the evaluation and research of interactive IR and digital libraries. The system already provides a rich set of working services and available information sources. These services and sources can be used as a foundation for further research going beyond basic functionalities. Besides the services and sources, the system supports a logging scheme for comparison of user behavior. In addition, the system can easily be extended regarding both services and sources. Daffodil’s highly flexible and extensible agent-based architecture allows for easy integration of additional components, and access to all existing services. Finally, the system provides a user-friendly graphical interface and facilitating services for log generation and analysis. The experimental framework can serve as a joint theoretical and practical platform for the evaluation of DLs, with the long-term goal of creating a community centered on interactive IR and DL evaluation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fuhr, N., Klas, C.-P., Schaefer, A., Mutschke, P.: Daffodil: An integrated desktop for supporting high-level search activities in federated digital libraries. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 597–612. Springer, Heidelberg (2002)
Goncalves, M.A., Fox, E.A., Cassel, L., Krowne, A., Ravindranathan, U., Panchanathan, G., Jagodzinski, F.: Standards, mark-up, and metadata: The xml log standard for digital libraries: analysis, evolution, and deployment. In: Proceedings of the third ACM/IEEE-CS joint conference on Digital libraries, IEEE Computer Society Press, Los Alamitos (2003)
Goncalves, M.A., Shen, R., Fox, E.A., Ali, M.F., Luo, M.: An xml log standard and tool for digital library logging analysis. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 129–143. Springer, Heidelberg (2002)
Klas, C.-P., Albrechtsen, H., Fuhr, N., Hansen, P., Kapidakis, S., Kovács, L., Kriewel, S., Micsik, A., Papatheodorou, C., Tsakonas, G., Jacob, E.: A logging scheme for comparative digital library evaluation. In: Gonzalo, J., Thanos, C., Verdejo, M.F., Carrasco, R.C. (eds.) ECDL 2006. LNCS, vol. 4172, pp. 267–278. Springer, Heidelberg (2006)
Klas, C.-P., Fuhr, N., Kriewel, S., Albrechtsen, H., Tsakonas, G., Kapidakis, S., Papatheodorou, C., Hansen, P., Kovacs, L., Micsik, A., Jacob, E.: An experimental framework for comparative digital library evaluation: the logging scheme. In: JCDL 2006. Proceedings of the 6th ACM/IEEE-CS joint conference on Digital libraries, pp. 308–309. ACM Press, New York (2006)
Klas, C.-P., Kriewel, S., Schaefer, A.: Daffodil - nutzerorientiertes zugangssystem für heterogene digitale bibliotheken. In: dvs Band (2004)
Krause, J.: Visualisation, multimodality and traditional graphical user interfaces. Review of Information Science 2(2) (1997)
Kriewel, S., Klas, C.-P., Schaefer, A., Fuhr, N.: Daffodil - strategic support for user-oriented access to heterogeneous digital libraries. D-Lib Magazine 10(6) (2004), http://www.dlib.org/dlib/june04/kriewel/06kriewel.html
Malik, S., Klas, C.-P., Fuhr, N., Larsen, B., Tombros, A.: Designing a user interface for interactive retrieval of structured documents — lessons learned from the inex interactive track. In: Gonzalo, J., Thanos, C., Verdejo, M.F., Carrasco, R.C. (eds.) ECDL 2006. LNCS, vol. 4172, Springer, Heidelberg (2006)
Saracevic, T., Covi, L.: Challenges for digital library evaluation. Proceedings of the American Society for Information Science 37, 341–350 (2000)
Schaefer, A., Jordan, M., Klas, C.-P., Fuhr, N.: Active support for query formulation in virtual digital libraries: A case study with DAFFODIL. In: Rauber, A., Christodoulakis, S., Tjoa, A.M. (eds.) ECDL 2005. LNCS, vol. 3652, Springer, Heidelberg (2005)
Theobald, M., Klas, C.-P.: BINGO! and Daffodil: Personalized exploration of digital libraries and web sources. In: In 7th International Conference on Computer-Assisted Information Retrieval (RIAO 2004). Test (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klas, CP., Kriewel, S., Fuhr, N. (2007). An Experimental Framework for Interactive Information Retrieval and Digital Libraries Evaluation. In: Thanos, C., Borri, F., Candela, L. (eds) Digital Libraries: Research and Development. DELOS 2007. Lecture Notes in Computer Science, vol 4877. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77088-6_14
Download citation
DOI: https://doi.org/10.1007/978-3-540-77088-6_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77087-9
Online ISBN: 978-3-540-77088-6
eBook Packages: Computer ScienceComputer Science (R0)