Abstract
Thanks to recent developments on automatic generation of metadata and interoperability between repositories, the production, management and consumption of learning object metadata is vastly surpassing the human capacity to review or process these metadata. However, we need to make sure that the presence of some low quality metadata does not compromise the performance of services that rely on that information. Consequently, there is a need for automatic assessment of the quality of metadata, so that tools or users can be alerted about low quality instances. In this paper, we present several quality metrics for learning object metadata. We applied these metrics to a sample of records from a real repository and compared the results with the quality assessment given to the same records by a group of human reviewers. Through correlation and regression analysis, we found that one of the metrics, the text information content, could be used as a predictor of the human evaluation. While this metric is not a definitive measurement of the “real” quality of the metadata record, we present several ways in which it can be used. We also propose new research in other quality dimensions of the learning object metadata.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
IEEE: IEEE Standard for Learning Object Metadata (2002), http://ltsc.ieee.org/doc/wg12/
Singh, A., Boley, H., Bhavsar, V.C.: A learning object metadata generator applied to computer science terminology. Presented at the Learning Objects Summit, March 29-30 (2004)
Cardinels, K., Meire, M., Duval, E.: Automating metadata generation: the simple indexing interface. In: Proceedings of the 14th WWW conference, pp. 548–556. ACM Press, New York (2005)
Simon, B., Massart, D., van Assche, F., Ternier, S., Duval, E., Brantner, S., et al.: A Simple Query Interface for Interoperable Learning Repositories. In: Simon, B., Olmedilla, D., Saito, N. (eds.), pp. 11–18 (2006)
GLOBE. Global Learning Objects Brokered Exchange (2006), http://taste.merlot.org/initiatives/globe.htm
Verbert, K., Jovanovic, J., Gaševic, D., Duval, E.: Repurposing Learning Object Components. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM-WS 2005. LNCS, vol. 3762, pp. 1169–1178. Springer, Heidelberg (2005)
Currier, S., Barton, J., O’Beirne, R., Ryan, B.: Quality assurance for digital learning object repositories: issues for the metadata creation process. ALT-J, Research in Learning Technology 12, 5–20 (2004)
Barton, J., Currier, S., Hey, J.: Building Quality Assurance into Metadata Creation: an Analysis based on the Learning Objects and e-Prints Communities of Practice. In: Proceedings 2003 Dublin Core Conference, Seatle, Washington, pp. 39–48 (2003)
Guy, M., Powell, A., Day, M.: Improving the quality of metadata in Eprint archives, Ariadne (2004)
Weibel, S.: Border Crossings: Reflections on a Decade of Metadata Consensus Building. D-Lib Magazine 11 (2005)
Ochoa, X., Duval, E.: Quality Metrics for Learning Object Metadata. In: Proceedings ED-Media 2006, pp. 1004–1011 (2006)
Bruce, T., Hillman, D.: The continuum of metadata quality: defining, expressing, exploiting. In: Hillman, D., Westbrooks, L. (eds.) Metadata in Practice. American Library Association, Chicago (2004)
Duval, E.: LearnRank: the real quality measure for learning materials. In: McCluskey, A. (ed.) Policy and Innovation in Education - Quality Criteria, European Schoolnet, pp. 26–29 (2005)
Resnik, P.: Using Information Content to Evaluate Semantic Similarity in a Taxonomy. In: Proceedings of International Joint Conferences on Artificial Intelligence, pp. 448–453 (1995)
Aizawa, A.: An information-theoretic perspective of tf–idf measures. Information Processing and Management 39, 45–65 (2003)
Foltz, P., Kintsch, W., Landauer, T.: The measurement of textual coherence with Latent Semantic Analysis. Discourse Processes 25, 285–307 (1998)
Ariadne Foundation. Ariadne Foundation (2005), http://www.ariadne-eu.org
Najjar, J., Ternier, S., Duval, E.: User Behavior in Learning Objects Repositories: An Empirical Analysis. In: Proceedings of ED-MEDIA 2004 (2004)
Shrout, P., Fleiss, J.: Intraclass Correlations: Uses in Assessing Rater Reliability. Psychological Bulletin 2, 420–428 (1979)
Bederson, B., Shneiderman, B., Wattenberg, M.: Ordered and Quantum Treemaps: Making Effective Use of 2D Space to Display Hierarchies. ACM Transactions on Graphics (TOG) 21, 833–854 (2002)
Shreeves, S., Knutson, E., Stvilia, B., Palmer, C., Twidale, M., Cole, T.: Is “Quality” Metadata "Shareable" Metadata? The Implications of Local Metadata Practices for Federated Collections. In: Proceedings of the Twelfth National Conference of the Association of College and Research Libraries, Association of College and Research Libraries, IL (2005)
Lei, Y., Sabou, M., Lopez, V., Zhu, J., Uren, V., Motta, E.: An Infrastructure for Acquiring High Quality Semantic Metadata. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 230–244. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ochoa, X., Duval, E. (2006). Towards Automatic Evaluation of Learning Object Metadata Quality. In: Roddick, J.F., et al. Advances in Conceptual Modeling - Theory and Practice. ER 2006. Lecture Notes in Computer Science, vol 4231. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11908883_44
Download citation
DOI: https://doi.org/10.1007/11908883_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-47703-7
Online ISBN: 978-3-540-47704-4
eBook Packages: Computer ScienceComputer Science (R0)