Skip to main content
SpringerLink
Log in

A Comparative Framework to Evaluate Recommender Systems in Technology Enhanced Learning: a Case Study

  • Conference paper
  • First Online:
Advances in Artificial Intelligence and Its Applications (MICAI 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9414))

Included in the following conference series:

Abstract

When proposing a novel recommender system, one difficult part is its evaluation. Especially in Technology Enhanced Learning (TEL), this phase is critical because those systems influence students or educators in educational tasks. Our research aims to propose a framework for conducting comparative experiments of different recommender systems in a same educational context. The framework is expected to provide the accuracy of subject systems within a single experiment, depicting the benefits of a novel system against others. We also present an application of such framework for a comparative experiment of popular systems in TEL like Google, Slideshare, Youtube, MERLOT, Connexions and ARIADNE. Our results show that the proposed framework has been effective in comparing the accuracy of those systems, with a clear picture of their performance compared one another. Moreover, the results of the experiment can be used as a benchmark when evaluating novel recommender systems in TEL.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://cnx.org

  2. 2.

    http://www.merlot.org

  3. 3.

    http://www.ariadne-eu.org

  4. 4.

    http://info.merlot.org/merlothelp/index.htm#merlot_collection.htm accessed on 12/05/2015.

  5. 5.

    http://cnx.org/ contents accessed on 12/05/2015.

  6. 6.

    http://globe-info.org/

  7. 7.

    IEEE 1484.12.1-2002, IEEE standard for learning object metadata.

  8. 8.

    https://docs.oracle.com/javase/tutorial/java/index.html

  9. 9.

    http://www.google.com/intl/en/policies/privacy/

  10. 10.

    https://www.linkedin.com/legal/privacy-policy

References

  1. Anand, D., Mampilli, B.S.: Folksonomy-based fuzzy user profiling for improved recommendations. Expert Syst. Appl. 41(5), 2424–2436 (2014)

    Article  Google Scholar 

  2. Bobadilla, J., Ortega, F., Hernando, A., Alcalá, J.: Improving collaborative filtering recommender system results and performance using genetic algorithms. Knowl.-Based Syst. 24(8), 1310–1316 (2011)

    Article  Google Scholar 

  3. Bozo, J., Alarcón, R., Iribarra, S.: Recommending learning objects according to a teachers’ contex model. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 470–475. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  4. Brent, I., Gibbs, G.R., Gruszczynska, A.K.: Obstacles to creating and finding open educational resources: the case of research methods in the social sciences. J. Interact. Media Educ. 2012(1):Art-5 (2012)

    Google Scholar 

  5. Casali, A., Gerling, V., Deco, C., Bender, C.: A recommender system for learning objects personalized retrieval. In: Educational Recommender Systems and Technologies: Practices and Challenges, pp. 182–210. PA, Hershey, Information Science Reference (2012)

    Google Scholar 

  6. Cechinel, C., Ochoa, X.: A brief overview of quality inside learning object repositories. In: Proceedings of the XV International Conference on Human Computer Interaction, p. 83. ACM (2014)

    Google Scholar 

  7. Drachsler, H., Verbert, K., Santos, O., Manouselis, N.: Panorama of recommender systems to support learning. status: accepted (2015)

    Google Scholar 

  8. Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22(1), 5–53 (2004)

    Article  Google Scholar 

  9. Karampiperis, P., Koukourikos, A., Stoitsis, G.: Collaborative filtering recommendation of educational content in social environments utilizing sentiment analysis techniques. In: Manouselis, N., Drachsler, H., Verbert, K., Santos, O.C. (eds.) Recommender Systems for Technology Enhanced Learning, pp. 3–23. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  10. Klerkx, J., Vandeputte, B., Parra, G., Santos, J.L., Van Assche, F., Duval, E.: How to share and reuse learning resources: the ARIADNE experience. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 183–196. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  11. Liang, C.: User profile for personalized web search. In: 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 3, pp. 1847–1850. IEEE (2011)

    Google Scholar 

  12. Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teacher model to speed up the process of building courses. In: Kurosu, M. (ed.) HCII/HCI 2013, Part II. LNCS, vol. 8005, pp. 434–443. Springer, Heidelberg (2013)

    Google Scholar 

  13. Limongelli, C., Lombardi, M., Marani, A., Sciarrone, F.: A teaching-style based social network for didactic building and sharing. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 774–777. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  14. Limongelli, C., Miola, A., Sciarrone, F., Temperini, M.: Supporting teachers to retrieve and select learning objects for personalized courses in the moodle_ls environment. In: 2012 IEEE 12th International Conference on Advanced Learning Technologies (ICALT), pp. 518–520. IEEE (2012)

    Google Scholar 

  15. Maloney, S., Moss, A., Keating, J., Kotsanas, G., Morgan, P.: Sharing teaching and learning resources: perceptions of a university’s faculty members. Med. Educ. 47(8), 811–819 (2013)

    Article  Google Scholar 

  16. Mangina, E., Kilbride, J.: Evaluation of keyphrase extraction algorithm and tiling process for a document/resource recommender within e-learning environments. Comput. Educ. 50(3), 807–820 (2008)

    Article  Google Scholar 

  17. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H., Koper, R.: Recommender systems handbook. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems in Technology Enhanced Learning, pp. 387–415. Springer, Heidelberg (2011)

    Google Scholar 

  18. Manouselis, N., Vuorikari, R., Assche, F.V.: Simulated analysis of maut collaborative filtering for learning object recommendation. In: Proceedings of the 1st Workshop on Social Information Retrieval for Technology Enhanced Learning, pp. 27–35 (2007)

    Google Scholar 

  19. Ochoa, X., Klerkx, J., Vandeputte, B., Duval, E.: On the use of learning object metadata: the globe experience. Towards Ubiquitous Learning, pp. 271–284. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  20. Palavitsinis, N., Manouselis, N., Sanchez-Alonso, S.: Metadata quality in learning object repositories: a case study. Electron. Libr. 32(1), 62–82 (2014)

    Article  Google Scholar 

  21. Salehi, M.: Application of implicit and explicit attribute based collaborative filtering and bide for learning resource recommendation. Data Knowl. Eng. 87, 130–145 (2013)

    Article  Google Scholar 

  22. Sergis, S., Sampson, D.: Learning object recommendations for teachers based on elicited ICT competence profiles. IEEE Trans. Learn. Technol. (2015)

    Google Scholar 

  23. Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  24. Shulman, L.S.: Those who understand: knowledge growth in teaching. Educ. Res. 15(2), 4–14 (1986)

    Article  Google Scholar 

  25. Sicilia, M.Á., García-Barriocanal, E., Sánchez-Alonso, S., Cechinel, C.: Exploring user-based recommender results in large learning object repositories: the case of merlot. Procedia Comput. Sci. 1(2), 2859–2864 (2010)

    Article  Google Scholar 

  26. Sosteric, M., Hesemeier, S.: When is a learning object not an object: a first step towards a theory of learning objects. Int. Rev. Res. Open Distance Learn. 3(2) (2002)

    Google Scholar 

  27. Spink, V., Jansen, B.J.: Searching multimedia federated content web collections. Online Inf. Rev. 30(5), 485–495 (2006)

    Article  Google Scholar 

  28. Tumer, D., Shah, M.A., Bitirim, Y.: An empirical evaluation on semantic search performance of keyword-based and semantic search engines: Google, yahoo, msn and hakia. In: Fourth International Conference on Internet Monitoring and Protection, ICIMP 2009, pp. 51–55. IEEE (2009)

    Google Scholar 

  29. Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Drachsler, H., Bosnic, I., Duval, E.: Context-aware recommender systems for learning: a survey and future challenges. IEEE Trans. Learn. Technol. 5(4), 318–335 (2012)

    Article  Google Scholar 

  30. Wan, X., Okamoto, T.: Utilizing learning process to improve recommender system for group learning support. Neural Comput. Appl. 20(5), 611–621 (2011)

    Article  Google Scholar 

  31. Zapata, A., Menéndez, V.H., Prieto, M.E., Romero, C.: A framework for recommendation in learning object repositories: an example of application in civil engineering. Adv. Eng. Softw. 56, 1–14 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information and Communication Technology, Griffith University, 170 Kessels Road, Nathan, QLD, 4111, Australia

    Matteo Lombardi & Alessandro Marani

Authors
  1. Matteo Lombardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Marani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matteo Lombardi .

Editor information

Editors and Affiliations

  1. Unidad Profesional Interdisciplinaria, México DF, Mexico

    Obdulia Pichardo Lagunas

  2. Universidad Autónoma Metropolitana, México DF, Mexico

    Oscar Herrera Alcántara

  3. Instituto de Investigaciones Eléctricas, Cuernavaca, Morelos, Mexico

    Gustavo Arroyo Figueroa

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Lombardi, M., Marani, A. (2015). A Comparative Framework to Evaluate Recommender Systems in Technology Enhanced Learning: a Case Study. In: Pichardo Lagunas, O., Herrera Alcántara, O., Arroyo Figueroa, G. (eds) Advances in Artificial Intelligence and Its Applications. MICAI 2015. Lecture Notes in Computer Science(), vol 9414. Springer, Cham. https://doi.org/10.1007/978-3-319-27101-9_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27101-9_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27100-2

  • Online ISBN: 978-3-319-27101-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation