Abstract
News recommender systems provide users with access to news stories that they find interesting and relevant. As other online, stream-based recommender systems, they face particular challenges, including limited information on users’ preferences and also rapidly fluctuating item collections. In addition, technical aspects, such as response time and scalability, must be considered. Both algorithmic and technical considerations shape working requirements for real-world recommender systems in businesses. NewsREEL represents a unique opportunity to evaluate recommendation algorithms and for students to experience realistic conditions and to enlarge their skill sets. The NewsREEL Challenge requires participants to conduct data-driven experiments in NewsREEL Replay as well as deploy their best models into NewsREEL Live’s ‘living lab’. This paper presents NewsREEL 2017 and also provides insights into the effectiveness of NewsREEL to support the goals of instructors teaching recommender systems to students. We discuss the experiences of NewsREEL participants as well as those of instructors teaching recommender systems to students, and in this way, we showcase NewsREEL’s ability to support the education of future data scientists.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Banks, F., Barlex, D.: Enabling the ‘E’ in STEM. In: Teaching STEM in the Secondary School. Routledge, London (2014)
Barr, M.: Video games can develop graduate skills in higher education students: a randomised trial. Computers & Education, (2017) (Accepted for publication)
Bennett, J., Lanning, S., et al.: The Netflix prize. In: Proceedings of KDD Cup and Workshop, New York, NY, USA, vol. 2007, p. 35 (2007)
Billsus, D., Pazzani, M.J.: Adaptive news access. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web. LNCS, vol. 4321, pp. 550–570. Springer, Heidelberg (2007). doi:10.1007/978-3-540-72079-9_18
Castells, P., Hurley, N.J., Vargas, S.: Novelty and diversity in recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 881–918. Springer, New York (2015)
Das, A., Datar, M., Garg, A., Rajaram, S.: Google news personalization - scalable online collaborative filtering. In: WWW, pp. 271–280. ACM, New York (2007)
Efthimis, E., Fernandez-Luna, J.M., Huete, J.F., MacFarlane, A. (eds.): Teaching and Learning in Information Retrieval. Springer, Heidelberg (2011)
Fry, H., Ketteridge, S., Marshall, S.: Understanding student learning. In: A Handbook for Teaching and Learning in Higher Education. Routledge, London (2003)
Garcin, F., Faltings, B., Donatsch, O., Alazzawi, A., Bruttin, C., Huber, A.: Offline and online evaluation of news recommender systems at swissinfo.ch. In: RecSys, pp. 169–176 (2014)
Harper, F.M., Konstan, J.A.: The Movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. (TiiS) 5(4), 19 (2016)
Hopfgartner, F., Lommatzsch, A., Kille, B., Larson, M., Brodt, T., Cremonesi, P., Karatzoglou, A.: The potentials of recommender systems challenges for student learning. In: Proceedings of CiML 2016: Challenges in Machine Learning: Gaming and Education, October 2016
Joachims, T., Swaminathan, A., Schnabel, T.: Unbiased learning-to-rank with biased feedback. In: The Tenth ACM International Conference, pp. 781–789. ACM Press, New York (2017)
Kille, B., Brodt, T., Heintz, T., Hopfgartner, F., Lommatzsch, A., Seiler, J.: NEWSREEL 2014: summary of the news recommendation evaluation lab. In: Working Notes for CLEF 2014 Conference, Sheffield, UK, 15–18 September 2014, pp. 790–801 (2014)
Kille, B., Hopfgartner, F., Brodt, T., Heintz, T.: The plista dataset. In: International News Recommender Systems Workshop and Challenge, pp. 16–23. ACM, New York, October 2013
Kille, B., Lommatzsch, A., Gebremeskel, G.G., Hopfgartner, F., Larson, M., Seiler, J., Malagoli, D., Serény, A., Brodt, T., Vries, A.P.: Overview of NewsREEL’16: multi-dimensional evaluation of real-time stream-recommendation algorithms. In: Fuhr, N., Quaresma, P., Gonçalves, T., Larsen, B., Balog, K., Macdonald, C., Cappellato, L., Ferro, N. (eds.) CLEF 2016. LNCS, vol. 9822, pp. 311–331. Springer, Cham (2016). doi:10.1007/978-3-319-44564-9_27
Kille, B., Lommatzsch, A., Hopfgartner, F., Larson, M., Seiler, J., Malagoli, D., Serény, A., Brodt, T.: CLEF NewsREEL 2016: Comparing multi-dimensional offline and online evaluation of news recommender systems. In: Working Notes of CLEF 2016 - Conference and Labs of the Evaluation forum, Évora, Portugal, 5–8 September, 2016, pp. 593–605 (2016)
Kille, B., Lommatzsch, A., Turrin, R., Serény, A., Larson, M., Brodt, T., Seiler, J., Hopfgartner, F.: Overview of CLEF NewsREEL 2015: news recommendation evaluation lab. In: Working Notes of CLEF 2015 - Conference and Labs of the Evaluation forum, 8–11 September 2015, Toulouse, France (2015)
Kille, B., Lommatzsch, A., Turrin, R., Serény, A., Larson, M., Brodt, T., Seiler, J., Hopfgartner, F.: Stream-based recommendations: online and offline evaluation as a service. In: Mothe, J., Savoy, J., Kamps, J., Pinel-Sauvagnat, K., Jones, G.J.F., SanJuan, E., Cappellato, L., Ferro, N. (eds.) CLEF 2015. LNCS, vol. 9283, pp. 497–517. Springer, Cham (2015). doi:10.1007/978-3-319-24027-5_48
Li, L., Schapire, R.E., Chu, W., Langford, J., Langford, J.: A contextual-bandit approach to personalized news article recommendation. In: The 19th International Conference, pp. 661–670. ACM Press, New York (2010)
Lopez-Garcia, R., Cacheda, F.: A technical approach to information retrieval pedagogy. In: Teaching and Learning in Information Retrieval. Springer (2011)
Mizzaro, S.: Teaching web information retrieval to computer science students: concrete approach and its analysis. In: Teaching and Learning in Information Retrieval (2011)
Özgöbek, Ö., Gulla, J.A., Erdur, R.C.: A survey on challenges and methods in news recommendation. In: WEBIST (2014)
Ribeiro, M.T., Lacerda, A., Veloso, A., Ziviani, N.: Pareto-efficient hybridization for multi-objective recommender systems. In: Proceedings of the Sixth ACM Conference on Recommender Systems, pp. 19–26. ACM (2012)
Said, A., Tikk, D., Stumpf, K., Shi, Y., Larson, M., Cremonesi, P., Evaluation, R.S.: A 3D benchmark. In: RUE@ RecSys, pp. 21–23 (2012)
Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Recommender Systems Handbook, pp. 257–297. Springer, Boston, October 2010
Smart, K.L., Csapo, N.: Learning by doing: engaging students through learner-centred activities. Bus. Prof. Commun. Q. 40, 451–457 (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lommatzsch, A. et al. (2017). CLEF 2017 NewsREEL Overview: A Stream-Based Recommender Task for Evaluation and Education. In: Jones, G., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2017. Lecture Notes in Computer Science(), vol 10456. Springer, Cham. https://doi.org/10.1007/978-3-319-65813-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-65813-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65812-4
Online ISBN: 978-3-319-65813-1
eBook Packages: Computer ScienceComputer Science (R0)