Skip to main content
SpringerLink
Log in

Evaluating Recommender Systems for Supportive Technologies

  • Chapter
  • First Online:
User Modeling and Adaptation for Daily Routines

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Recommender systems have evolved in recent years into sophisticated support tools that assist users in dealing with the decisions faced in everyday life. Recommender systems were designed to be invaluable in situations, where a large number of options are available, such as deciding what to watch on television, what information to access online, what to purchase in a supermarket, or what to eat. Recommender system evaluations are carried out typically during the design phase of recommender systems to understand the suitability of approaches to the recommendation process, in the usability phase to gain insight into interfacing and user acceptance, and in live user studies to judge the uptake of recommendations generated and impact of the recommender system. In this chapter, we present a detailed overview of evaluation techniques for recommender systems covering a variety of tried and tested methods and metrics. We illustrate their use by presenting a case study that investigates the applicability of a suite of recommender algorithms in a recipe recommender system aimed to assist individuals in planning their daily food intake. The study details an offline evaluation, which compares algorithms, such as collaborative, content-based, and hybrid methods, using multiple performance metrics, to determine the best candidate algorithm for a recipe recommender application.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.mturk.com

  2. 2.

    http://www.mealopedia.com

  3. 3.

    Note that the number of relevant items varies across users, as each profile contains a different number of ratings.

References

  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans knowl Data Eng 17: 734–749

    Article  Google Scholar 

  2. Bambini R, Cremonesi P, Turrin R (2011) A recommender system for an iptv service provider: a real large-scale production environment. In: Recommender systems handbook. Springer, Boston, pp 299–331

    Google Scholar 

  3. Berkovsky S, Freyne J (2010) Group-based recipe recommendations: analysis of data aggregation strategies. In: Amatriain X, Torrens M, Resnick P, Zanker M (eds) RecSys. ACM, New York, pp 111–118

    Chapter  Google Scholar 

  4. Billsus D, Pazzani M (1998) Learning collaborative information filters. In: Proceedings of the fifteenth international conference on machine learning, vol 54, p 48

    Google Scholar 

  5. Bollen D, Knijnenburg B, Willemsen M, Graus M (2010) Understanding choice overload in recommender systems. In: Proceedings of the fourth ACM conference on recommender systems. ACM, New York, pp 63–70

    Google Scholar 

  6. Burke R (2007) The adaptive web, chap. In: Hybrid web recommender systems. Springer, Berlin/Heidelberg, pp 377–408. URL http://dl.acm.org/citation.cfm?id=1768197.1768211

  7. Burke R, Hammond K, Young B (1996) Knowledge-based navigation of complex information spaces. In: Proceedings of the national conference on artificial intelligence, pp 462–468

    Google Scholar 

  8. Cremonesi P, Garzotto F, Negro S, Papadopoulos A, Turrin R (2011) Comparative evaluation of recommender system quality. In: Proceedings of the 2011 annual conference extended abstracts on human factors in computing systems. ACM, Vancouver, pp 1927–1932

    Google Scholar 

  9. Desrosiers C, Karypis G (2011) A comprehensive survey of neighborhood-based recommendation methods. In: Recommender systems handbook. Springer, Boston, pp 107–144

    Google Scholar 

  10. Farzan R, Coyle M, Freyne J, Brusilovsky P, Smyth B (2007) Assist: adaptive social support for information space traversal. In: Proceedings of the eighteenth conference on hypertext and hypermedia, HT ’07. ACM, New York, pp 199–208. doi:http://doi.acm.org/10.1145/1286240.1286299. URL:http://doi.acm.org/10.1145/1286240.1286299

  11. Farzan R, DiMicco J, Millen D, Geyer W, Brownholtz E (2008) Results from deploying a participation incentive mechanism within the enterprise

    Google Scholar 

  12. Freyne J, Berkovsky S (2010) Intelligent food planning: personalized recipe recommendation. In: Proceedings of the 2010 international conference on intelligent user interfaces (IUI 2010), pp 321–324

    Google Scholar 

  13. Freyne J, Berkovsky S, Baghaei N, Kimani S, Smith G (2011) Personalized techniques for lifestyle change. In: Proceedings artificial intelligence in medicine, AIME, Bled, Slovenia, 2–6 Jul 2011, pp 139–148

    Google Scholar 

  14. Herlocker J, Konstan J, Borchers A, Riedl J (1999) An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 230–237

    Google Scholar 

  15. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22(1):5–53. doi:http://doi.acm.org/10.1145/963770.963772

    Google Scholar 

  16. Koenigstein N, Dror G, Koren Y (2011) Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy. In: Mobasher B, Burke RD, Jannach D, Adomavicius G (eds) RecSys. ACM, Chicago, pp 165–172

    Google Scholar 

  17. Kohavi R, Longbotham R, Sommerfield D, Henne R (2009) Controlled experiments on the web: survey and practical guide. Data Min Knowl Dis 18(1):140–181

    Article  MathSciNet  Google Scholar 

  18. Konstan J, Miller B, Maltz D, Herlocker J, Gordon L, Riedl J (1997) GroupLens: applying collaborative filtering to Usenet news. Commun ACM 40(3):87

    Article  Google Scholar 

  19. Koren Y, Bell R (2011) Advances in collaborative filtering. In: Recommender systems handbook. Springer, Boston, pp 145–186

    Google Scholar 

  20. Lops P, Gemmis M, Semeraro G (2011) Content-based recommender systems: state of the art and trends. In: Recommender systems handbook. Springer, Boston, pp 73–105

    Book  Google Scholar 

  21. Masthoff J (2004) Group modeling: selecting a sequence of television items to suit a group of viewers. User Model User-Adap Interact 14(1):37–85

    Article  Google Scholar 

  22. McJones P (1997) Eachmovie collaborative filtering dataset, DEC systems research center. http://www.research.compaq.com/src/eachmovie/

  23. McNee S, Albert I, Cosley D, Gopalkrishnan P, Lam S, Rashid A, Konstan J, Riedl J (2002) On the recommending of citations for research papers. In: Proceedings of the 2002 ACM conference on computer supported cooperative work. ACM, New York, pp 116–125

    Google Scholar 

  24. Noakes M, Clifton P (2005) The CSIRO total wellbeing diet book. Penguin Group, Australia

    Google Scholar 

  25. Noakes M, Clifton P (2006) The CSIRO total wellbeing diet book 2. Penguin Group, Australia

    Google Scholar 

  26. van Pinxteren Y, Geleijnse G, Kamsteeg P (2011) Deriving a recipe similarity measure for recommending healthful meals. In: Proceedings of the 2011 international conference on intelligent user interfaces, IUI 2011, pp 105–114

    Google Scholar 

  27. Pu P, Chen L (2006) Trust building with explanation interfaces. In: Proceedings of the 11th international conference on intelligent user interfaces, IUI ’06. ACM, New York, pp 93–100. doi:http://doi.acm.org/10.1145/1111449.1111475. URL:http://doi.acm.org/10.1145/1111449.1111475

  28. Quinlan J (1992) Learning with continuous classes. In: Proceedings of the 5th Australian joint conference on artificial intelligence, Citeseer, pp 343–348

    Google Scholar 

  29. Rashid A, Karypis G, Riedl J (2008) Learning preferences of new users in recommender systems: an information theoretic approach. ACM SIGKDD Explor Newslett 10(2):90–100

    Article  Google Scholar 

  30. Rashid AM, Albert I, Cosley D, Lam SK, McNee SM, Konstan JA, Riedl J (2001) Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th international conference on intelligent user interfaces, IUI ’02. ACM, New York, pp 127–134. doi:http://doi.acm.org/10.1145/502716.502737. URL:http://doi.acm.org/10.1145/502716.502737

  31. Ricci F, Rokach L, Shapira B, Kantor P (2010) Recommender systems handbook. Springer, New York

    Google Scholar 

  32. Said A, Berkovsky S, De Luca EW (2010) Putting things in context: challenge on context-aware movie recommendation. In: Proceedings of the workshop on context-aware movie recommendation, CAMRa ’10. ACM, New York, pp 2–6. doi:http://doi.acm.org/10.1145/1869652.1869665. URL:http://doi.acm.org/10.1145/1869652.1869665

  33. Shani G, Gunawardana A (2009) Evaluating recommender systems. Microsoft research report MSR-TR-2009-159, Nov 2009

    Google Scholar 

  34. Swearingen K, Sinha R (2001) Beyond algorithms: an hci perspective on recommender systems. In: ACM SIGIR 2001 workshop on recommender systems, Citeseer

    Google Scholar 

  35. Wang Y, Witten I (1996) Induction of model trees for predicting continuous classes

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Engineering Laboratory, ICT Center, CSIRO, 76, Epping, NSW, 1710, Australia

    Jill Freyne & Shlomo Berkovsky

  2. Network Research Group, NICTA, Locked Bag 9013, Alexandria, NSW, 1435, Australia

    Shlomo Berkovsky

Authors
  1. Jill Freyne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shlomo Berkovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jill Freyne .

Editor information

Editors and Affiliations

  1. Department of Languages and Computer Sys, Universidad Rey Juan Carlos, Tulipán s/n, Móstoles, 28933, Madrid, Spain

    EstefanĂ­a MartĂ­n

  2. IIC-Knowledge Engineering Institute, Universidad AutĂłnoma de Madrid, Tomas y Valiente 11, Madrid, 28049, Madrid, Spain

    Pablo A. Haya

  3. Department of Computer Science and Engin, Universidad AutĂłnoma de Madrid, Tomas y Valiente 11, Madrid, 28049, Madrid, Spain

    Rosa M. Carro

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Freyne, J., Berkovsky, S. (2013). Evaluating Recommender Systems for Supportive Technologies. In: Martín, E., Haya, P., Carro, R. (eds) User Modeling and Adaptation for Daily Routines. Human–Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-4471-4778-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4778-7_8

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4777-0

  • Online ISBN: 978-1-4471-4778-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation