How to Get the Recommender Out of the Lab?

  • Jérome Picault
  • Myriam Ribière
  • David Bonnefoy
  • Kevin Mercer


A personalised system is a complex system made of many interacting parts, from data ingestion to presenting the results to the users. A plethora of methods, tools, algorithms and approaches exist for each piece of such a system: many data and metadata processing methods, many user models, many filtering techniques, many accuracy metrics, many personalisation levels. In addition, a realworld recommender is a piece of an even larger and more complex environment on which there is little control: often the recommender is part of a larger application introducing constraints for the design of the recommender, e.g. the data may not be in a suitable format, or the environment may impose some architectural or privacy constraints. This can make the task of building such a recommender system daunting, and it is easy to make errors. Based on the experience of the authors and the study of other works, this chapter intends to be a guide on the design, implementation and evaluation of personalised systems. It presents the different aspects that must be studied before the design is even started, and how to avoid pitfalls, in a hands-on approach. The chapter presents the main factors to take into account to design a recommender system, and illustrates them through case studies of existing systems to help navigate in the many and complex choices that have to be faced.


Recommender System User Preference User Satisfaction News Item Recommendation Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adomavicius, G., Tuzhilin, E.: Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17(6), 734–749 (2005)CrossRefGoogle Scholar
  2. 2.
    Agrawal, R., Imielinski, T., Swami, A.: Mining association rules between sets of items in large databases. In: Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, vol. 22, pp. 207–216. ACM (1993)Google Scholar
  3. 3.
    Ali, K., van Stam, W.: Tivo: making show recommendations using a distributed collaborative filtering architecture. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. Seattle, WA, USA (2004)Google Scholar
  4. 4.
    Anand, S.S., Mobasher, B.: Contextual recommendation. In: From Web to Social Web: Discovering and Deploying User and Content Profiles, Lecture Notes in Computer Science, pp. 142–160. Springer-Verlag (2007)Google Scholar
  5. 5.
    Anderson, C.: The long tail. Wired (2004)Google Scholar
  6. 6.
    Ardissono, L., Goy, A., Petrone, G., Segnan, M., Torasso, P.: Intrigue: Personalized recommendation of tourist attractions for desktop and handset devices. In: Applied Artificial Intelligence, p. 687–714. Taylor and Francis (2003)Google Scholar
  7. 7.
    Balabanovi´c, M., Shoham, Y.: Fab: content-based, collaborative recommendation. Communications of the ACM 40(3), 66–72 (1997)CrossRefGoogle Scholar
  8. 8.
    Bernhaupt, R., Wilfinger, D., Weiss, A., Tscheligi, M.: An ethnographic study on recommendations in the living room: Implications for the design of itv recommender systems. In: EUROITV’ 08: Proceedings of the 6th European conference on Changing Television Environments, pp. 92–101. Springer-Verlag (2008)Google Scholar
  9. 9.
    Bias, R., Mayhew, D.: Cost-Justifying usability. Morgan Kaufman Publishing (1994)Google Scholar
  10. 10.
    Bolger, N., Davis, A., Rafaeli, E.: Diary methods: Capturing life as it is lived. In Annual Review of Psychology 54(1), 579–616 (2003)CrossRefGoogle Scholar
  11. 11.
    Bonnefoy, D., Bouzid, M., Lhuillier, N., Mercer, K.: More like this or not for me: Delivering personalised recommendations in multi-user environments. In: UM’07: Proceedings of the 11th international conference on User Modeling, pp. 87–96. Springer-Verlag (2007)Google Scholar
  12. 12.
    Bonnefoy, D., Drgehorn, O., Kernchen, R.: Enabling Technologies for Mobile Services: The Mobilife Book, chap. Multimodality and Personalisation. John Wiley & Sons Ltd (2007)Google Scholar
  13. 13.
    Bonnefoy, D., Picault, J., Bouzid, M.: Distributed user profile. Patent applications EP1934901, GB2430281 & WO2007037870 (2005)Google Scholar
  14. 14.
    Burke, R.: Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction 12(4), 331–370 (2002)Google Scholar
  15. 15.
    Cantador, I.: Exploiting the conceptual space in hybrid recommender systems: a semanticbased approach. Ph.D. thesis, Universidad Autnoma de Madrid (UAM), Spain (2008)Google Scholar
  16. 16.
    Cantador, I., Bellogn, A., Castells, P.: News@hand: A semantic web approach to recommending news. In: Proceedings of the 5th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH 2008), Lecture Notes in Computer Science, vol. 5149, pp. 279–283. Springer-Verlag (2008)Google Scholar
  17. 17.
    Cantador, I., Fernández, M., Vallet, D., Castells, P., Picault, J., Ribière, M.: Advances in Semantic Media Adaptation and Personalization, Studies in Computational Intelligence, vol. 93, chap. A Multi-Purpose Ontology-Based Approach for Personalised Content Filtering and Retrieval, pp. 25–51. Springer (2008)Google Scholar
  18. 18.
    Cantador, I., Szomszor, M., Alani, H., Fernández, M., Castells, P.: Enriching ontological user profiles with tagging history for multi-domain recommendations. In: Proceedings of the 1st International Workshop on Collective Semantics: Collective Intelligence and the Semantic Web (CISWeb 2008), pp. 5–19 (2008)Google Scholar
  19. 19.
    Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V.: Gate: A framework and graphical development environment for robust nlp tools and applications. In: Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics (ACL’02) (2002)Google Scholar
  20. 20.
    Dong, J., Martin, S., Waldo, P.: A conference on human factors in computing systemser input and analysis tool for information architecture. In: Conference on Human Factors in Computing Systems, pp. 23–24 (2001)Google Scholar
  21. 21.
    Duda, R.O., Hart, P., Stork, D.G.: Pattern Classification. John Wiley, New York (2001)MATHGoogle Scholar
  22. 22.
    Gilb, T.: Principles of software engineering management. Arron Marcus Associates (1988)Google Scholar
  23. 23.
    Ha, S.H.: An intelligent system for personalized advertising on the internet. LNCS 3182 pp. 21–30 (2004) 364 J´erome Picault, Myriam Ribière, David Bonnefoy and Kevin MercerGoogle Scholar
  24. 24.
    Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: CSCW’00: Proceedings of the 2000 ACM conference on Computer supported cooperative work, pp. 241–250. ACM, New York, NY, USA (2000)CrossRefGoogle Scholar
  25. 25.
    Herlocker, J.L., Terveen, L.G., Konstan, J.A., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Transactions on information systems 22, 5–53 (2004)CrossRefGoogle Scholar
  26. 26.
    International Organisation for Standardisation (ISO): ISO 13407: Human centred design processes for interactive systems.Google Scholar
  27. 27.
    Jameson, A., Baldes, S., Kleinbauer, T.: Two methods for enhancing mutual awareness in a group recommender system. In: AVI’04: Proceedings of the working conference on Advanced visual interfaces, pp. 447–449. ACM, New York, NY, USA (2004)CrossRefGoogle Scholar
  28. 28.
    Lhuillier, N., Bonnefoy, D., Bouzid, M., Millerat, J., Picault, J., Ribière, M.: A recommendation system and method of operation therefor. Patent application WO2008073595 (2006)Google Scholar
  29. 29.
    Lhuillier, N., Bouzid, M., Gadanho, S.: Context-sensitive user preference prediction. Patent application GB2442024 (2006)Google Scholar
  30. 30.
    Lhuillier, N., Bouzid, M., Mercer, K., Picault, J.: System for content item recommendation. Patent application GB2438645 (2006)Google Scholar
  31. 31.
    Masthoff, J.: Group modeling: Selecting a sequence of television items to suit a group of viewers. User Modeling and User-Adapted Interaction 14(1), 37–85 (2004)Google Scholar
  32. 32.
    McCarthy, K., Salamó, M., Coyle, L., McGinty, L., Smyth, B., Nixon, P.: Cats: A synchronous approach to collaborative group recommendation. In: G. Sutcliffe, R. Goebel (eds.) Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference, pp. 86–91. AAAI Press (2006)Google Scholar
  33. 33.
    McNee, S.M., Riedl, J., Konstan, J.A.: Accurate is not always good: How accuracy metrics have hurt recommender systems. In: CHI’06 extended abstracts on Human factors in computing systems, pp. 1097–1101. ACM, New York, NY, USA (2006)CrossRefGoogle Scholar
  34. 34.
    Mobasher, B., Burke, R., Bhaumik, R., Williams, C.: Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness. ACM Trans. Internet Technol. 7(4) (2007)Google Scholar
  35. 35.
    Mobasher, B., Jin, X., Zhou, Y.: Semantically enhanced collaborative filtering on the web. Web Mining: FromWeb to SemanticWeb pp. 57–76 (2004)Google Scholar
  36. 36.
    Oppenheim, A.: Questionnaire Design, Interviewing and Attitude Measurement. Continuum International Publishing, New York (2001)Google Scholar
  37. 37.
    Papadogiorgaki, M., Papastathis, V., Nidelkou, E., Kompatsiaris, Y., Waddington, S., Bratu, B., Ribière, M.: Distributed user modeling for personalized news delivery in mobile devices. In: 2nd International Workshop on Semantic Media Adaptation and Personalization (2007)Google Scholar
  38. 38.
    icault, J., Ribière, M.: An empirical user profile adaptation mechanism that reacts to shifts of interests. Submitted to the 18th European Conference on Artificial Intelligence (2008).
  39. 39.
    Picault, J., Ribière, M.: Method of adapting a user profile including user preferences and communication device. European Patent EP08290033 (2008)Google Scholar
  40. 40.
    Ribière, M., Picault, J.: Progressive display of user interests. Tech. rep., Motorola (2008). Scholar
  41. 41.
    Ribière, M., Picault, J.: Method and apparatus for modifying a user preference profile. Patent application WO2009064585 (2009)Google Scholar
  42. 42.
    Rich, E.M.: User modeling via stereotypes. Cognitive Science 3, 329–354 (1979)CrossRefGoogle Scholar
  43. 43.
    Rokach, L. and Maimon, O. and Arbel, R., Selective voting-getting more for less in sensor fusion, International Journal of Pattern Recognition and Artificial Intelligence 20(3):329–350 (2006)CrossRefGoogle Scholar
  44. 44.
    Rugg, G., McGeorge, P.: The sorting techniques: a tutorial paper on card sorts, picture sorts and item sorts. Expert Systems, The journal of Knowledge Engineering 14(2), 80–93 (2002)Google Scholar
  45. 45.
    Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: WWW’01: Proceedings of the 10th international conference on World Wide Web, pp. 285–295. ACM, New York, NY, USA (2001)Google Scholar
  46. 46.
    Shepherd, M.: Tutorial on personalization and filtering on the web. Web Information Filtering Lab, Dalhousie University, Canada.
  47. 47.
    Shoval, P., Maidel, V., Shapira, B.: An ontology-content-based filtering method. International Journal of Information Theories and Applications (15), 303–318 (2008)Google Scholar
  48. 48.
    Sieg, A., Mobasher, B., Burke, R.: Ontological user profiles for personalized web search. In: Proceedings of AAAI 2007 Workshop on Intelligent Techniques for Web Personalization, pp. 84–91. Vancouver, BC, Canada (2007)Google Scholar
  49. 49.
    Signa, R.: Design strategies for recommender systems. UIE Web App Summit.
  50. 50.
    Snyder, C.: Paper prototyping: The fast and east way to design and refine user interfaces. Morgan Kaufmann Publishing, San Francisco (2003)Google Scholar
  51. 51.
    Terveen, L., Hill, W.: Human-Computer Interaction in the New Millennium, chap. Beyond Recommender Systems: Helping People Help Each Other, pp. 487–509Google Scholar
  52. 52.
    Villegas, P., Sarris, N., Picault, J., Kompatsiaris, I.: Creating a mesh of multimedia news feeds. In: European Workshop on the Integration of Knowledge, Semantics and Digital Media Technologies (EWIMT), pp. 453–454. IEE (2005)Google Scholar
  53. 53.
    Yu, Z., Zhou, X., Hao, Y., Gu, J.: Tv program recommendation for multiple viewers based on user profile merging. User Modeling and User-Adapted Interaction 16(1), 63–82 (2006)Google Scholar
  54. 54.
    Zhang, M., Hurley, N.: Avoiding monotony: improving the diversity of recommendation lists. In: RecSys’08: Proceedings of the 2008 ACM conference on Recommender systems, pp. 123–130. ACM, New York, NY, USA (2008)CrossRefGoogle Scholar
  55. 55.
    Ziegler, C.N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: WWW’05: Proceedings of the 14th international conference on World Wide Web, pp. 22–32. ACM, New York (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Jérome Picault
    • 1
  • Myriam Ribière
    • 1
  • David Bonnefoy
    • 2
  • Kevin Mercer
    • 3
  1. 1.Alcatel-Lucent Bell LabsNew ProvidenceUSA
  2. 2.PearltreesParisFrance
  3. 3.Loughborough UniversityLoughboroughUK

Personalised recommendations