Multimedia Tools and Applications

, Volume 75, Issue 6, pp 3323–3351 | Cite as

A user-centric evaluation of context-aware recommendations for a mobile news service

  • Toon De Pessemier
  • Cédric Courtois
  • Kris Vanhecke
  • Kristin Van Damme
  • Luc Martens
  • Lieven De Marez


Traditional recommender systems provide personal suggestions based on the user’s preferences, without taking into account any additional contextual information, such as time or device type. The added value of contextual information for the recommendation process is highly dependent on the application domain, the type of contextual information, and variations in users’ usage behavior in different contextual situations. This paper investigates whether users utilize a mobile news service in different contextual situations and whether the context has an influence on their consumption behavior. Furthermore, the importance of context for the recommendation process is investigated by comparing the user satisfaction with recommendations based on an explicit static profile, content-based recommendations using the actual user behavior but ignoring the context, and context-aware content-based recommendations incorporating user behavior as well as context. Considering the recommendations based on the static profile as a reference condition, the results indicate a significant improvement for recommendations that are based on the actual user behavior. This improvement is due to the discrepancy between explicitly stated preferences (initial profile) and the actual consumption behavior of the user. The context-aware content-based recommendations did not significantly outperform the content-based recommendations in our user study. Context-aware content-based recommendations may induce a higher user satisfaction after a longer period of service operation, enabling the recommender to overcome the cold-start problem and distinguish user preferences in various contextual situations.


Recommender system News recommendation User evaluation Context-aware Algorithm-based news 



This research was performed in the context of the iMinds-MIX Stream Store project. Stream Store is a project cofunded by iMinds (Interdisciplinary institute for Technology) a research institute founded by the Flemish Government. Companies and organizations involved in the project are De Persgroep, Roularta Media Group nv, iMinds-iLab.o, VMMA, and Limecraft with project support of IWT.

The authors would also like to thank the researchers of MIX for the development of the Stream Store client application and the team of the iMinds-MMLab research group for the processing of the metadata. For the setup of the user experiment and performing the evaluation, the authors would like to express their gratitude to the students of the iMinds-MICT research group.


  1. 1.
    Adomavicius G, Sankaranarayanan R, Sen S, Tuzhilin A (2005) Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans Inf Syst 23(1):103–145. doi: 10.1145/1055709.1055714 CrossRefGoogle Scholar
  2. 2.
    Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749. doi: 10.1109/TKDE.2005.99 CrossRefGoogle Scholar
  3. 3.
    Adomavicius G, Tuzhilin A (2008) Tutorial on context-aware recommender systems. In: Proceedings of the 2nd ACM conference on recommender systems (RecSys ’08)Google Scholar
  4. 4.
    Adomavicius G, Tuzhilin A (2011) Context-aware recommender systems. In: Ricci F, Rokach L, Shapira B, Kantor PB (eds) Recommender systems handbook. doi: 10.1007/978-0-387-85820-3_7. Springer, Berlin Heidelberg New York, pp 217–253CrossRefGoogle Scholar
  5. 5.
    Bagdonavicius V, Julius K, Nikulin MS (2013) Chi-squared tests. Wiley, New York, pp 17–75. doi: 10.1002/9781118557716.ch2 Google Scholar
  6. 6.
    Baltrunas L, Ricci F (2009) Context-based splitting of item ratings in collaborative filtering. In: Proceedings of the 3rd ACM conference on recommender systems, RecSys ’09. doi: 10.1145/1639714.1639759. ACM, New York, pp 245–248
  7. 7.
    Bazire M, Brézillon P (2005) Understanding context before using it. In: Dey A, Kokinov B, Leake D, Turner R (eds) Modeling and using context. Lecture Notes in Computer Science. doi: 10.1007/11508373_3, vol 3554. Springer, Berlin Heidelberg, pp 29–40
  8. 8.
    Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 14th conference on uncertainty in artificial intelligence, UAI’98. Morgan Kaufmann, San Francisco, pp 43–52
  9. 9.
    Brown PJ, Bovey JD, Chen X (1997) Context-aware applications: from the laboratory to the marketplace. IEEE Pers Commun 4(5):58–64. doi: 10.1109/98.626984 CrossRefGoogle Scholar
  10. 10.
    Burke R (2002) Hybrid recommender systems: Survey and experiments. User Model User-Adap Inter 12(4):331–370CrossRefMATHGoogle Scholar
  11. 11.
    Cantador I, Bellogín A, Castells P (2008) News@hand: a semantic web approach to recommending news. In: Nejdl W, Kay J, Pu P, Herder E (eds) Adaptive hypermedia and adaptive web-based systems. Lecture Notes in Computer Science. doi: 10.1007/978-3-540-70987-9_34, vol 5149. Springer, Berlin Heidelberg, pp 279–283
  12. 12.
    De Pessemier T, Coppens S, Geebelen K, Vleugels C, Bannier S, Mannens E, Vanhecke K, Martens L (2012) Collaborative recommendations with content-based filters for cultural activities via a scalable event distribution platform. Multimed Tools Appl 58(1):167–213. doi: 10.1007/s11042-010-0715-8 CrossRefGoogle Scholar
  13. 13.
    De Pessemier T, Deryckere T, Vanhecke K, Martens L (2008) Proposed architecture and algorithm for personalized advertising on idtv and mobile devices. IEEE Trans Consum Electron 54 (2):709–713. doi: 10.1109/TCE.2008.4560151 CrossRefGoogle Scholar
  14. 14.
    De Pessemier T, Dooms S, Martens L (2013) Comparison of group recommendation algorithms. Multimedia Tools Appl:1–45. doi: 10.1007/s11042-013-1563-0
  15. 15.
    De Pessemier T, Dooms S, Martens L (2013) Context-aware recommendations through context and activity recognition in a mobile environment. Multimed Tools Appl:1–24. doi: 10.1007/s11042-013-1582-x
  16. 16.
    Dey AK (2001) Understanding and using context. Pers Ubiquit Comput 5(1):4–7CrossRefGoogle Scholar
  17. 17.
    Dey AK, Abowd GD, Salber D (2001) A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum Comput Interact 16(2):97–166. doi: 10.1207/S15327051HCI16234_02 CrossRefGoogle Scholar
  18. 18.
    Fisher RJ (1993) Social desirability bias and the validity of indirect questioning. J Consum Res 20 (2):303–315. CrossRefGoogle Scholar
  19. 19.
    Følstad A (2008) Living labs for innovation and development of information and communication technology: a literature review. Electron J Organ Virtualness 10:99–131Google Scholar
  20. 20.
    Gavalas D, Konstantopoulos C, Mastakas K, Pantziou G (2014) Mobile recommender systems in tourism. J Netw Comput Appl 39(0):319–333. doi: 10.1016/j.jnca.2013.04.006 CrossRefGoogle Scholar
  21. 21.
    Han BJ, Rho S, Jun S, Hwang E (2010) Music emotion classification and context-based music recommendation. Multimedia Tools Appl 47(3):433–460CrossRefGoogle Scholar
  22. 22.
    Huang Z, Chen H, Zeng D (2004) Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering. ACM Trans Inf Syst 22(1):116–142. doi: 10.1145/963770.963775 CrossRefGoogle Scholar
  23. 23.
    Jannach D, Zanker M, Felfernig A, Friedrich G (2010) Recommender systems: an introduction, 1st edn. Cambridge University Press, New YorkCrossRefGoogle Scholar
  24. 24.
    Kabassi K (2010) Personalizing recommendations for tourists. Telematics Inform 27 (1):51–66. doi: 10.1016/j.tele.2009.05.003 CrossRefGoogle Scholar
  25. 25.
    Kenteris M, Gavalas D, Mpitziopoulos A (2010) A mobile tourism recommender system. In: Proceedings of the the IEEE symposium on computers and communications, ISCC ’10. IEEE Computer Society, Washington, pp 840–845Google Scholar
  26. 26.
    Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models, 5th edn. McGraw-Hill, New YorkGoogle Scholar
  27. 27.
    Panniello U, Tuzhilin A, Gorgoglione M (2014) Comparing context-aware recommender systems in terms of accuracy and diversity. User Model User-Adap Inter 24 (1-2):35–65. doi: 10.1007/s11257-012-9135-y CrossRefGoogle Scholar
  28. 28.
    Papagelis M, Plexousakis D, Kutsuras T (2005) Alleviating the sparsity problem of collaborative filtering using trust inferences. In: Herrmann P, Issarny V, Shiu S (eds) Trust management. Lecture Notes in Computer Science. doi: 10.1007/11429760_16, vol 3477. Springer, Berlin Heidelberg, pp 224–239
  29. 29.
    Resnick P, Varian HR (1997) Recommender systems. Commun ACM 40(3):56–58. doi: 10.1145/245108.245121 CrossRefGoogle Scholar
  30. 30.
    Ricci F (2010) Mobile recommender systems. Inf Technol Tour 12(3):205–231. doi: 10.3727/109830511X12978702284390 CrossRefGoogle Scholar
  31. 31.
    Ricci F (2012) Contextualizing recommendations. In: ACM RecSys workshop on context-aware recommender systems (CARS ’12), In conjunction with the 6th ACM conference on recommender systems (RECSYS ’12). ACMGoogle Scholar
  32. 32.
    Ricci F, Rokach L, Shapira B, Kantor PB (2010) Recommender systems handbook, 1st edn. Springer, New YorkMATHGoogle Scholar
  33. 33.
    Schilit BN, Theimer MM (1994) Disseminating active map information to mobile hosts. IEEE Netw 8(5):22–32. doi: 10.1109/65.313011 CrossRefGoogle Scholar
  34. 34.
    Shani G, Gunawardana A (2013) Tutorial on application-oriented evaluation of recommendation systems. AI Commun 26(2):225–236MathSciNetGoogle Scholar
  35. 35.
    Telematica Instituut / Novay (2009) Duine Framework. Available at
  36. 36.
    Thomson Reuters (Unknown Month 2008) Open Calais. Available at
  37. 37.
    Weisstein EW (1999) Chi-squared testGoogle Scholar
  38. 38.
    Yeung KF, Yang Y (2010) A proactive personalized mobile news recommendation system. In: Developments in e-systems engineering (DESE). doi: 10.1109/DeSE.2010.40, pp 207–212
  39. 39.
    Yu Z, Zhou X, Zhang D, Chin CY, Wang X, men J(2006) Supporting context-aware media recommendations for smart phones. IEEE Pervasive Comput 5(3):68–75. doi: 10.1109/MPRV.2006.61 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Toon De Pessemier
    • 1
  • Cédric Courtois
    • 2
  • Kris Vanhecke
    • 1
  • Kristin Van Damme
    • 2
  • Luc Martens
    • 1
  • Lieven De Marez
    • 2
  1. 1.Department of Information TechnologyiMinds - WiCa - Ghent UniversityGhentBelgium
  2. 2.Department of Communication SciencesiMinds - MICT - Ghent UniversityGhentBelgium

Personalised recommendations