A user-centric evaluation of context-aware recommendations for a mobile news service
- 622 Downloads
Traditional recommender systems provide personal suggestions based on the user’s preferences, without taking into account any additional contextual information, such as time or device type. The added value of contextual information for the recommendation process is highly dependent on the application domain, the type of contextual information, and variations in users’ usage behavior in different contextual situations. This paper investigates whether users utilize a mobile news service in different contextual situations and whether the context has an influence on their consumption behavior. Furthermore, the importance of context for the recommendation process is investigated by comparing the user satisfaction with recommendations based on an explicit static profile, content-based recommendations using the actual user behavior but ignoring the context, and context-aware content-based recommendations incorporating user behavior as well as context. Considering the recommendations based on the static profile as a reference condition, the results indicate a significant improvement for recommendations that are based on the actual user behavior. This improvement is due to the discrepancy between explicitly stated preferences (initial profile) and the actual consumption behavior of the user. The context-aware content-based recommendations did not significantly outperform the content-based recommendations in our user study. Context-aware content-based recommendations may induce a higher user satisfaction after a longer period of service operation, enabling the recommender to overcome the cold-start problem and distinguish user preferences in various contextual situations.
KeywordsRecommender system News recommendation User evaluation Context-aware Algorithm-based news
This research was performed in the context of the iMinds-MIX Stream Store project. Stream Store is a project cofunded by iMinds (Interdisciplinary institute for Technology) a research institute founded by the Flemish Government. Companies and organizations involved in the project are De Persgroep, Roularta Media Group nv, iMinds-iLab.o, VMMA, and Limecraft with project support of IWT.
The authors would also like to thank the researchers of MIX for the development of the Stream Store client application and the team of the iMinds-MMLab research group for the processing of the metadata. For the setup of the user experiment and performing the evaluation, the authors would like to express their gratitude to the students of the iMinds-MICT research group.
- 3.Adomavicius G, Tuzhilin A (2008) Tutorial on context-aware recommender systems. In: Proceedings of the 2nd ACM conference on recommender systems (RecSys ’08)Google Scholar
- 6.Baltrunas L, Ricci F (2009) Context-based splitting of item ratings in collaborative filtering. In: Proceedings of the 3rd ACM conference on recommender systems, RecSys ’09. doi: 10.1145/1639714.1639759. ACM, New York, pp 245–248
- 7.Bazire M, Brézillon P (2005) Understanding context before using it. In: Dey A, Kokinov B, Leake D, Turner R (eds) Modeling and using context. Lecture Notes in Computer Science. doi: 10.1007/11508373_3, vol 3554. Springer, Berlin Heidelberg, pp 29–40
- 8.Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 14th conference on uncertainty in artificial intelligence, UAI’98. http://dl.acm.org/citation.cfm?id=2074094.2074100. Morgan Kaufmann, San Francisco, pp 43–52
- 11.Cantador I, Bellogín A, Castells P (2008) News@hand: a semantic web approach to recommending news. In: Nejdl W, Kay J, Pu P, Herder E (eds) Adaptive hypermedia and adaptive web-based systems. Lecture Notes in Computer Science. doi: 10.1007/978-3-540-70987-9_34, vol 5149. Springer, Berlin Heidelberg, pp 279–283
- 12.De Pessemier T, Coppens S, Geebelen K, Vleugels C, Bannier S, Mannens E, Vanhecke K, Martens L (2012) Collaborative recommendations with content-based filters for cultural activities via a scalable event distribution platform. Multimed Tools Appl 58(1):167–213. doi: 10.1007/s11042-010-0715-8 CrossRefGoogle Scholar
- 14.De Pessemier T, Dooms S, Martens L (2013) Comparison of group recommendation algorithms. Multimedia Tools Appl:1–45. doi: 10.1007/s11042-013-1563-0
- 15.De Pessemier T, Dooms S, Martens L (2013) Context-aware recommendations through context and activity recognition in a mobile environment. Multimed Tools Appl:1–24. doi: 10.1007/s11042-013-1582-x
- 19.Følstad A (2008) Living labs for innovation and development of information and communication technology: a literature review. Electron J Organ Virtualness 10:99–131Google Scholar
- 25.Kenteris M, Gavalas D, Mpitziopoulos A (2010) A mobile tourism recommender system. In: Proceedings of the the IEEE symposium on computers and communications, ISCC ’10. IEEE Computer Society, Washington, pp 840–845Google Scholar
- 26.Kutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied linear statistical models, 5th edn. McGraw-Hill, New YorkGoogle Scholar
- 28.Papagelis M, Plexousakis D, Kutsuras T (2005) Alleviating the sparsity problem of collaborative filtering using trust inferences. In: Herrmann P, Issarny V, Shiu S (eds) Trust management. Lecture Notes in Computer Science. doi: 10.1007/11429760_16, vol 3477. Springer, Berlin Heidelberg, pp 224–239
- 31.Ricci F (2012) Contextualizing recommendations. In: ACM RecSys workshop on context-aware recommender systems (CARS ’12), In conjunction with the 6th ACM conference on recommender systems (RECSYS ’12). ACMGoogle Scholar
- 35.Telematica Instituut / Novay (2009) Duine Framework. Available at http://duineframework.org/
- 36.Thomson Reuters (Unknown Month 2008) Open Calais. Available at http://www.opencalais.com/
- 37.Weisstein EW (1999) Chi-squared testGoogle Scholar
- 38.Yeung KF, Yang Y (2010) A proactive personalized mobile news recommendation system. In: Developments in e-systems engineering (DESE). doi: 10.1109/DeSE.2010.40, pp 207–212