Abstract
Online recommendation is an important feature in many applications. In practice, the interaction between the users and the recommender system might be sparse, i.e., the users are not always interacting with the recommender system. For example, some users prefer to sweep around the recommendation instead of clicking into the details. Therefore, a response of 0 may not necessarily be a negative response, but a non-response. It comes worse to distinguish these two situations when only one item is recommended to the user each time and few further information is reachable. Most existing recommendation strategies ignore the difference between non-responses and negative responses. In this paper, we propose a novel approach, named SAOR, to make online recommendations via sparse interactions. SAOR uses positive and negative responses to build the user preference model, ignoring all non-responses. Regret analysis of SAOR is provided, experiments on both real and synthetic datasets also show that SAOR outperforms competing methods.
H. Wang—This work was done when this author was an assistant professor at Nanjing University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
See, for example, https://en.wikipedia.org/wiki/Fourier_transform.
- 3.
Here and hereafter, we may assume that the maximum possible distance
, without loss of generality. This is because, when 
, a simple rescaling can transform all the data into [0, 1]. - 4.
We do not consider the case where the user mistakenly clicks some item.
- 5.
, without loss of generality. This is because, when 
, a simple rescaling can transform all the data into [0, 1].References
Abbasi-Yadkori, Y., Pál, D., Szepesvári, C.: Improved algorithms for linear stochastic bandits. In: Advances in Neural Information Processing Systems, pp. 2312–2320 (2011)
Agrawal, S., Goyal, N.: Thompson sampling for contextual bandits with linear payoffs. In: Proceedings 30th International Conference on Machine Learning (ICML 2013), pp. 127–135 (2013)
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3(Nov), 397–422 (2002)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)
Chu, W., Li, L., Reyzin, L., Schapire, R.E.: Contextual bandits with linear payoff functions. In: Proceedings 14th International Conference on Artificial Intelligence and Statistics (AISTTS 2011), pp. 208–214 (2011)
Chuklin, A., Markov, I., Rijke, M.d.: Click models for web search. In: Synthesis Lectures on Information Concepts, Retrieval, and Services, vol. 7, no. 3, pp. 1–115 (2015)
Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19(90), 297–301 (1965)
Craswell, N., Zoeter, O., Taylor, M.J., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of International Conference on Web Search Data Min (WSDM 2008), pp. 87–94 (2008)
Gentile, C., Li, S., Kar, P., Karatzoglou, A., Zappella, G., Etrue, E.: On context-dependent clustering of bandits. In: Proceedings of 34th International Conference on Machine Learning (ICML 2013), pp. 1253–1262 (2017)
Koren, Y.: Collaborative filtering with temporal dynamics. ACM Commun. 53(4), 89–97 (2010)
Kveton, B., Szepesvari, C., Wen, Z., Ashkan, A.: Cascading bandits: learning to rank in the cascade model. In: Proceedings of 32nd International Conference on Machine Learning (ICML 2015), pp. 767–776 (2015)
Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. In: Proceedings of 19th International Conference on World Wide Web (WWW 2010), pp. 661–670 (2010)
Li, L., Chu, W., Langford, J., Wang, X.: Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In: Proceedings of 4th ACM International Conference on Web Search Data Mining (WSDM 2011), pp. 297–306. ACM (2011)
Li, L., Lu, Y., Zhou, D.: Provable optimal algorithms for generalized linear contextual bandits. arXiv preprint arXiv:1703.00048 (2017)
Li, S., Karatzoglou, A., Gentile, C.: Collaborative filtering bandits. In: Proceedings of 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 539–548. ACM (2016)
Liu, J., Dolan, P., Pedersen, E.R.: Personalized news recommendation based on click behavior. In: Proceedings of 15th International Conference on Intelligent User Interfaces (IUI 2010), pp. 31–40 (2010)
Ren, L., Gu, J., Xia, W.: A temporal item-based collaborative filtering approach. In: Signal Processing, Image Processing and Pattern Recognition (SIP 2011), pp. 414–421 (2011)
Acknowledgments
This work was supported by the National Key R&D Program of China (2017YFB0702600, 2017YFB0702601) and the National Natural Science Foundation of China (61432008, U1435214, 61503178).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, C., Wang, H., Yang, S., Gao, Y. (2019). A Contextual Bandit Approach to Personalized Online Recommendation via Sparse Interactions. In: Yang, Q., Zhou, ZH., Gong, Z., Zhang, ML., Huang, SJ. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2019. Lecture Notes in Computer Science(), vol 11440. Springer, Cham. https://doi.org/10.1007/978-3-030-16145-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-16145-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-16144-6
Online ISBN: 978-3-030-16145-3
eBook Packages: Computer ScienceComputer Science (R0)