CF4CF-META: Hybrid Collaborative Filtering Algorithm Selection Framework
The algorithm selection problem refers to the ability to predict the best algorithms for a new problem. This task has been often addressed by Metalearning, which looks for a function able to map problem characteristics to the performance of a set of algorithms. In the context of Collaborative Filtering, a few studies have proposed and validated the merits of different types of problem characteristics for this problem (i.e. dataset-based approach): using systematic metafeatures and performance estimations obtained by subsampling landmarkers. More recently, the problem was tackled using Collaborative Filtering models in a novel framework named CF4CF. This framework leverages the performance estimations as ratings in order to select the best algorithms without using any data characteristics (i.e algorithm-based approach). Given the good results obtained independently using each approach, this paper starts with the hypothesis that the integration of both approaches in a unified algorithm selection framework can improve the predictive performance. Hence, this work introduces CF4CF-META, an hybrid framework which leverages both data and algorithm ratings within a modified Label Ranking model. Furthermore, it takes advantage of CF4CF’s internal mechanism to use samples of data at prediction time, which has proven to be effective. This work starts by explaining and formalizing state of the art Collaborative Filtering algorithm selection frameworks (Metalearning, CF4CF and CF4CF-META) and assess their performance via an empirical study. The results show CF4CF-META is able to consistently outperform all other frameworks with statistically significant differences in terms of meta-accuracy and requires fewer landmarkers to do so.
This work is financed by the Portuguese funding institution FCT - Fundação para a Ciência e a Tecnologia through the PhD grant SFRH/BD/117531/2016. The work is also financed by European Regional Development Fund (ERDF), through the Incentive System to Research and Technological development, within the Portugal2020 Competitiveness and Internationalization Operational Program within project PushNews (POCI-01- 0247-FEDER-0024257). Lastly, the authors also acknowledge the support from Brazilian funding agencies (CNPq and FAPESP) and IBM Research and Intel.
- 3.Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to Data Mining, 1st edn. Springer Publishing, Berlin (2009)Google Scholar
- 5.Cunha, T., Soares, C., de Carvalho, A.: CF4CF: recommending collaborative filtering algorithms using collaborative filtering. ArXiv e-prints (2018)Google Scholar
- 7.Cunha, T., Soares, C., de Carvalho, A.C.P.L.F.: Recommending collaborative filtering algorithms using subsampling landmarkers. In: Yamamoto, A., Kida, T., Uno, T., Kuboyama, T. (eds.) DS 2017. LNCS (LNAI), vol. 10558, pp. 189–203. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67786-6_14CrossRefGoogle Scholar
- 8.Cunha, T., Soares, C., de Carvalho, A.C.: A label ranking approach for selecting rankings of collaborative filtering. In: ACM SAC, pp. 1393–1395 (2018)Google Scholar
- 11.Ekstrand, M., Riedl, J.: When recommenders fail: predicting recommender failure for algorithm selection and combination. In: ACM RecSys, pp. 233–236 (2012)Google Scholar
- 12.Griffith, J., O’Riordan, C., Sorensen, H.: Investigations into user rating information and accuracy in collaborative filtering. In: ACM SAC, pp. 937–942 (2012)Google Scholar
- 15.Lemke, C., Budka, M., Gabrys, B.: Metalearning: a survey of trends and technologies. Artif. Intell. Rev. 1–14 (2013)Google Scholar
- 17.Matuszyk, P., Spiliopoulou, M.: Predicting the performance of collaborative filtering algorithms. In: Web Intelligence, Mining and Semantics, pp. 38:1–38:6 (2014)Google Scholar
- 18.Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: ICML, pp. 743–750 (2000)Google Scholar
- 21.Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Analysis of recommendation algorithms for e-commerce. In: ACM Electronic Commerce, pp. 158–167 (2000)Google Scholar
- 24.Soares, C.: Labelrank: Predicting Rankings of Labels (2015). https://cran.r-project.org/package=labelrank
- 25.Vanschoren, J.: Understanding machine learning performance with experiment databases. Ph.D. thesis, Katholieke Universiteit Leuven (2010)Google Scholar