CF4CF-META: Hybrid Collaborative Filtering Algorithm Selection Framework

  • Tiago CunhaEmail author
  • Carlos Soares
  • André C. P. L. F. de Carvalho
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11198)


The algorithm selection problem refers to the ability to predict the best algorithms for a new problem. This task has been often addressed by Metalearning, which looks for a function able to map problem characteristics to the performance of a set of algorithms. In the context of Collaborative Filtering, a few studies have proposed and validated the merits of different types of problem characteristics for this problem (i.e. dataset-based approach): using systematic metafeatures and performance estimations obtained by subsampling landmarkers. More recently, the problem was tackled using Collaborative Filtering models in a novel framework named CF4CF. This framework leverages the performance estimations as ratings in order to select the best algorithms without using any data characteristics (i.e algorithm-based approach). Given the good results obtained independently using each approach, this paper starts with the hypothesis that the integration of both approaches in a unified algorithm selection framework can improve the predictive performance. Hence, this work introduces CF4CF-META, an hybrid framework which leverages both data and algorithm ratings within a modified Label Ranking model. Furthermore, it takes advantage of CF4CF’s internal mechanism to use samples of data at prediction time, which has proven to be effective. This work starts by explaining and formalizing state of the art Collaborative Filtering algorithm selection frameworks (Metalearning, CF4CF and CF4CF-META) and assess their performance via an empirical study. The results show CF4CF-META is able to consistently outperform all other frameworks with statistically significant differences in terms of meta-accuracy and requires fewer landmarkers to do so.



This work is financed by the Portuguese funding institution FCT - Fundação para a Ciência e a Tecnologia through the PhD grant SFRH/BD/117531/2016. The work is also financed by European Regional Development Fund (ERDF), through the Incentive System to Research and Technological development, within the Portugal2020 Competitiveness and Internationalization Operational Program within project PushNews (POCI-01- 0247-FEDER-0024257). Lastly, the authors also acknowledge the support from Brazilian funding agencies (CNPq and FAPESP) and IBM Research and Intel.


  1. 1.
    Adomavicius, G., Zhang, J.: Impact of data characteristics on recommender systems performance. ACM Manag. Inf. Syst. 3(1), 1–17 (2012)CrossRefGoogle Scholar
  2. 2.
    Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowl.-Based Syst. 46, 109–132 (2013)CrossRefGoogle Scholar
  3. 3.
    Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning: Applications to Data Mining, 1st edn. Springer Publishing, Berlin (2009)Google Scholar
  4. 4.
    Brazdil, P., Soares, C., da Costa, J.: Ranking learning algorithms: using IBL and meta-learning on accuracy and time. Mach. Learn. 50(3), 251–277 (2003)CrossRefGoogle Scholar
  5. 5.
    Cunha, T., Soares, C., de Carvalho, A.: CF4CF: recommending collaborative filtering algorithms using collaborative filtering. ArXiv e-prints (2018)Google Scholar
  6. 6.
    Cunha, T., Soares, C., de Carvalho, A.: Selecting collaborative filtering algorithms using metalearning. In: ECML-PKDD, pp. 393–409 (2016)CrossRefGoogle Scholar
  7. 7.
    Cunha, T., Soares, C., de Carvalho, A.C.P.L.F.: Recommending collaborative filtering algorithms using subsampling landmarkers. In: Yamamoto, A., Kida, T., Uno, T., Kuboyama, T. (eds.) DS 2017. LNCS (LNAI), vol. 10558, pp. 189–203. Springer, Cham (2017). Scholar
  8. 8.
    Cunha, T., Soares, C., de Carvalho, A.C.: A label ranking approach for selecting rankings of collaborative filtering. In: ACM SAC, pp. 1393–1395 (2018)Google Scholar
  9. 9.
    Cunha, T., Soares, C., de Carvalho, A.C.: Metalearning and recommender systems: a literature review and empirical study on the algorithm selection problem for collaborative filtering. Inf. Sci. 423, 128–144 (2018)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Ekstrand, M., Riedl, J.: When recommenders fail: predicting recommender failure for algorithm selection and combination. In: ACM RecSys, pp. 233–236 (2012)Google Scholar
  12. 12.
    Griffith, J., O’Riordan, C., Sorensen, H.: Investigations into user rating information and accuracy in collaborative filtering. In: ACM SAC, pp. 937–942 (2012)Google Scholar
  13. 13.
    Herlocker, J.L., Konstan, J.a., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Inf. Syst. 22(1), 5–53 (2004)CrossRefGoogle Scholar
  14. 14.
    Hüllermeier, E., Fürnkranz, J., Cheng, W., Brinker, K.: Label ranking by learning pairwise preferences. Artif. Intell. 172(16–17), 1897–1916 (2008)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Lemke, C., Budka, M., Gabrys, B.: Metalearning: a survey of trends and technologies. Artif. Intell. Rev. 1–14 (2013)Google Scholar
  16. 16.
    Lü, L., Medo, M., Yeung, C.H., Zhang, Y.C., Zhang, Z.K., Zhou, T.: Recommender systems. Phys. Rep. 519(1), 1–49 (2012)CrossRefGoogle Scholar
  17. 17.
    Matuszyk, P., Spiliopoulou, M.: Predicting the performance of collaborative filtering algorithms. In: Web Intelligence, Mining and Semantics, pp. 38:1–38:6 (2014)Google Scholar
  18. 18.
    Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: ICML, pp. 743–750 (2000)Google Scholar
  19. 19.
    Pinto, F., Soares, C., Mendes-Moreira, J.: Towards automatic generation of metafeatures. In: PAKDD, pp. 215–226 (2016)CrossRefGoogle Scholar
  20. 20.
    Rice, J.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)CrossRefGoogle Scholar
  21. 21.
    Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Analysis of recommendation algorithms for e-commerce. In: ACM Electronic Commerce, pp. 158–167 (2000)Google Scholar
  22. 22.
    Serban, F., Vanschoren, J., Bernstein, A.: A survey of intelligent assistants for data analysis. ACM Comput. Surv. V(212), 1–35 (2013)CrossRefGoogle Scholar
  23. 23.
    Smith-Miles, K.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 6:1–6:25 (2008)CrossRefGoogle Scholar
  24. 24.
    Soares, C.: Labelrank: Predicting Rankings of Labels (2015).
  25. 25.
    Vanschoren, J.: Understanding machine learning performance with experiment databases. Ph.D. thesis, Katholieke Universiteit Leuven (2010)Google Scholar
  26. 26.
    Vembu, S., Gärtner, T.: Label ranking algorithms: a survey. In: Preference Learning, pp. 45–64 (2010)CrossRefGoogle Scholar
  27. 27.
    Yang, X., Guo, Y., Liu, Y., Steck, H.: A survey of collaborative filtering based social recommender systems. Comput. Commun. 41, 1–10 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Tiago Cunha
    • 1
    Email author
  • Carlos Soares
    • 1
  • André C. P. L. F. de Carvalho
    • 2
  1. 1.Faculdade de Engenharia da Universidade do PortoPortoPortugal
  2. 2.Universidade de São Paulo, ICMCSão CarlosBrazil

Personalised recommendations