Proposal for the 1st Interdisciplinary Workshop on Algorithm Selection and Meta-Learning in Information Retrieval (AMIR)

  • Joeran BeelEmail author
  • Lars Kotthoff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11438)


The algorithm selection problem describes the challenge of identifying the best algorithm for a given problem space. In many domains, particularly artificial intelligence, the algorithm selection problem is well studied, and various approaches and tools exist to tackle it in practice. Especially through meta-learning impressive performance improvements have been achieved. The information retrieval (IR) community, however, has paid little attention to the algorithm selection problem, although the problem is highly relevant in information retrieval. This workshop will bring together researchers from the fields of algorithm selection and meta-learning as well as information retrieval. We aim to raise the awareness in the IR community of the algorithm selection problem; identify the potential for automatic algorithm selection in information retrieval; and explore possible solutions for this context. In particular, we will explore to what extent existing solutions to the algorithm selection problem from other domains can be applied in information retrieval, and also how techniques from IR can be used for automated algorithm selection and meta-learning.


  1. 1.
    Beel, J., Gipp, B., Langer, S., Breitinger, C.: Research paper recommender systems: a literature survey. Int. J. Digit. Libr. 305–338 (2016)CrossRefGoogle Scholar
  2. 2.
    Rice, J.R.: The algorithm selection problem (1975)Google Scholar
  3. 3.
    Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla: portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. 32, 565–606 (2008)CrossRefGoogle Scholar
  4. 4.
    Kotthoff, L., Thornton, C., Hoos, H.H., Hutter, F., Leyton-Brown, K.: Auto-WEKA 2.0: automatic model selection and hyperparameter optimization in WEKA. J. Mach. Learn. Res. 18, 826–830 (2017)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Lindauer, M., van Rijn, J.N., Kotthoff, L.: The algorithm selection competition series 2015-17. arXiv preprint arXiv:1805.01214 (2018)
  6. 6.
    Tu, W.-W.: The 3rd AutoML challenge: AutoML for lifelong machine learning. In: NIPS 2018 Challenge (2018)Google Scholar
  7. 7.
    Brazdil, P.: Metalearning and algorithm selection. In: 21st European Conference on Artificial Intelligence (ECAI) (2014)Google Scholar
  8. 8.
    Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection and configuration. Report from Dagstuhl Seminar 16412, vol. 6 (2016)Google Scholar
  9. 9.
    Vanschoren, J., Brazdil, P., Giraud-Carrier, C., Kotthoff, L.: Meta-learning and algorithm selection workshop at ECMLPKDD. In: CEUR Workshop Proceedings (2015)Google Scholar
  10. 10.
    Calandra, R., Hutter, F., Larochelle, H., Levine, S.: Workshop on meta-learning (MetaLearn 2017) @NIPS (2017).
  11. 11.
    Miikkulainen, R., Le, Q., Stanley, K., Fernando, C.: Metalearning symposium @NIPS (2017).
  12. 12.
    Vanschoren, J., Van Rijn, J.N., Bischl, B., Torgo, L.: OpenML: networked science in machine learning. ACM SIGKDD Explor. Newsl. 15, 49–60 (2014)CrossRefGoogle Scholar
  13. 13.
    Ahsan, M., Ngo-Ye, L.: A Conceptual model of recommender system for algorithm selection. In: AMCIS 2005 Proceedings, p. 122 (2005)Google Scholar
  14. 14.
    Collins, A., Tkaczyk, D., Beel, J.: A novel approach to recommendation algorithm selection using meta-learning. In: Proceedings of the 26th Irish Conference on Artificial Intelligence and Cognitive Science (AICS), CEUR-WS, pp. 210–219 (2018)Google Scholar
  15. 15.
    Cunha, T., Soares, C., de Carvalho, A.C.: Metalearning and recommender systems: a literature review and empirical study on the algorithm selection problem for collaborative filtering. Inf. Sci. 423, 128–144 (2018)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Cunha, T., Soares, C., de Carvalho, A.C.: CF4CF: recommending collaborative filtering algorithms using collaborative filtering. arXiv preprint arXiv:1803.02250 (2018)
  17. 17.
    Cunha, T., Soares, C., de Carvalho, A.C.P.L.F.: Selecting collaborative filtering algorithms using metalearning. In: Frasconi, P., Landwehr, N., Manco, G., Vreeken, J. (eds.) ECML PKDD 2016. LNCS (LNAI), vol. 9852, pp. 393–409. Springer, Cham (2016). Scholar
  18. 18.
    Matuszyk, P., Spiliopoulou, M.: Predicting the performance of collaborative filtering algorithms. In: Proceedings of the 4th International Conference on Web Intelligence, Mining and Semantics (WIMS 2014), p. 38. ACM (2014)Google Scholar
  19. 19.
    Mısır, M., Sebag, M.: ALORS: an algorithm recommender system. Artif. Intell. 244, 291–314 (2017)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Romero, C., Olmo, J.L., Ventura, S.: A meta-learning approach for recommending a subset of white-box classification algorithms for Moodle datasets. In: Educational Data Mining (2013)Google Scholar
  21. 21.
    Vartak, M., Thiagarajan, A., Miranda, C., Bratman, J., Larochelle, H.: A meta-learning perspective on cold-start recommendations for items. In: Advances in Neural Information Processing Systems, pp. 6907–6917 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Computer Science and Statistics, Artificial Intelligence Discipline, ADAPT CentreTrinity College DublinDublinIreland
  2. 2.Department of Computer Science, Meta Algorithmics, Learning and Large-Scale Empirical Testing LabUniversity of WyomingLaramieUSA

Personalised recommendations