Abstract
Landmarking is a recent and promising meta-learning strategy, which defines meta-features that are themselves efficient learning algorithms. However, the choice of landmarkers is often made in an ad hoc manner. In this paper, we propose a new perspective and set of criteria for landmarkers. Based on the new criteria, we propose a landmarker generation algorithm, which generates a set of landmarkers that are each subsets of the algorithms being landmarked. Our experiments show that the landmarkers formed, when used with linear regression are able to estimate the accuracy of a set of candidate algorithms well, while only utilising a small fraction of the computational cost required to evaluate those candidate algorithms via ten-fold cross-validation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bensusan, H.: God doesn’t always shave with Occam’s Razor: learning when and how to prune. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 119–124. Springer, Heidelberg (1998)
Bensusan, H., Giraud-Carrier, C.: Casa Batló is in Passeig de Grà cia or landmarking the expertise space. In: Proc. ECML, Workshop. on Meta-learning: Building automatic advice strategies for model selection and method combination, pp. 29–46 (2000)
Blake, C., Merz, C.: UCI repository of machine learning databases. Dept. of Information and Computer Sciences. University of California, Irvine (1998)
Brazdil, P., Gama, J., Henery, R.: Characterizing the applicability of classification algorithms using meta level learning. In: Bergadano, F., De Raedt, L. (eds.) ECML 1994. LNCS, vol. 784, pp. 84–102. Springer, Heidelberg (1994)
Brazdil, P., Soares, C., Costa, J.: Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning 50(3), 251–277 (2003)
Fürnkranz, J., Petrak, J.: An evaluation of landmarking variants. In: Proc. ECML, Workshop. on integrating aspects of data mining, decision support and meta-learning, pp. 57–68 (2001)
Fürnkranz, J., Petrak, J., Brazdil, P., Soares, C.: On the use of fast subsampling estimates for algorithm recommendation. Technical Report, Austrian Research Institute for Artificial Intelligence (2002)
Giraud-Carrier, C., Vilalta, R., Brazdil, P.: Introduction to the special issue on meta-learning. Machine Learning 54(3), 187–193 (2004)
Kalousis, A., Hilario, M.: Feature selection for meta-learning. In: Cheung, D., Williams, G.J., Li, Q. (eds.) PAKDD 2001. LNCS (LNAI), vol. 2035, pp. 222–233. Springer, Heidelberg (2001)
Kalousis, A., Hilario, M.: Model selection via meta-learning: a comparative study. Int. J. Artificial Intelligence Tools 10(4), 525–554 (2001)
Köpf, C., Taylor, C., Keller, J.: Meta-analysis: from data characterisation for meta-learning to meta-regression. In: Proc. PKDD, Workshop. on Data Mining, Decision Support, Meta-learning and ILP (2000)
Michie, D., Spiegelhalter, D., Taylor, C.: Machine learning, neural and statistical classification, Ellis Horwood (1994)
Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: Proc. ICML, pp. 743–750 (2000)
Schaffer, C.: Technical note: selecting a classification method by cross-validation. Machine Learning 13(1), 135–143 (1993)
Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. J. Artificial Intelligence Review 18(2), 77–95 (2002)
Wickens, T.: The geometry of multivariate statistics. LEA Publishers (1995)
Witten, I., Frank, E.: Data mining: practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)
Wolpert, D.: The supervised learning no-free-lunch theorems. In: Proc. Soft Computing in Industry - Recent Applications, pp. 25–42 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ler, D., Koprinska, I., Chawla, S. (2004). A Landmarker Selection Algorithm Based on Correlation and Efficiency Criteria. In: Webb, G.I., Yu, X. (eds) AI 2004: Advances in Artificial Intelligence. AI 2004. Lecture Notes in Computer Science(), vol 3339. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30549-1_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-30549-1_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24059-4
Online ISBN: 978-3-540-30549-1
eBook Packages: Computer ScienceComputer Science (R0)