Abstract
Meta-learning for model selection, as reported in the symbolic machine learning community, can be described as follows. First, it is cast as a purely data-driven predictive task. Second, it typically relies on a mapping of dataset characteristics to some measure of generalization performance (e.g., error). Third, it tends to ignore the role of algorithm parameters by relying mostly on default settings. This paper describes a case-based system for model selection which combines knowledge and data in selecting a (set of) algorithm(s) to recommend for a given task. The knowledge consists mainly of the similarity measures used to retrieve records of past learning experiences as well as profiles of learning algorithms incorporated into the conceptual meta-model. In addition to the usual dataset characteristics and error rates, the case base includes objects describing the evaluation strategy and the learner parameters used. These have two major roles: they ensure valid and meaningful comparisons between independently reported findings, and they facilitate replication of past experiments. Finally, the case-based meta-learner can be used not only as a predictive tool but also as an exploratory tool for gaining further insight into previously tested algorithms and datasets.
Chapter PDF
References
A. Agnar and E. Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI Communications, 7(1), 1994.
H. Blockeel. Cumulativity as inductive bias. In Data Mining, Decision Support, Meta-Learning and ILP: Forum for Practical Problem Presentation and Prospective Solutions, pages 61–70, Lyon, France, July 2000.
W. W. Cohen. Fast effective rule induction. In A. Prieditis and S. Russell, editors, Proc. of the 11th International Conference on Machine Learning, pages 115–123, Tahoe City, CA, 1995. Morgan Kaufmann.
J. Gama and P. Brazdil. Linear tree. Intelligent Data Analysis, 3:1–22, 1999.
M. Hilario and A. Kalousis. Quantifying the resilience of inductive classification algorithms. In Principles of Data Mining and Knowledge Discovery. Proceedings of the 4th European Conference, pages 106–115, Lyon, France, 2000. Springer-Verlag.
A. Kalousis and M. Hilario. Supervised knowledge discovery from incomplete data. In International Conference on Data Mining, Cambridge, UK, 2000.
R. Kohavi, G. John, R. Long, D. Manley, and K. Pfleger. MLC++: A machine learning library in C++. Technical report, CSD, Stanford University, August 1994. An abridged version of this report appears in AI’94: Tools in AI.
R. Kohavi and D. Wolpert. Bias plus variance decomposition for zero-one loss functions. In L. Saitta, editor, Proc. of the 13th International Conference on Machine Learning, pages 275–283, Bari (Italy), 1996. Morgan Kaufmann.
H. Linhart and W. Zucchini. Model Selection. J. Wiley, NY, 1986.
R. J. Little and D. B. Rubin. Statistical Analysis withMissing Data. Wiley, 1987.
MetaL Consortium. Project Homepage. http://www.metal-kdd.org/.
D. Michie, D. J. Spiegelhalter, and C. C. Taylor, editors. Machine learning, neural and statistical classification. Ellis-Horwood, 1994.
Bernhard Pfahringer, Hilan Bensusan, and Christophe Giraud-Carrier. Metalearning by landmarking various learning algorithms. In Proc. Seventeenth International Conference on Machine Learning, ICML’2000, pages 743–750, San Francisco, California, June 2000. Morgan Kaufmann.
J. R. Quinlan. Comparing connectionist and symbolic learning methods. In S. J. Hanson, G. A. Drastal, and R. L. Rivest, editors, Computational Learning Theory and Natural Learning Systems, volume I, chapter 15, pages 446–456. MIT Press, 1994.
B.D. Ripley. Pattern Recognition and Neural Networks. Cambridge U. Press, 1996.
L. Todorowski. Experiments in meta-level learning with ILP. In International Workshop on Inductive Logic Programming, Bled, Slovenia, 1999. Springer-Verlag.
D. Wolpert. The lack of a priori distinctions between learning algorithms. Neural Computation, 8(7):1381–1390, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hilario, M., Kalousis, A. (2001). Fusion of Meta-knowledge and Meta-data for Case-Based Model Selection. In: De Raedt, L., Siebes, A. (eds) Principles of Data Mining and Knowledge Discovery. PKDD 2001. Lecture Notes in Computer Science(), vol 2168. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44794-6_15
Download citation
DOI: https://doi.org/10.1007/3-540-44794-6_15
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42534-2
Online ISBN: 978-3-540-44794-8
eBook Packages: Springer Book Archive