Advertisement

Active Generation of Training Examples in Meta-Regression

  • Ricardo B. C. Prudêncio
  • Teresa B. Ludermir
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)

Abstract

Meta-Learning predicts the performance of learning algorithms based on features of the learning problems. Meta-Learning acquires knowledge from a set of meta-examples, which store the experience obtained from applying the algorithms to problems in the past. A limitation of Meta-Learning is related to the generation of meta-examples. In order to construct a meta-example, it is necessary to empirically evaluate the algorithms on a given problem. Hence, the generation of a set of meta-examples may be costly depending on the context. In order to minimize this limitation, the use of Active Learning is proposed to reduce the number of required meta-examples. In this paper, we evaluate this proposal on a promising Meta-Learning approach, called Meta-Regression. Experiments were performed in a case study to predict the performance of learning algorithms for MLP networks. A significant performance gain was observed in the case study when Active Learning was used to support the generation of meta-examples.

Keywords

Empirical Evaluation Active Method Target Attribute Random Method Performance Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Giraud-Carrier, C., Vilalta, R., Brazdil, P.: Introduction to the special issue on meta-learning. Machine Learning 54(3), 187–193 (2004)CrossRefGoogle Scholar
  2. 2.
    Vilalta, R., Giraud-Carrier, C., Brazdil, P., Soares, C.: Using meta-learning to support data- mining. Intern. Journal of Computer Science Application I(31), 31–45 (2004)zbMATHGoogle Scholar
  3. 3.
    Gama, J., Brazdil, P.: Characterization of classification algorithms. In: Pinto-Ferreira, C., Mamede, N.J. (eds.) EPIA 1995. LNCS, vol. 990, pp. 189–200. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  4. 4.
    Bensusan, H., Alexandros, K.: Estimating the predictive accuracy of a classifier. In: Proc. of the 12th European Conference on Machine Learning, pp. 25–36 (2001)Google Scholar
  5. 5.
    Koepf, C.: Meta-regression: performance prediction. In: Meta-Learning: Strategies, Implementations, and Evaluations for Algorithm Selection, pp. 89–106 (2006)Google Scholar
  6. 6.
    Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Machine Learning 15, 201–221 (1994)Google Scholar
  7. 7.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by backpropagating errors. Nature 323, 533–536 (1986)CrossRefGoogle Scholar
  8. 8.
    Brazdil, P., Soares, C., da Costa, J.: Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results. Machine Learning 50(3), 251–277 (2003)CrossRefzbMATHGoogle Scholar
  9. 9.
    Kalousis, A., Gama, J., Hilario, M.: On data and algorithms - understanding inductive performance. Machine Learning 54(3), 275–312 (2004)CrossRefzbMATHGoogle Scholar
  10. 10.
    Aha, D.: Generalizing from case studies: A case study. In: Proceedings of the 9th International Workshop on Machine Learning, pp. 1–10. Morgan Kaufmann, San Francisco (1992)Google Scholar
  11. 11.
    Kalousis, A., Hilario, M.: Representational issues in meta-learning. In: Proceedings of the 20th International Conferente on Machine Learning, pp. 313–320 (2003)Google Scholar
  12. 12.
    Prudêncio, R.B.C., Ludermir, T.B.: Meta-learning approaches to selecting time series models. Neurocomputing 61, 121–137 (2004)CrossRefGoogle Scholar
  13. 13.
    Leite, R., Brazdil, P.: Predicting relative performance of classifiers from samples. In: Proceedings of the 22nd International Conference on Machine Learning (2005)Google Scholar
  14. 14.
    Prudêncio, R.B.C., Ludermir, T.B.: Selective generation of training examples in active meta-learning. Intern. Journal of Hybrid Intelligent Systems 5, 59–70 (2008)CrossRefzbMATHGoogle Scholar
  15. 15.
    Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly Journal of Applied Mathmatics II(2), 164–168 (1944)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Prechelt, L.: A set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Universitä Karlsruhe, Germany (1994)Google Scholar
  17. 17.
    Soares, C., Brazdil, P., Kuba, P.: A meta-learning approach to select the kernel width in support vector regression. Machine Learning 54(3), 195–209 (2004)CrossRefzbMATHGoogle Scholar
  18. 18.
    Montgomery, D., Peck, E., Vining, G.: Introduction to Linear Regression Analysis, 4th edn. Wiley, Chichester (2006)zbMATHGoogle Scholar
  19. 19.
    Lindenbaum, M., Markovitch, S., Rusakov, D.: Selective sampling for nearest neighbor classifiers. Machine Learning 54, 125–152 (2004)CrossRefzbMATHGoogle Scholar
  20. 20.
    Schohn, G., Cohn, D.: Less is more: Active learning with support vector machines. In: Proc. of the 17th Intern. Conference on Machine Learning, pp. 839–846 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Ricardo B. C. Prudêncio
    • 1
  • Teresa B. Ludermir
    • 1
  1. 1.Center of InformaticsFederal University of PernambucoRecifeBrazil

Personalised recommendations