Skip to main content
SpringerLink
Log in

Meta-learning and Neurocomputing – A New Perspective for Computational Intelligence

  • Chapter
Foundations of Computational, Intelligence Volume 1

Part of the book series: Studies in Computational Intelligence ((SCI,volume 201))

  • 1013 Accesses

Abstract

In this chapter an analysis of computational mechanisms of induction is brought forward, in order to assess the potentiality of meta-learning methods versus the common base-learning practices. To this aim, firstly a formal investigation of inductive mechanisms is accomplished, sketching a distinction between fixed and dynamical bias learning. Then a survey is presented with suggestions and examples which have been proposed in literature to increase the efficiency of common learning algorithms. The peculiar laboratory for this kind of investigation is represented by the field of connectionist learning. To explore the meta-learning possibilities of neural network systems, knowledge-based neurocomputing techniques are considered. Among them, some kind of hybridisation strategies are particularly analysed and addressed as peculiar illustrations of a new perspective of Computational Intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abraham, A.: Meta-learning evolutionary artificial neural networks. Neurocomputing Journal 56, 1–38 (2004)

    Article  Google Scholar 

  2. Aha, D.W.: Generalizing from case studies: a case study. In: Proceedings of the Ninth International Conference on Machine Learning (MLC 1992) (1992)

    Google Scholar 

  3. Anderson, M.L., Oates, T.: A review of recent research in reasoning and metareasoning. AI Magazine 28(1), 7–16 (2007)

    Google Scholar 

  4. Baxter, J.: A model of inductive bias learning. Journal of artificial intelligence research 12, 149–198 (2000)

    MATH  MathSciNet  Google Scholar 

  5. Bensusan, H., Giraud-Carrier, C., Kennedy, C.: A higher-order approach to meta-learning. In: Proc. of the ECML 2000 workshop on MetaLearning: Building Automatic Advice Strategies for Model Selection and Method Combination, pp. 109–117 (2000)

    Google Scholar 

  6. Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)

    Google Scholar 

  7. Brazdil, P., Soares, C., Costa, J.: Ranking learning algorithms: using IBL and meta-learning on accuracy and time results. Machine Learning 50(3), 251–277 (2003)

    Article  MATH  Google Scholar 

  8. Brazdil, P.B., Soares, C.: Ranking classification algorithms based on relevant performance information. In: Proc. of the ECML 2000 Workshop on Meta-Learning: Building Automatic Advice Strategies for Model Selection and Method Combination, Barcelona, Spain (2000)

    Google Scholar 

  9. Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996)

    MATH  MathSciNet  Google Scholar 

  10. Brodley, C.: Addressing the selective superiority problem: automatic algorithm/model class selection. In: Proceedings of the Tenth International Conference on Machine Learning (MLC 1993) (1993)

    Google Scholar 

  11. Caruana, R.A.: Multitask learning. Machine Learning 28, 7–40 (1997)

    Article  Google Scholar 

  12. Castellano, G., Castiello, C., Fanelli, A.M.: KERNEL: a System for Knowledge Extraction and Refinement by NEural Learning. In: Damiani, E., Howlett, R.J., Jain, L.C., Ichalkaranje, N. (eds.) Knowledge-Based Intelligent Information Engineering Systems & Allied Technologies, pp. 443–447. IOS Press, Amsterdam (2002)

    Google Scholar 

  13. Castiello, C., Castellano, G., Fanelli, A.M.: Meta-data: Characterization of input features for meta-learning. In: Torra, V., Narukawa, Y., Miyamoto, S. (eds.) MDAI 2005. LNCS (LNAI), vol. 3558, pp. 457–468. Springer, Heidelberg (2005)

    Google Scholar 

  14. Castiello, C., Castellano, G., Fanelli, A.M.: MINDFUL: a framework for Meta-INDuctive neuro-FUzzy Learning. Information Sciences 178, 3253–3274 (2008)

    Article  MathSciNet  Google Scholar 

  15. Castiello, C., Fanelli, A.M.: Hybrid strategies and meta-learning: an inquiry into the epistemology of artificial learning. Research on Computing Science 16, 153–162 (2005)

    Google Scholar 

  16. Castiello, C., Fanelli, A.M.: Hybrid Systems for Meta-Learning (Part I): Epistemological Concerns. In: Proceedings of the International Conference on Computational Intelligence for Modelling Control and Automation (CIMCA ), pp. 180–185 (2005)

    Google Scholar 

  17. Castiello, C., Fanelli, A.M., Torsello, M.A.: The Mindful system: a meta-learning strategy into action. In: Proceedings of Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2006), pp. 548–555 (2006)

    Google Scholar 

  18. Chan, P.K., Stolfo, S.J.: Experiments on multistrategy learning by meta-learning. In: Proc. Second International Conference Information and Knowledge Management, pp. 314–323 (1993)

    Google Scholar 

  19. Chan, P.K., Stolfo, S.J.: Toward parallel and distributed learning by meta-learning. In: Working Notes AAAI Work. Know. Disc. Databases, pp. 227–240 (1993)

    Google Scholar 

  20. Chan, P.K., Stolfo, S.J.: On the accuracy of meta-learning for scalable data mining. J. Intelligent Information Systems 9, 5–28 (1997)

    Article  Google Scholar 

  21. Cloete, I., Zurada, J.M. (eds.): Knowledge-Based Neurocomputing. The MIT Press, Cambridge (2000)

    Google Scholar 

  22. Cordeschi, R., Tamburrini, G.: Intelligenza artificiale: la storia e le idee. In: Burattini, E., Cordeschi, R. (eds.) Intelligenza Artificiale. Manuale per le discipline della comunicazione. Carocci (2001)

    Google Scholar 

  23. Desjardins, M., Gordon, D.: Evaluation and selection of bias in machine learning. Machine Learning 20, 5–22 (1995)

    Google Scholar 

  24. Dietterich, T.G.: Limitations of Inductive Learning. In: Proceedings of the sixth international workshop on Machine Learning (IWML 1989), pp. 124–128 (1989)

    Google Scholar 

  25. Dietterich, T.G.: Machine Learning. Annual Review of computer science 4, 255–306 (1990)

    Article  Google Scholar 

  26. Dietterich, T.G.: Machine Learning Research: Four Current Directions. AI Magazine 18(4), 97–136 (1997)

    Google Scholar 

  27. Domingos, P.: Knowledge Discovery Via Multiple Models. Intelligent Data Analysis 2, 187–202 (1998)

    Article  Google Scholar 

  28. Duch, W., Grudzinski, K.: Meta-learning: searching in the model space. In: Proc. of the Int. Conf. on Neural Information Processing (ICONIP), pp. 235–240 (2001)

    Google Scholar 

  29. Ehrenfeucht, A., Haussler, D., Kearns, M., Valiant, L.: A general lower bound on the number of examples needed for learning. In: Proceedings of the 1988 workshop on computational learning theory, pp. 110–120. Morgan Kaufmann, San Mateo (1988)

    Google Scholar 

  30. Fogel, D.: Evolutionary Computation: Towards a New Philosophy of Machine Intelligence, 2nd edn. IEEE Press, Los Alamitos (1999)

    MATH  Google Scholar 

  31. Geman, S., Bienenstock, E., Doursat, R.: Neural network and the bias/variance dilemma. Neural Computation 4(1), 1–58 (1992)

    Article  Google Scholar 

  32. Ghosn, J., Bengio, Y.: Bias learning, knowledge sharing. IEEE Trans. Neural Network 14, 748–765 (2003)

    Article  Google Scholar 

  33. Giraud-Carrier, C., Vilalta, R., Brazdil, P.: Introduction to the special issue on meta-learning. Machine Learning 54, 187–193 (2004)

    Article  Google Scholar 

  34. Haykin, S.: Neural Networks - A Comprehensive Foundation, 2nd edn. Prentice Hall, Englewood Cliffs (1999)

    MATH  Google Scholar 

  35. Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the theory of neural computation. Addison Wesley, Reading (1991)

    Google Scholar 

  36. Holland, J.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)

    Google Scholar 

  37. Jang, J.S.R., Sun, C.T.: Neuro-Fuzzy Modeling and Control. Proceedings of the IEEE 83, 378–406 (1995)

    Article  Google Scholar 

  38. Kalousis, A., Hilario, M.: Model Selection Via Meta-Learning: a Comparative Study. In: Proceedings of the 12th International IEEE Conference on Tools with AI. IEEE Press, Los Alamitos (2000)

    Google Scholar 

  39. Kopf, C., Taylor, C., Keller, J.: Meta-analysis: from data characterisation for meta-learning to meta-regression. In: Proceedings of PKDD, workshop on data mining decision support, meta-learning and ILP (2000)

    Google Scholar 

  40. Kosko, B.: Neural Networks and Fuzzy Systems: a Dynamical Systems Approach to machine intelligence. Prentice Hall, Englewood Cliffs (1991)

    Google Scholar 

  41. Lin, C.T., Lee, C.S.G.: Neural Fuzzy System: a Neural-Fuzzy Synergism to Intelligent Systems. Prentice-Hall, Englewood Cliffs (1996)

    Google Scholar 

  42. Michalewicz, Z.: Genetic Algorithms + Data Structure = Evolution Programs. Springer, Heidelberg (1992)

    Google Scholar 

  43. Michie, D., Spiegelhalter, D.J., Taylor, C.C.: Machine learning, neural and statistical classification. Ellis Horwood Series in Artificial Intelligence (1994)

    Google Scholar 

  44. Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  45. Pfahringer, B., Bensusan, H., Giraud-Carrier, C.: Meta-learning by landmarking various learning algorithms. In: Proc. of the Seventeenth International Conference on Machine Learning, pp. 743–750 (2000)

    Google Scholar 

  46. Prodromidis, A.L., Chan, P.K., Stolfo, S.J.: Meta-learning in distributed data mining systems: Issues and approaches. In: Kargupta, H., Chan, P. (eds.) Advances of Distributed Data Mining. AAAI Press, Menlo Park (2000)

    Google Scholar 

  47. Prudêncio, R.B.C., Ludermir, T.B.: Meta-learning approaches to selecting time series models. Neurocomputing 61, 121–137 (2004)

    Article  Google Scholar 

  48. Quirolgico, S.: Communicating Neural Networks in a Multi-Agent System. Ph.D. thesis, University of Maryland, Baltimora (2002)

    Google Scholar 

  49. Rumelhart, D.E., Widrow, B., Lehr, M.: The basic ideas in neural networks. Communications of the ACM 37(3), 87–92 (1994)

    Article  Google Scholar 

  50. Schaffer, C.: A conservation law for generalization performance. In: Proceedings of the eleventh International Conference on Machine Learning (ICML 1994), pp. 259–265 (1994)

    Google Scholar 

  51. Schapire, R.E.: A Brief Introduction to Boosting. In: Proceedings of the sixteenth International Joint Conference on Artificial Intelligence (1999)

    Google Scholar 

  52. Schweighofer, N., Doya, K.: Meta-Learning in Reinforcement Learning. Neural Networks 16, 5–9 (2003)

    Article  Google Scholar 

  53. Silver, D.L.: Selective Transfer of Neural Network task knowledge. Ph.D. thesis, University of Western Ontario, London, Ontario (2000)

    Google Scholar 

  54. Silver, D.L., Mercer, R.E.: Toward a Model of Consolidation: The Retention and Transfer of Neural Net task knowledge. In: Proceedings of INNS World Congress on Neural Networks, pp. 164–169 (1995)

    Google Scholar 

  55. Soares, C., Brazdil, P.B., Kuba, P.: A Meta-Learning Method to Select the Kernel Width in Support Vector Regression. Machine Learning 54, 195–209 (2004)

    Article  MATH  Google Scholar 

  56. Thrun, S.: Lifelong learning algorithms. In: Thrun, S., Pratt, L. (eds.) Learning to learn, pp. 181–209. Kluwer Academic Publishers, Dordrecht (1998)

    Google Scholar 

  57. Thrun, S., Pratt, L. (eds.): Learning to Learn. Kluwer Academic Publishers, Dordrecht (1998)

    MATH  Google Scholar 

  58. Utgoff, P.: Shift of bias for inductive concept learning. In: Machine learning: an artificial intelligence approach. Morgan Kauffman, San Mateo (1986)

    Google Scholar 

  59. Valiant, L.: A theory of the learnable. Comm. of ACM 27, 1134–1142 (1984)

    Article  MATH  Google Scholar 

  60. Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory of probability and its applications 16, 264–280 (1971)

    Article  MATH  Google Scholar 

  61. Vilalta, R., Drissi, Y.: Research directions in Meta-Learning. In: Proceedings of the International Conference on Artificial Intelligence (ICAI 2001), Las Vegas, Nevada (2001)

    Google Scholar 

  62. Vilalta, R., Drissi, Y.: A perspective view and survey of Meta-Learning. Artificial Intelligence Review 18, 77–95 (2002)

    Article  Google Scholar 

  63. Vilalta, R., Giraud-Carrier, C., Brazdil, P.: Meta-Learning: Concepts and Techniques. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, ch. 1. Springer, Heidelberg (2005)

    Google Scholar 

  64. Wolpert, D.H.: Stacked Generalisation. Neural Networks 5, 241–259 (1992)

    Article  Google Scholar 

  65. Wolpert, D.H.: A rigorous investigation of evidence and occam factor in Bayesian reasoning. Technical Report, The Santa Fe Institute (2000)

    Google Scholar 

  66. Wolpert, D.H.: The Supervised Learning No-Free-Lunch Theorems. In: Roy, R., Koppen, M., Ovaska, S., Furuhashi, T., Hoffmann, F. (eds.) Proceedings of the Sixth Online World Conference on Soft Computing in Industrial Applications, pp. 25–42 (2001)

    Google Scholar 

  67. Wolpert, D.H., Macready, W.G.: No Free Lunch Theorems for Search. Technical Report, Santa Fe Institute (1995)

    Google Scholar 

  68. Wolpert, D.H., Macready, W.G.: No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation 1(1), 67–82 (1997)

    Article  Google Scholar 

  69. Yao, X.: Evolving Artificial Neural Networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)

    Article  Google Scholar 

  70. Yao, X., Liu, Y.: Making Use of Population Information in Evolutionary Artificial Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics - Part B 28(3), 417–425 (1998)

    MathSciNet  Google Scholar 

  71. Yao, X., Liu, Y.: Towards Designing Artificial Neural Networks by Evolution. Applied Mathematics and Computation 91, 83–90 (1998)

    Article  MATH  Google Scholar 

  72. Zadeh, L.A.: Fuzzy Sets. Information and Control 8, 338–353 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  73. Zadeh, L.A., Kacprzyk, J. (eds.): Computing with Words in Information. Physica-Verlag (1999)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Università degli Studi di Bari, via Orabona, 4, 70125, Bari, Italy

    Ciro Castiello

Authors
  1. Ciro Castiello
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Business Administration, Quantitative and Information System Department, Kuwait University, P.O. Box 5486, 13055, Safat, Kuwait

    Aboul-Ella Hassanien

  2. Center of Excellence for Quantifiable, Quality of Service, Norwegian University of Science & Technology, O.S. Bragstads plass 2E, 7491, Trondheim, Norway

    Ajith Abraham

  3. Department of Computer and Telecommunications Engineering, University ofWestern Macedonia, Agios Dimitrios Park, 50 100, Kozani, Greece

    Athanasios V. Vasilakos

  4. Dept. Electrical and Computer Engineering, University of Alberta, T6J 2V4, Edmonton,Alberta, Canada

    Witold Pedrycz

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Castiello, C. (2009). Meta-learning and Neurocomputing – A New Perspective for Computational Intelligence. In: Hassanien, AE., Abraham, A., Vasilakos, A.V., Pedrycz, W. (eds) Foundations of Computational, Intelligence Volume 1. Studies in Computational Intelligence, vol 201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01082-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01082-8_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01081-1

  • Online ISBN: 978-3-642-01082-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation