Application of Learning to Learn to Real-World Pattern Recognition
Pattern recognition has for decades played an important role in the development of intelligent systems and numerous algorithms have been proposed to deal with this important task. Though much interest and research has been invested into the construction of advanced pattern recognition systems, almost all of these algorithms are incapable of meta-learning. These one-shot learning systems do not address the problem of automatic adaptation of learning bias and hence lack one of the most fundamental requirements inherent to any truly autonomous pattern recognition system: the ability to learn to learn. The purpose of this paper is to report the findings of an extensive empirical study (i.e., including 26 real-world data sets) involving a system capable of meta-learning. Statistically significant reductions in both network units and training time are observed when utilizing a multi-shot learning environment.
KeywordsHide Unit Network Unit Pattern Recognition Problem Pruning Algorithm Pattern Recognition System
Unable to display preview. Download preview PDF.
- 1.Baffes, P.T. and Zelle, J.: Growing Layers of Perceptrons: Introducing the Extentron Algorithm, Proceedings of the 1992 International Joint Conference on Neural Networks (pp. II-392–II-397), Baltimore, MD., June, 1992.Google Scholar
- 2.Belew, R.K., Mclnerney, J., and Schraudolph, N.N.: Evolving Network: using the genetic algorithm with connectionist learning, In C.G. Langton, C. Taylor, J.D. Farmer and S. Rasmussen (Eds.) Artificial Life I I, Redwood City, CA: Addison-Wesley, 1991.Google Scholar
- 3.Bellgard, M. I., Tsang, C. P.: Some Experiments on the Use of Genetic Algorithms in a Boltzmann Machine. International Joint Conference on Neural Networks, pp. 2645–2652, Singapore, 1991.Google Scholar
- 4.Fahlman, S.E. and Lebiere, C.: The Cascade-Correlation Learning Architecture, In D. Touretzky (Ed.), Advances in Neural Information Processing Systems 2 (pp. 524–532 ). San Mateo, CA.: Morgan Kaufmann, 1990.Google Scholar
- 8.Marti, L.: Genetically Generated Neural Networks II: Search for an Optimal Representation, IJCNN92, Vol. II, Baltimore, June, 1992.Google Scholar
- 9.Miller, G.F., Todd, P.M., and Hedge, S.U.: Designing neural networks using genetic algorithms, In J.D. SchafFer (Ed.) 3rd ICGA, San Mateo, CAMorgan Kaufmann, 1989.Google Scholar
- 10.Romaniuk, S.G.: Evolutionary Growth Perceptrons, In S. Forrest (Ed.) Genetic Algorithms: Proceedings of the 5th International Conference, Morgan Kaufmann, 1993.Google Scholar
- 11.Romaniuk, S.G.: Trans-Dimensional Learning, International Journal of Neural Systems, Vol. 4, No. 2 (June), 171–185, 1993.Google Scholar
- 13.Romaniuk, S.G.: Learning to Learn: Automatic Adaptation of Learning Bias, AAAI-94, Seattle, WA, USA, 1994.Google Scholar
- 14.Romaniuk, S.G.: Learning to Learn Real-World Pattern Recognition Problems, Unpublished report, Available from author, 1995.Google Scholar
- 15.Romaniuk, S.G.: Learning to Forget, Unpublished report, Available from author, 1995.Google Scholar
- 16.Romaniuk, S. G.: Semi-Weighted Neural Networks, Unpublished report, Available from author, 1995.Google Scholar