Application of Learning to Learn to Real-World Pattern Recognition

  • Steve G. Romaniuk
Conference paper


Pattern recognition has for decades played an important role in the development of intelligent systems and numerous algorithms have been proposed to deal with this important task. Though much interest and research has been invested into the construction of advanced pattern recognition systems, almost all of these algorithms are incapable of meta-learning. These one-shot learning systems do not address the problem of automatic adaptation of learning bias and hence lack one of the most fundamental requirements inherent to any truly autonomous pattern recognition system: the ability to learn to learn. The purpose of this paper is to report the findings of an extensive empirical study (i.e., including 26 real-world data sets) involving a system capable of meta-learning. Statistically significant reductions in both network units and training time are observed when utilizing a multi-shot learning environment.


Hide Unit Network Unit Pattern Recognition Problem Pruning Algorithm Pattern Recognition System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baffes, P.T. and Zelle, J.: Growing Layers of Perceptrons: Introducing the Extentron Algorithm, Proceedings of the 1992 International Joint Conference on Neural Networks (pp. II-392–II-397), Baltimore, MD., June, 1992.Google Scholar
  2. 2.
    Belew, R.K., Mclnerney, J., and Schraudolph, N.N.: Evolving Network: using the genetic algorithm with connectionist learning, In C.G. Langton, C. Taylor, J.D. Farmer and S. Rasmussen (Eds.) Artificial Life I I, Redwood City, CA: Addison-Wesley, 1991.Google Scholar
  3. 3.
    Bellgard, M. I., Tsang, C. P.: Some Experiments on the Use of Genetic Algorithms in a Boltzmann Machine. International Joint Conference on Neural Networks, pp. 2645–2652, Singapore, 1991.Google Scholar
  4. 4.
    Fahlman, S.E. and Lebiere, C.: The Cascade-Correlation Learning Architecture, In D. Touretzky (Ed.), Advances in Neural Information Processing Systems 2 (pp. 524–532 ). San Mateo, CA.: Morgan Kaufmann, 1990.Google Scholar
  5. 5.
    Fogel, D.B., Fogel, L.J., and Porto, V.W.: Evolving Neural Networks, Biological Cybernetics, 63, 487–493, 1990.CrossRefGoogle Scholar
  6. 6.
    Frean, M.: The Upstart Algorithm: A Method for Constructing and Training FeedForward Neural Networks, Neural Computation, 2, 198–209, 1991.CrossRefGoogle Scholar
  7. 8.
    Marti, L.: Genetically Generated Neural Networks II: Search for an Optimal Representation, IJCNN92, Vol. II, Baltimore, June, 1992.Google Scholar
  8. 9.
    Miller, G.F., Todd, P.M., and Hedge, S.U.: Designing neural networks using genetic algorithms, In J.D. SchafFer (Ed.) 3rd ICGA, San Mateo, CAMorgan Kaufmann, 1989.Google Scholar
  9. 10.
    Romaniuk, S.G.: Evolutionary Growth Perceptrons, In S. Forrest (Ed.) Genetic Algorithms: Proceedings of the 5th International Conference, Morgan Kaufmann, 1993.Google Scholar
  10. 11.
    Romaniuk, S.G.: Trans-Dimensional Learning, International Journal of Neural Systems, Vol. 4, No. 2 (June), 171–185, 1993.Google Scholar
  11. 12.
    Romaniuk, S.G.: Pruning Divide & Conquer Networks, Network: Computation in Neural Systems, 4, 481–494, 1993.CrossRefGoogle Scholar
  12. 13.
    Romaniuk, S.G.: Learning to Learn: Automatic Adaptation of Learning Bias, AAAI-94, Seattle, WA, USA, 1994.Google Scholar
  13. 14.
    Romaniuk, S.G.: Learning to Learn Real-World Pattern Recognition Problems, Unpublished report, Available from author, 1995.Google Scholar
  14. 15.
    Romaniuk, S.G.: Learning to Forget, Unpublished report, Available from author, 1995.Google Scholar
  15. 16.
    Romaniuk, S. G.: Semi-Weighted Neural Networks, Unpublished report, Available from author, 1995.Google Scholar

Copyright information

© Springer-Verlag/Wien 1995

Authors and Affiliations

  • Steve G. Romaniuk
    • 1
  1. 1.Universal Problem SolversClearwaterUSA

Personalised recommendations