Guaranteed Convergence of Learning in Neural Networks
This paper describes schedules for the learning parameter that guarantee convergence to the optimal solution. It focuses on the diifference between local and global optimization, i.e., learning in the presence of just one minimum and learning in the presence of several minima. In case of one minimum, the fastest possible cooling is an algebraic function of the number of learning steps, whereas in case of several minima the cooling must be “exponentially slow”.
KeywordsNeural Network Network State Master Equation Physical Review Error Potential
Unable to display preview. Download preview PDF.
- T. Heskes, E. Slijpen, and B. Kappen. Cooling schedules for learning in neural networks. Physical Review E, 1993.Google Scholar
- C. Darken and J. Moody. Note on learning rate schedules for stochastic optimization. In R. Lippmann, J. Moody, and D. Touretzky, editors, Advances in Neural Information Processing Systems 3, pages 832–838, San Mateo, 1990. Morgan Kaufmann.Google Scholar
- Y. Kasbashima and S. Shinomoto. Learning a decision boundary from stochastic examples: incremental algorithms with and without queries. Preprint Kyoto University, 1992.Google Scholar
- T. Heskes and B. Kappen. On-line learning processes in artificial neural networks. In J. Taylor, editor, Mathematical Foundations of Neural Networks. Elsevier, Amsterdam, 1993.Google Scholar