Abstract
The performance of on-line training is compared with off-line or batch training using an unrealizable learning task. In naive off-line training this task shows a tendency to strong overfitting on the other hand its optimal training scheme is known. In the regime, where overfitting occurs, on-line training outperforms batch training quite easily. Asymptotically, off-line training is better but if the learning rate is chosen carefully on-line training remains competitive.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
M. Biehl, and H. Schwarze (1995), ”Learning by on-line gradient descent”, J. Phys. A 28 p.643–656.
S. Bös (1995), ”Avoiding overfitting by finite temperature learning and cross-validation”, in ICANN'95, edited by EC2 & Cie, Vol.2, p.111–116.
S. Bös (1996), ”Optimal weight decay in a perceptron”, this volume.
D. Saad, and S. Solla (1995), Workshop at NIPS'95, see World-Wide-Web page: http://neural-server.aston.ac.uk/nips95/workshop.html and references therein.
J.W. Kim, and H. Sompolinsky (1995) ”On-line Gibbs-learning”, preprint.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bös, S. (1996). Learning curves of on-line and off-line training. In: von der Malsburg, C., von Seelen, W., Vorbrüggen, J.C., Sendhoff, B. (eds) Artificial Neural Networks — ICANN 96. ICANN 1996. Lecture Notes in Computer Science, vol 1112. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61510-5_19
Download citation
DOI: https://doi.org/10.1007/3-540-61510-5_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61510-1
Online ISBN: 978-3-540-68684-2
eBook Packages: Springer Book Archive