Abstract
The general problem of reconstructing an unknown function from a finite collection of samples is considered, in case the position of each input vector in the training set is not fixed beforehand, but is part of the learning process. In particular, the consistency of the Empirical Risk Minimization (ERM) principle is analyzed, when the points in the input space are generated by employing a purely deterministic algorithm (deterministic learning). When the output generation is not subject to noise, classical number-theoretic results, involving discrepancy and variation, allow to establish a sufficient condition for the consistency of the ERM principle. In addition, the adoption of low-discrepancy sequences permits to achieve a learning rate of O(1/L), being L the size of the training set. An extension to the noisy case is discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems 2, 303–314 (1989)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1995)
MacKay, D.: Information-based objective functions for active data selection. Neural Computation 4, 305–318 (1992)
Fukumizu, K.: Statistical active learning in multilayer perceptrons. IEEE Transactions on Neural Networks 11, 17–26 (2000)
Cervellera, C., Muselli, M.: Deterministic design for neural network learning: an approach based on discrepancy. To appear in the IEEE Trans. on Neural Networks (2003)
Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia (1992)
Alon, N., Spencer, J.H.: The Probabilistic Method. John Wiley & Sons, New York (2000)
Hlawka, E.: Funktionen von Beschränkter Variation in der Theorie der Gleichverteilung. Ann Mat. Pura Appl. 54, 325–333 (1961)
Blumlinger, M., Tichy, R.F.: Bemerkungen zu einigen Anwendungen gleichverteilter Folgen. Sitzungsber. Österr. Akad. Wiss. Math.-Natur. Kl. II 195, 253–265 (1986)
Cherkassky, V., Mulier, F.: Learning from Data: Concepts, Theory, and Methods. John Wiley & Sons, New York (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cervellera, C., Muselli, M. (2003). A Deterministic Learning Approach Based on Discrepancy. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds) Neural Nets. WIRN 2003. Lecture Notes in Computer Science, vol 2859. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45216-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-45216-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20227-1
Online ISBN: 978-3-540-45216-4
eBook Packages: Springer Book Archive