Abstract
The PAC learning theory creates a framework to assess the learning properties of models such as the required size of the training samples and the similarity between the training and training performances. These properties, along with stochastic stability, form the main characteristics of a typical dynamic ARX modeling using neural networks. In this paper, an extension of PAC learning theory is defined which includes ARX modeling tasks, and then based on the new learning theory the learning properties of a family of neural ARX models are evaluated. The issue of stochastic stability of such networks is also addressed. Finally, using the obtained results, a cost function is proposed that considers the learning properties as well as the stochastic stability of a sigmoid neural network and creates a balance between the testing and training performances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J.P. La Salle, “Sability theory for difference equations,” MAA Studies in Mathematics, American Math. Assoc., pp. 1–31, 1977.
H.J. Kushner, Stochastic stability, in Lecture Notes in Math., Springer, New York, 1972.
H.J. Kushner, “On the stability of processes defined by stochastic difference-differential equations,” J. Diffrential Equations, vol. 4, no. 3, pp. 424–443, 1968.
P. Doukham, Mixing, properties and examples, Springer-Verlog, 1985.
H. Tong, Non-linear time series, Oxford Sceince Publications, 1990.
M.C. Campi and P.R. Kumar, “Learning dynamical systems in a stationary environment,” Proc. 31th IEEE Conf. Decision and Control, vol. 16, no. 2, pp. 2308–2311, 1996.
D. Aldous and U. Vazirani, “A Markovian extension of Valiant’s learning model,” Proc. 31th Annual IEEE Symp. on the Foundations of Comp. Sci., pp. 392–396, 1990.
P. Bartlett, Fischer, Hoeffgen, “Exploiting random walks for learning,” Proc. 7th ACM COLT, pp. 318–327, 1994.
K. Najarian, Appliation of learning theory in neural modeling of dynamic systems, Ph.D. thesis, Dpartment of Electrical and Computer Engineering, University of British Columbia, 2000.
A. Mokaddem, “Mixing properties of polynomial autoregressive processes,” Ann. Inst. H. Poincare Probab. Statist., vol. 26, no. 2, pp. 219–260, 1990.
A.R. Barron, “Approximation and estimation bounds for artificial neural net-works,” Machine Learning, vol. 14, pp. 115–133, 1994.
M. Vidyasagar, A Theory of Learning and Generalization, Springer, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Najarian, K., Dumont, G.A., Davies, M.S., Heckman, N.E. (2000). Neural ARX Models and PAC Learning. In: Hamilton, H.J. (eds) Advances in Artificial Intelligence. Canadian AI 2000. Lecture Notes in Computer Science(), vol 1822. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45486-1_25
Download citation
DOI: https://doi.org/10.1007/3-540-45486-1_25
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67557-0
Online ISBN: 978-3-540-45486-1
eBook Packages: Springer Book Archive