# A Fixed-Distribution PAC Learning Theory for Neural FIR Models

Article

Received:

Revised:

Accepted:

- 46 Downloads
- 2 Citations

## Abstract

The PAC learning theory creates a framework to assess the learning properties of static models. This theory has been extended to include learning of modeling tasks with m-dependent data given that the data are distributed according to a uniform distribution. The extended theory can be applied for learning of nonlinear FIR models with the restriction that the data are unformly distributed.

In this paper, The PAC learning scheme is extended to deal with any FIR model regardless of the distribution of the data. This fixed-distribution m-dependent extension of the PAC learning theory is then applied to the learning of FIR three-layer feedforward sigmoid neural networks.

PAC learning nonlinear FIR model multi-layer feedforward neural networks m-dependency

## Preview

Unable to display preview. Download preview PDF.

## References

- Bartlett, P.(1996). The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important Than the Size of the Network.
*Amer. Statistical Assoc. Math. Soc. Transactions*, 17, 277–364.Google Scholar - Barron, A.R. (1994). Approximation and Estimation Bounds for Artificial Neural Networks.
*Machine Learning*, 14, 115–133.MATHGoogle Scholar - Barron, A.R. (1994). Approximation and Estimation Bounds for Artificial Neural Networks.
*Machine Learning*, 14, 115–133.MATHGoogle Scholar - Benedek, G.M. and Itai, A. (1991a). Dominating Distributions and Learnability. In
*Proc. Fifth Workshop on Computational Learning Theory, ACM*(pp. 253–264).Google Scholar - Benedek, G.M. and Itai, A. (1991a). Dominating Distributions and Learnability. In
*Proc. Fifth Workshop on Computational Learning Theory, ACM*(pp. 253–264).Google Scholar - Benedek, G.M. and Itai, A. (1991b). Learnability with Respect to Fixed Distributions.
*Theoretical Computer Sys.*, 86(2), 377–390.MathSciNetGoogle Scholar - Benedek, G.M. and Itai, A. (1991b). Learnability with Respect to Fixed Distributions.
*Theoretical Computer Sys.*, 86(2), 377–390.MathSciNetGoogle Scholar - Campi, M.C. and Kumar P.R. (1996). Learning Dynamical Systems in a Stationary Environment.
*Proc. 31th IEEE Conf. Decision and Control*, 16(2), 2308–2311.Google Scholar - Campi, M.C. and Kumar P.R. (1996). Learning Dynamical Systems in a Stationary Environment.
*Proc. 31th IEEE Conf. Decision and Control*, 16(2), 2308–2311.Google Scholar - Hoeffding, W. (1961). Probability Inequalities for Sum of Bounded Random Variables.
*Amer. Statistical Assoc. Math. Soc. Transactions*, 17, 277–364.Google Scholar - Hoeffding, W. (1961). Probability Inequalities for Sum of Bounded Random Variables.
*Amer. Statistical Assoc. Math. Soc. Transactions*, 17, 277–364.Google Scholar - Iosifesco, M. and Theodorescu, R. (1969).
*Random Processes and Learning*. Springer-Verlog.Google Scholar - Iosifesco, M. and Theodorescu, R. (1969).
*Random Processes and Learning*. Springer-Verlog.Google Scholar - Najarian, K. (2001a). On Learning of Sigmoid Neural Networks.
*Complexity*, 6(4), 39–45.CrossRefMathSciNetGoogle Scholar - Najarian, K. (2001a). On Learning of Sigmoid Neural Networks.
*Complexity*, 6(4), 39–45.CrossRefMathSciNetGoogle Scholar - Najarian, K. (2001b). On Learning and Computational Complexity of FIR Radial Basis Function Networks Part I: Learning. In
*Proceedings of The 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'2001)*.Google Scholar - Najarian, K. (2001c). On Learning and Computational Complexity of FIR Radial Basis Function Networks Part II: Complexity Measures. In
*Proceedings of The 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'2001)*. Google Scholar - Najarian, K. (2002a). On Stochastic Stability of Dynamic Neural Networks.
*Discrete and Continuous Dynamical Systems*, Sup., 656–663.Google Scholar - Najarian, K. (2002b). Learning of FIR and ARX Neural Networks With Empirical Risk Minimization Algorithm. In
*Proceedings of the 15th IFAC World Conference on Automatic Control (IFAC2002), Barcelona, Spain*.Google Scholar - Najarian, K. (2002c). Learning-Based Complexity Evaluation of Radial Basis Function Networks.
*Neural Processing Letters*, 16(2), 137–150.CrossRefGoogle Scholar - Najarian, K. (2002d).FIR Volterra Kernel Neural Models and PAC Learning.
*Complexity*, 7(6), 48–55.CrossRefMathSciNetGoogle Scholar - Najarian, K. (2002d).FIR Volterra Kernel Neural Models and PAC Learning.
*Complexity*, 7(6), 48–55.CrossRefMathSciNetGoogle Scholar - Najarian, K., Dumont, G.A., and Davies, M.S. (1999a). A Learning-Theory-Based Training Algorithm for Variable-Structure Dynamic Neural Modeling. In
*Proc. Inter. Joint Conf. Neural Networks (IJCNN99)*. Google Scholar - Najarian, K., Dumont, G.A., and Davis, M.S. (2001a). Modeling of Neuromuscular Blockade System Using Neural Networks. In
*Proc. of IEEE-EMBS'2001*.Google Scholar - Najarian, K., Dumont, G.A., and Davies, M.S. (2001b). PAC Learning in Nonlinear FIR Models.
*International Journal of Adaptive Control and Signal Processing*, 15(1), 37–52.CrossRefGoogle Scholar - Najarian, K., Dumont, G.A., and Davies, M.S. (2001b). PAC Learning in Nonlinear FIR Models.
*International Journal of Adaptive Control and Signal Processing*, 15(1), 37–52.CrossRefGoogle Scholar - Najarian, K., Dumont, G.A., Davies, M.S., and Heckman, N.E. (1999b) Learning of FIR Models Under Uniform Distribution. In
*Proc. of The American Control Conference, San Dieo, U.S.A. (ACC1999)*(pp. 864–869).Google Scholar - Najarian, K., Dumont, G.A., Davies, M.S., and Heckman, N.E. (1999b) Learning of FIR Models Under Uniform Distribution. In
*Proc. of The American Control Conference, San Dieo, U.S.A. (ACC1999)*(pp. 864–869).Google Scholar - Valiant, L.G. (1984). A Theory of Learnable.
*Comm. ACM*, (pp. 1134–1142).Google Scholar - Valiant, L.G. (1984). A Theory of Learnable.
*Comm. ACM*, (pp. 1134–1142).Google Scholar - Vapnik, V.N. (1996).
*Statistical Learning Theory*. New York: Wiley.Google Scholar - Vidyasagar, M. (1997).
*A Theory of Learning and Generalization*. Springer.Google Scholar - Vidyasagar, M. (1997).
*A Theory of Learning and Generalization*. Springer.Google Scholar - Warren, D.S. and Najarian, K. (2002). Learning Theory Applied to Sigmoid Network Classification of Protein Biological Function Using Primary Structure.
*Discrete and Continuous Dynamical Systems*, Sup., 898–904.Google Scholar - Weyer, E., Williamson, R.C., and Mareels, I.M.Y. (1996). Sample Complexity of Least Squares Identification of FIR Models.
*13th IFAC Triennial World Conggrss*(pp. 239–243).Google Scholar

## Copyright information

© Springer Science+Business Media, Inc. 2005