Advertisement

Automation and Remote Control

, Volume 79, Issue 9, pp 1621–1629 | Cite as

Learning Radial Basis Function Networks with the Trust Region Method for Boundary Problems

  • L. N. Elisov
  • V. I. Gorbachenko
  • M. V. Zhukov
Intellectual Control Systems, Data Analysis
  • 21 Downloads

Abstract

We consider the solution of boundary value problems of mathematical physics with neural networks of a special form, namely radial basis function networks. This approach does not require one to construct a difference grid and allows to obtain an approximate analytic solution at an arbitrary point of the solution domain. We analyze learning algorithms for such networks. We propose an algorithm for learning neural networks based on the method of trust region. The algorithm allows to significantly reduce the learning time of the network.

Keywords

boundary value problems of mathematical physics radial basis function networks learning of neural networks method of trust region 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Tolstykh, A.I. and Shirobokov, D.A., Mesh-Free Method Based on Radial Basis Functions, Zh. Vychisl. Mat. Mat. Fiz., 2005, vol. 45, no. 8, pp. 1498–1505.MathSciNetzbMATHGoogle Scholar
  2. 2.
    Liu, G.R., Mesh-Free Methods: Moving Beyond the Finite Element Method, Boca Raton: CRC Press, 2003.zbMATHGoogle Scholar
  3. 3.
    Kansa, E.J., Motivation for Using Radial Basis Function to Solve PDEs. http://www.cityu.edu.hk/rbf-pde/files/overview-pdf.pdf
  4. 4.
    Buhmann, M.D., Radial Basis Functions: Theory and Implementations, Cambridge: Cambridge Univ. Press, 2004.zbMATHGoogle Scholar
  5. 5.
    Fasshauer, G.E., Meshfree Approximation Methods with MATLAB, River Edge: World Scientific, 2007.CrossRefzbMATHGoogle Scholar
  6. 6.
    Chen, W. and Fu, Z.J., Recent Advances in Radial Basis Function Collocation Methods, New York: Springer, 2014.CrossRefzbMATHGoogle Scholar
  7. 7.
    Fornberg, B. and Flyer, N., Solving PDEs with Radial Basis Functions, Acta Numerica, 2015, vol. 4, pp. 215–258.MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Jianyu, L., Siwei, L., Yingjian, Q., and Yaping, H., Numerical Solution of Differential Equations by Radial Basis Function Neural Networks, Proc. Int. Joint Conf. on Neural Networks, 2002, vol. 1, pp. 773–777.Google Scholar
  9. 9.
    Mai-Duy, N. and Tran-Cong, T., Solving High Order Ordinary Differential Equations with Radial Basis Function Networks, Int. J. Numer. Methods Eng., 2005, vol. 62, pp. 824–852.MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Sarra, S., Adaptive Radial Basis Function Methods for Time Dependent Partial Differential Equations, Appl. Numer. Math., 2005, vol. 54, no. 1, pp. 79–94.MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Chen, H., Kong, L., and Leng, W., Numerical Solution of PDEs via Integrated Radial Basis Function Networks with Adaptive Training Algorithm, Appl. Soft Comput., 2011, vol. 11, no. 1, pp. 855–860.CrossRefGoogle Scholar
  12. 12.
    Kumar, M. and Yadav, N., Multilayer Perceptrons and Radial Basis Function Neural Network Methods for the Solution of Differential Equations: A Survey, Comput. Math. Appl., 2011, vol. 62, pp. 3796–3811.MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Yadav, N., Yadav, A., and Kumar, M., An Introduction to Neural Network Methods for Differential Equations, New York: Springer, 2015.CrossRefzbMATHGoogle Scholar
  14. 14.
    Tarkhov, D.A., Neirosetevye modeli i algoritmy. Spravochnik (Neural Network Models and Algorithms. Reference), Moscow: Radiotekhnika, 2014.Google Scholar
  15. 15.
    Gorbachenko, V.I. and Artyukhina, E.V., Mesh-Free Methods and Their Implementation with Radial Basis Neural Networks, Neirokomp’yut.: Razrabotka, Primen., 2010, no. 11, pp. 4–10.Google Scholar
  16. 16.
    Gorbachenko, V.I. and Zhukov, M.V., Solving Boundary Value Problems of Mathematical Physics Using Radial Basis Function Networks, Comput. Math. Math. Phys., 2017, vol. 57, no. 1, pp. 145–155.MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Goodfellow, I., Bengio, Y., and Courville, A., Deep Learning, Boston: MIT Press, 2017. Translated under the title Glubokoe obuchenie, Moscow: DMK Press, 2018.zbMATHGoogle Scholar
  18. 18.
    Jia, W., Zhao, D., Shen, T., Su, C., Hu, C., and Zhao, Y., A New Optimized GA-RBF Neural Network Algorithm, Comput. Intelligence Neurosci., 2014, Article ID982045.Google Scholar
  19. 19.
    Gill, P.E., Murray, W., and Wright, M.H., Practical Optimization, London: Academic, 1981. Translated under the title Prakticheskaya optimizatsiya, Moscow: Mir, 1985.zbMATHGoogle Scholar
  20. 20.
    Sutskever, I., Martens, J., Dahl, G., and Hinton, G., On the Importance of Initialization and Momentum in Deep Learning, Proc. 30th Int. Conf. on Machine Learning, 2013, vol. 28, pp. 1139–1147.Google Scholar
  21. 21.
    Alkezuini, M.M. and Gorbachenko, V.I., Training Networks of Radial Basis Functions with Nesterov’s Method for Solving Boundary Problems of Mathematical Physics, Proc. XII Intl. Sci.-Tech. Conf. Analytic and Numerical Methods of Modeling Natural Science and Social Problems, Penza: PGU, 2017, pp. 171–175.Google Scholar
  22. 22.
    Fletcher, R. and Reeves, C.M., Function Minimization by Conjugate Gradients, Comput. J., 1964, vol. 7, pp. 149–154.MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Polak, E. and Ribiére, G., Note sur la convergence de méthodes de directions conjugués, Revue Française d’Inform. Recherche Opération., Série Rouge, 1969, vol. 3, no. 1, pp. 35–43.zbMATHGoogle Scholar
  24. 24.
    Zhang, L., Li, K., He, H., and Irwin, G.W., A New Discrete-Continuous Algorithm for Radial Basis Function Networks Construction, IEEE Trans. Neural Networks Learning Syst., 2013, vol. 24, no. 11, pp. 1785–1798.CrossRefGoogle Scholar
  25. 25.
    Xie, T., Yu, H., Hewlett, J., Rozycki, P., and Wilamowski, B., Fast and Efficient Second-Order Method for Training Radial Basis Function Networks, IEEE Trans. Neural Networks Learning Syst., 2012, vol. 23, no. 4, pp. 609–619.CrossRefGoogle Scholar
  26. 26.
    Markopoulos, A.P., Georgiopoulos, S., and Manolakos, D.E., On the Use of Back Propagation and Radial Basis Function Neural Networks in Surface Roughness Prediction, J. Indust. Engin. Int., 2016, vol. 12, no. 3, pp. 389–400.CrossRefGoogle Scholar
  27. 27.
    Zhang, L., Li, K., and Wang, W., An Improved Conjugate Gradient Algorithm for Radial Basis Function (RBF) Networks Modelling, in Proc. UKACC Int. Conf. on Control, 2012, pp. 19–23.Google Scholar
  28. 28.
    Sadeghi, M., Pashaie, M., and Jafarian, A., RBF Neural Networks Based on BFGS Optimization Method for Solving Integral Equations, Adv. Appl. Math. Biosci., 2016, vol. 7, no. 1, pp. 1–22.Google Scholar
  29. 29.
    Conn, A.R., Gould, N.I.M., and Toint, P.L., Trust-Region Methods, MPS-SIAM, 1987.zbMATHGoogle Scholar
  30. 30.
    Wild, S.M., Regis, R.G., and Shoemaker, C.A., ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions, SIAM J. Scientific Comput., 2008, vol. 30, no. 6, pp. 3197–3219.MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Bernal, F., Trust-Region Methods for Nonlinear Elliptic Equations with Radial Basis Functions, Comput. Math. Appl., 2016, vol. 72, no. 7, pp. 1743–1763.MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Gorbachenko, V.I. and Zhukov, M.V., The Learning of Radial Basis Function Network Using Trust Region Method to Solve Poisson’s Equation, Inform. Tekhnol., 2013, no. 9, pp. 65–70.Google Scholar
  33. 33.
    Brink, H., Richards, J.W., and Fetherolf, M., Real-World Machine Learning, Shelter Island: Manning, 2016. Translated under the title Mashinnoe obuchenie, St. Petersburg: Piter, 2017.Google Scholar
  34. 34.
    Steihaug, T., The Conjugate Gradient Method and Trust Region in Large Scale Optimization, SIAM J. Numer. Anal., 1983, vol. 20, no. 3, pp. 626–637.MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Watkins, D.S., Fundamentals of Matrix Computations, New York: Springer, 1991. Translated under the title Osnovy matrichnykh vychislenii, Moscow: Binom, 2012.zbMATHGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2018

Authors and Affiliations

  • L. N. Elisov
    • 1
  • V. I. Gorbachenko
    • 2
  • M. V. Zhukov
    • 2
  1. 1.Moscow State Technical University of Civil AviationMoscowRussia
  2. 2.Penza State UniversityPenzaRussia

Personalised recommendations