Skip to main content
SpringerLink
Log in
Book cover

Advances in Soft Computing, Intelligent Robotics and Control pp 127–143Cite as

Modeling of Complex Multidimensional Nonlinear Systems Using Neural System with Deep Architectures

  • Conference paper

  • 1039 Accesses

Part of the book series: Topics in Intelligent Engineering and Informatics ((TIEI,volume 8))

Abstract

Reviews of several methods for modeling of complex multidimensional nonlinear systems were presented. It turns out that power of neural networks grows linearly with its width and exponentially with its depth. Unfortunately training of traditional MLP Multi-Layer Perceptron deep architectures is very difficult. The paper presents couple solution to this problem. One solution is to use neural network with connections across layers such as FCC Fully Connected Cascade or BMLP - Bridged Multi-Layer Perceptron. Another alternative is to use DNN - Dual Neural Networks which is described with more details.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Wilamowski, B.M.: Neural Network Architectures and Learning algorithms – How Not to Be Frustrated with Neural Networks. IEEE Industrial Electronics Magazine 3(4), 56–63 (2009) (best paper award)

    Article  Google Scholar 

  2. Hecht-Nielsen, R.: Theory of the Back Propagation Neural Network. In: Proc. 1989 IEEE IJCNN, pp. 1593–1605. IEEE Press, New York (1989)

    Google Scholar 

  3. Peng, J., Li, K., Irwin, G.W.: A New Jacobian Matrix for Optimal Learning of Single-Layer Neural Networks. IEEE Trans. on Neural Networks 19(1), 119–129 (2008)

    Article  Google Scholar 

  4. Wilamowski, B.M., Yu, H.: Improved Computation for Levenberg Marquardt Training. IEEE Trans. on Neural Networks 21(6), 930–937 (2010)

    Article  Google Scholar 

  5. Wilamowski, B.M., Hunter, D., Malinowski, A.: Solving parity-N problems with feedforward neural networks. In: Proc. 2003 IEEE IJCNN, pp. 2546–2551. IEEE Press (2003)

    Google Scholar 

  6. Wilamowski, B.M., Yu, H., Chung, K.T.: Parity-N Problems as a Vehicle to Compare Efficiency of Neural Network Architectures. In: Industrial Electronics Handbook, 2nd edn. Intelligent Systems, vol. 5, ch. 10, pp. 10-1–10-8. CRC Press (2011)

    Google Scholar 

  7. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  8. Levenberg, K.: A method for the solution of certain problems in least squares. Quarterly of Applied Mathematics 2(2), 164–168 (1944)

    MATH  MathSciNet  Google Scholar 

  9. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Trans. on Neural Networks 5(6), 989–993 (1994)

    Article  Google Scholar 

  10. Zadeh, L.A.: Fuzzy Sets. Information and Control 8, 338–353 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  11. Mamdani, E.H.: Application of Fuzzy Algorithms for Control of Simple Dynamic Plant. IEEE Proceedings 121(12), 1585–1588 (1974)

    Google Scholar 

  12. Sugeno, M., Kang, G.T.: Structure Identification of Fuzzy Model. Fuzzy Sets and Systems 28(1), 15–33 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  13. Roger, J.S., Sun, C.T.: Functional Equivalence Between Radial Basis Function Networks and Fuzzy Inference Systems. IEEE Trans. on Neural Networks 4(1), 156–159 (1993)

    Article  Google Scholar 

  14. Jin, Y., Sendhoff, B.: Extracting Interpretable Fuzzy Rules from RBF Networks. Neural Process. Lett. 17(2), 149–164 (2003)

    Article  MATH  Google Scholar 

  15. Li, W., Hori, Y.: An Algorithm for Extracting Fuzzy Rules Based on RBF Neural Network. IEEE Trans. on Industrial Electronics 53(4) (August 2006)

    Google Scholar 

  16. Moody, J., Darken, C.J.: Learning with Localized Receptive Fields. In: Touretzky, D., Hinton, G., Sejnowski, T. (eds.) Proc. 1988 Connectionist Models Summer School, Carnegie Mellon University. Morgan Kaufmann Publishers (1988)

    Google Scholar 

  17. Hecht-Nielsen, R.: Counterpropagation networks. Applied Optics 26(23), 4979–4984 (1987)

    Article  Google Scholar 

  18. Pao, Y.H.: Adaptive Pattern Recognition and Neural Networks. Addison-Wesley Publishing Co., Reading (1989)

    MATH  Google Scholar 

  19. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer (1995)

    Google Scholar 

  20. Sch’́olkopf, B., Burges, C.J.C., Smola, A.J. (eds.): Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge (1999)

    Google Scholar 

  21. Arel, I., Rose, D.C., Karnowski, T.P.: Deep Machine Learning – A New Frontier in Artificial Intelligence Research. IEEE Computational Intelligence Magazine 5(4), 13–18 (2010)

    Article  Google Scholar 

  22. Chen, K., Salman, A.: Learning Speaker-Specific Characteristics with a Deep Neural Architecture. IEEE Trans. on Neural Networks 22(11), 1744–1756 (2011)

    Article  Google Scholar 

  23. Binfet, J., Wilamowski, B.: Microprocessor Implementation of Fuzzy Systems and Neural Networks. In: International Joint Conference on Neural Networks (IJCNN 2001), Washington, DC, pp. 234–239 (2001)

    Google Scholar 

  24. Xie, T., Yu, H., Wilamowski, B.: Replacing fuzzy systems with neural networks. In: HSI 2010, May 13-15, pp. 189–193 (2010)

    Google Scholar 

  25. Rutkowska, D., Hayashi, Y.: Neuro-fuzzy systems approaches. International Journal of Advanced Computational Intelligence 3(3), 177–185 (1999)

    Google Scholar 

  26. Yu, H., Xie, T., Wilamowski, B.M.: Neuro-Fuzzy System. In: Industrial Electronics Handbook, 2nd edn. Intelligent Systems, vol. 5, ch. 20, pp. 20-1–20-9. CRC Press (2011)

    Google Scholar 

  27. Cover, T.M.: Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition. IEEE Trans. on Electronic Computers EC–14, 326–334 (1965)

    Article  Google Scholar 

  28. Wilamowski, B.M., Jaeger, R.C.: Implementation of RBF Networks by Feedforward Sigmoidal Neural Networks. In: Dagli, C.H., et al. (eds.) Intelligent Engineering Systems Through Artificial Neural Networks, New York, vol. 7, pp. 183–188 (1997)

    Google Scholar 

  29. Park, J., Sandberg, I.W.: Universal Approximation Using Radial-Basis-Function Networks. Neural Computation 3(2), 246–257 (1991)

    Article  Google Scholar 

  30. Hunter, D., Yu, H., Pukish, M.S., Kolbusz, J., Wilamowski, B.M.: Selection of Proper Neural Network Sizes and Architectures - Comparative Study. IEEE Trans. on Industrial Informatics 8, 228–240 (2012)

    Article  Google Scholar 

  31. Alvarez-Sanchez, J.R.: Injecting knowledge into the solution of the two-spiral problem. Neural Computing and Applications 8, 265–272 (1999)

    Article  Google Scholar 

  32. Wilamowski, B.M., Cotton, N.J., Kaynak, O., Dundar, G.: Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks. IEEE Trans. on Industrial Electronics 55(10), 3784–3790 (2008)

    Article  Google Scholar 

  33. Wilamowski, B.M., Yu, H.: Neural Network Learning without Backpropgation. IEEE Trans. on Neural Networks 21(11), 1793–1803 (2010)

    Article  Google Scholar 

  34. Demuth, H.B., Beale, M.: Neural Network Toolbox: for use with MATLAB: Mathworks Natick, MA, USA (2000)

    Google Scholar 

  35. Yu, H., Xie, T., Paszczynski, S., Wilamowski, B.M.: Advantages of Radial Basis Function Networks for Dynamic System Design. IEEE Trans. on Industrial Electronics 58(12), 5438–5450 (2011)

    Article  Google Scholar 

  36. Yu, H., Reiner, P.D., Xie, T., Bartczak, T., Wilamowski, B.M.: An Incremental Design of Radial Basis Function Networks. IEEE Trans. on Neural Networks and Learning Systems 25 (2014)

    Google Scholar 

  37. Xie, T., Yu, H., Hewlett, J., Rozycki, P., Wilamowski, B.M.: Fast and Efficient Second-Order Method for Training Radial Basis Function Networks. IEEE Trans. on Neural Networks and Learning Systems 23(4), 609–619 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Dept., Auburn University, Auburn, AL, USA

    Bogdan M. Wilamowski

  2. University of IT and Management in Rzeszow, Rzeszow, Poland

    Janusz Korniak

Authors
  1. Bogdan M. Wilamowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janusz Korniak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan M. Wilamowski .

Editor information

Editors and Affiliations

  1. Institute of Applied Mathematics, Óbuda University, Budapest, Hungary

    János Fodor

  2. Department of Applied Mathematics, John von Neumann Faculty of Informatics, Óbuda University, Budapest, Hungary

    Robert Fullér

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Wilamowski, B.M., Korniak, J. (2014). Modeling of Complex Multidimensional Nonlinear Systems Using Neural System with Deep Architectures. In: Fodor, J., Fullér, R. (eds) Advances in Soft Computing, Intelligent Robotics and Control. Topics in Intelligent Engineering and Informatics, vol 8. Springer, Cham. https://doi.org/10.1007/978-3-319-05945-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-05945-7_8

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-05944-0

  • Online ISBN: 978-3-319-05945-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation