# Comparative study of neural networks for dynamic nonlinear systems identification

## Abstract

In this paper, a comparative study is performed to test the approximation ability of different neural network structures. It involves three neural networks multilayer feedforward neural network (MLFFNN), diagonal recurrent neural network (DRNN), and nonlinear autoregressive with exogenous inputs (NARX) neural network. Their robustness is also tested and compared when the system is subjected to parameter variations and disturbance signals. Further, dynamic back-propagation algorithm is used to update the parameters associated with these neural networks. Four dynamical systems of different complexities including motor-driven robotic link are considered on which the comparative study is performed. The simulation results show the superior performance of DRNN identification model over NARX and MLFFNN identification models.

## Keywords

Diagonal recurrent neural network NARX model Identification Multilayer feedforward neural network Robustness## Notes

### Compliance with Ethical Standards

### Funding

This study is not funded by any agency.

### Conflict of Interest

Rajesh Kumar declares that he has no conflict of interest. Smriti Srivastava declares that she has no conflict of interest. J.R.P Gupta declares that he has no conflict of interest. Amit Mohindru declares that he has no conflict of interest.

### Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

## References

- Abdollahi F, Talebi HA, Patel RV (2006) Stable identification of nonlinear systems using neural networks: theory and experiments. IEEE/ASME Trans Mechatron 11(4):488–495Google Scholar
- Al Seyab RK, Cao Y (2008) Nonlinear system identification for predictive control using continuous time recurrent neural networks and automatic differentiation. J Process Control 18(6):568–581Google Scholar
- Arqub OA (2017) Adaptation of reproducing kernel algorithm for solving fuzzy fredholm-volterra integrodifferential equations. Neural Comput Appl 28(7):1591–1610Google Scholar
- Arqub OA, Mohammed AL-S, Momani S, Hayat T (2016) Numerical solutions of fuzzy differential equations using reproducing kernel hilbert space method. Soft Comput 20(8):3283–3302zbMATHGoogle Scholar
- Arqub OA, Al-Smadi M, Momani S, Hayat T (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206zbMATHGoogle Scholar
- Atencia M, Joya G, Sandoval F (2013) Identification of noisy dynamical systems with parameter estimation based on hopfield neural networks. Neurocomputing 121:14–24Google Scholar
- Banakar A, Azeem MF (2012) Local recurrent sigmoidal-wavelet neurons in feed-forward neural network for forecasting of dynamic systems: Theory. Appl Soft Comput 12(3):1187–1200Google Scholar
- Baruch IS, Flores JM, Garrido R (2001) A fuzzy neural recurrent multi-model for systems identification and control. In: Control conference (ECC), 2001 European. IEEE, pp 3540–3545Google Scholar
- Chen S, Billings SA, Grant PM (1992) Recursive hybrid algorithm for non-linear system identification using radial basis function networks. Int J Control 55(5):1051–1070zbMATHGoogle Scholar
- Chow TWS, Fang Y (1998) A recurrent neural-network-based real-time learning control strategy applying to nonlinear systems with unknown dynamics. IEEE Trans Industr Electron 45(1):151–161Google Scholar
- de Jesús Rubio J, Yu W (2005) Dead-zone kalman filter algorithm for recurrent neural networks. In: Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC’05. 44th IEEE Conference on. IEEE, pp 2562–2567Google Scholar
- Efe MO, Kaynak O (1999) A comparative study of neural network structures in identification of nonlinear systems. Mechatronics 9(3):287–300Google Scholar
- Fahmi A, Abdullah S, Amin F, Ali A (2018) Weighted average rating (war) method for solving group decision making problem using triangular cubic fuzzy hybrid aggregation (tcfha). Punjab Univ J Math 50(1):23–34MathSciNetGoogle Scholar
- Fahmi A, Abdullah S, Amin F, Ahmed R, Ali A. Triangular cubic linguistic hesitant fuzzy aggregation operators and their application in group decision making. J Intell Fuzzy Syst, pp 1–15 (Preprint)Google Scholar
- Fahmi A, Abdullah S, Amin F, Ali A (2017) Precursor selection for sol–gel synthesis of titanium carbide nanopowders by a new cubic fuzzy multi-attribute group decision-making model. J Intell SystGoogle Scholar
- Fahmi A, Abdullah S, Amin F, Siddiqui N (2017) Aggregation operators on triangular cubic fuzzy numbers and its application to multi-criteria decision making problems. J Intell Fuzzy Syst, pp 1–15 (Preprint)Google Scholar
- Gabrijel I, Dobnikar A (2003) On-line identification and reconstruction of finite automata with generalized recurrent neural networks. Neural Netw 16(1):101–120zbMATHGoogle Scholar
- Haykin S, Network N (2004) A comprehensive foundation. Neural Netw 2:2004Google Scholar
- Hecht-Nielsen R et al (1988) Theory of the backpropagation neural network. Neural Netw 1(Supplement–1):445–448Google Scholar
- Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366zbMATHGoogle Scholar
- Jin L, Gupta MM (1999) Stable dynamic backpropagation learning in recurrent neural networks. IEEE Trans Neural Netw 10(6):1321–1334Google Scholar
- Ku C-C, Lee KY (1992) Diagonal recurrent neural network based control using adaptive learning rates. In: Proceedings of the 31st IEEE Conference on Decision and Control. IEEE, pp 3485–3490Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2017) Modeling and adaptive control of nonlinear dynamical systems using radial basis function network. Soft Comput 21(15):4447–4463Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2017) Diagonal recurrent neural network based adaptive control of nonlinear dynamical systems using lyapunov stability criterion. ISA Trans 67:407–427Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2016) Artificial neural network based pid controller for online control of dynamical systems. In: IEEE international conference on, in power electronics, intelligent control and energy systems (ICPEICES). IEEE, pp 1–6Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2016) Modeling and control of one-link robotic manipulator using neural network based pid controller. In: Advances in computing, communications and informatics (ICACCI), 2016 International conference on. IEEE, pp 243–249Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2016) Online modeling and adaptive control of robotic manipulators using gaussian radial basis function networks. Neural Comput Appl, pp 1–17Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2017) A soft computing approach for modeling of nonlinear dynamical systems. In Proceedings of the 5th international conference on frontiers in intelligent computing: theory and applications. Springer, pp 407–415Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2017) Lyapunov stability-based control and identification of nonlinear dynamical systems using adaptive dynamic programming. Soft Comput, pp 1–16Google Scholar
- Kumar R, Srivastava S, Gupta JRP (2018) Comparative study of neural networks for control of nonlinear dynamical systems with lyapunov stability-based adaptive learning rates. Arabian J Sci Eng, pp 1–23Google Scholar
- Lee C-H, Teng C-C (2000) Identification and control of dynamic systems using recurrent fuzzy neural networks. IEEE Trans Fuzzy Syst 8(4):349–366Google Scholar
- Lilly JH (2011) Fuzzy control and identification. Wiley, HobokenzbMATHGoogle Scholar
- Lin C-M, Hsu C-F (2005) Recurrent-neural-network-based adaptive-backstepping control for induction servomotors. IEEE Trans Industr Electron 52(6):1677–1684Google Scholar
- Liu Y-C, Liu S-Y, Wang N (2016) Fully-tuned fuzzy neural network based robust adaptive tracking control of unmanned underwater vehicle with thruster dynamics. Neurocomputing 196:1–13Google Scholar
- Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Networks 1(1):4–27Google Scholar
- Narendra KS, Parthasarathy K (1991) Gradient methods for the optimization of dynamical systems containing neural networks. IEEE Trans Neural Netw 2(2):252–262Google Scholar
- Pan Y, Wang J (2012) Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks. IEEE Trans Industr Electron 59(8):3089–3101MathSciNetGoogle Scholar
- Paul RP (1981) Robot manipulators: mathematics, programming, and control: the computer control of robot manipulators. Richard Paul, Los AngelesGoogle Scholar
- Polycarpou MM, Ioannou PA (1992) Modelling, identification and stable adaptive control of continuous-time nonlinear dynamical systems using neural networks. In: American Control Conference. IEEE, pp 36–40Google Scholar
- Polycarpou MM (1996) Stable adaptive neural control scheme for nonlinear systems. IEEE Trans Autom Control 41(3):447–451MathSciNetzbMATHGoogle Scholar
- Rajesh MV, Archana R, Unnikrishnan A, Gopikakaumari R (2009) Particle filter based neural network modeling of nonlinear systems for state space estimation. In: Control and Decision Conference, 2009. CCDC’09. IEEE, Chinese, pp 1477–1482Google Scholar
- Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: The rprop algorithm. In: Neural Networks, IEEE International Conference on. IEEE, pp 586–591Google Scholar
- Roudbari A, Saghafi F (2014) Intelligent modeling and identification of aircraft nonlinear flight. Chin J Aeronaut 27(4):759–771Google Scholar
- Sadegh N (1993) A perceptron network for functional identification and control of nonlinear systems. IEEE Trans Neural Netw 4(6):982–988Google Scholar
- Savran A (2007) Multifeedback-layer neural network. IEEE Trans Neural Netw 18(2):373–384Google Scholar
- Sciavicco L, Siciliano B (2012) Modelling and control of robot manipulators. Springer, BerlinzbMATHGoogle Scholar
- Shin YC (1994) Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems. IEEE Trans Neural Netw 5(4):594–603Google Scholar
- Simon H (1994) Neural networks: a comprehensive foundation. Prentice Hall PTR, Upper Saddle RiverzbMATHGoogle Scholar
- Singh M, Srivastava S, Gupta JRP, Handmandlu M (2007) Identification and control of a nonlinear system using neural networks by extracting the system dynamics. IETE J Res 53(1):43–50Google Scholar
- Singh M, Srivastava S, Hanmandlu M, Gupta JRP (2009) Type-2 fuzzy wavelet networks (t2fwn) for system identification using fuzzy differential and lyapunov stability algorithm. Appl Soft Comput 9(3):977–989Google Scholar
- Srivastava S, Singh M, Hanmandlu M, Jha AN (2005) New fuzzy wavelet neural networks for system identification and control. Appl Soft Comput 6(1):1–17Google Scholar
- Srivastava S, Singh M, Madasu VK, Hanmandlu M (2008) Choquet fuzzy integral based modeling of nonlinear system. Appl Soft Comput 8(2):839–848Google Scholar
- Teeter J, Chow M-Y (1998) Application of functional link neural network to hvac thermal dynamic system identification. IEEE Trans Industr Electron 45(1):170–176Google Scholar
- Wlas M, Krzeminski Z, Toliyat HA (2008) Neural-network-based parameter estimations of induction motors. IEEE Trans Industr Electron 55(4):1783–1794Google Scholar
- Yang G-B, Donath M (1988) Dynamic model of a one-link robot manipulator with both structural and joint flexibility. In: Robotics and Automation, 1988. Proceedings., 1988 IEEE International Conference on. IEEE, pp 476–481Google Scholar
- Zhu Y-Q, Xie W-F, Yao J (2006) Nonlinear system identification using genetic algorithm based recurrent neural networks. In Electrical and Computer Engineering, 2006. CCECE’06. Canadian Conference on. IEEE, pp 571–575Google Scholar