Advertisement

Neural Processing Letters

, Volume 37, Issue 2, pp 163–187 | Cite as

Quantized Neural Modeling: Hybrid Quantized Architecture in Elman Networks

  • Penghua Li
  • Yi Chai
  • Qingyu Xiong
Article

Abstract

This paper presents a novel neural network model with hybrid quantized architecture to improve the performance of the conventional Elman networks. The quantum gate technique is introduced for solving the pattern mismatch between the inputs stream and one-time-delay state feedback. A quantized back-propagation training algorithm with an adaptive dead zone scheme is developed for providing an optimal or suboptimal tradeoff between the convergence speed and the generalization performance. Furthermore, the effectiveness of the new real time learning algorithm is demonstrated by proving the quantum gate parameter convergence based on Lyapunov method. The numerical experiments are carried out to demonstrate the accuracy of the theoretical results.

Keywords

Elman networks Hybrid quantized structure Quantum Elman back-propagation training Quantized parameter convergence Generalization performance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Elman JL (1990) Finding structure in time. Cogn Sci 14: 179–211CrossRefGoogle Scholar
  2. 2.
    Jordan M (1986) Serial order: a parallel distributed processing approach. ICS Report 8604, Institute of Cognitive Science, University of California San Diego, San DiegoGoogle Scholar
  3. 3.
    Lee SW, Song HH (1997) New recurrent neural network architecture for visual pattern recognition. IEEE Trans Neural Netw 8(2): 331–340MathSciNetCrossRefGoogle Scholar
  4. 4.
    Goh WY, Lim CP, Peh KK (2003) Predicting drug dissolution profiles with an ensemble of boosted neural networks: a time series approach. IEEE Trans Neural Netw 14(2): 459–463CrossRefGoogle Scholar
  5. 5.
    Toqeer RS, Bayindir NS (2003) Speed estimation of an induction motor using Elman neural network. Neurocomputing 55(3–4): 727–730CrossRefGoogle Scholar
  6. 6.
    Valle-Lisboa JC, Reali F, Anastasa H (2005) Elman topology with sigma–pi units: an application to the modeling of verbal hallucinations in schizophrenia. Neural Netw 18(7): 863–877CrossRefGoogle Scholar
  7. 7.
    Liou C, Huang J, Yang W (2008) Modeling word perception using the Elman network. Neurocomputing 71(16–18): 3150–3157CrossRefGoogle Scholar
  8. 8.
    Ardalani-Farsa M, Zolfaghari S (2010) Chaotic time series prediction with residual analysis method using hybrid Elman–NARX neural networks. Neurocomputing 73(13–15): 2540–2553CrossRefGoogle Scholar
  9. 9.
    Song Q, Wu Y, Soh YC (2008) Robust adaptive gradient-descent training algorithm for recurrent neural networks in discrete time domain. IEEE Trans Neural Netw 19(11): 1841–1853CrossRefGoogle Scholar
  10. 10.
    Kremer SC (1995) On the computational power of Elman-style recurrent networks. IEEE Trans Neural Netw 6(4): 1000–1004CrossRefGoogle Scholar
  11. 11.
    Wang D, Liu X, Ahalt SC (1996) On temporal generalization of simple recurrent networks. Neural Netw 9(7): 1099–1118CrossRefGoogle Scholar
  12. 12.
    Li X, Chen G, Chen Z, Yuan Z (2002) Chaotifying linear Elman networks. IEEE Trans Neural Netw 13(5): 1193–1199CrossRefGoogle Scholar
  13. 13.
    Deutsch D, Jozsa R (1992) Rapid solution of problems by quantum computation. Proc Math Phys Sci A 439: 553–558MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Ventura D, Martinez T (2000) Quantum associative memory. Inf Sci 124(1): 273–296MathSciNetCrossRefGoogle Scholar
  15. 15.
    Menneer T, Narayanan A (1995) Quantum-inspired neural networks. Technical Report R329, University of Exeter, ExeterGoogle Scholar
  16. 16.
    Kouda N, Matsui N, Nishimura H (2001) Image compression by layered quantum neural networks. Neural Process Lett 16(1): 67–80CrossRefGoogle Scholar
  17. 17.
    Kouda N, Matsui N, Nishimura H, Peper F (2005) An examination of qubit neural network in controlling an inverted pendulum. Neural Process Lett 22: 277–290CrossRefGoogle Scholar
  18. 18.
    Gopathy P, Nicolaos BK (1997) Quantum neural networks (QNN’s): inherently fuzzy feedforward neural networks. IEEE Trans Neural Netw 8(3): 679–693CrossRefGoogle Scholar
  19. 19.
    Matsui N, Takai M, Nishimura H (1988) A network model based on qubit-like neuron corresponding to quantum circuit. IEICE J81-A(12):1687–1692Google Scholar
  20. 20.
    Luitel B, Venayagamoorthy GK (2010) Quantum inspired PSO for the optimization of simultaneous recurrent neural networks as MIMO learning systems. Neural Netw 23: 583–586CrossRefGoogle Scholar
  21. 21.
    Li PH, Chai Y, Xiong QY (2011) Quantized neuronal modeling: quantum gate structure in Elman networks. In: Proceedings of fourth international workshop on advanced computational intelligence. IEEE Press, Wu Han, pp 315–320Google Scholar
  22. 22.
    Song Q (2010) On the weight convergence of Elman networks. IEEE Trans Neural Netw 21(3): 463–480CrossRefGoogle Scholar
  23. 23.
    Vidyasagar M (1980) Nonlinear systems analysis. Prentice-Hall, Englewood CliffsGoogle Scholar
  24. 24.
    Hammer B, Tino P (2003) Recurrent neural networks with small weights implement definite memory machines. Neurocomputing 15: 1897–1929MATHGoogle Scholar
  25. 25.
    Song Q, Spall J, Soh YC, Ni J (2008) Robust neural network tracking controller using simultaneous perturbation stochastic approximation. IEEE Trans Neural Netw 19(5): 817–835CrossRefGoogle Scholar
  26. 26.
    Phat VN, Botmart T, Niamsupb P (2008) Switching design for exponential stability of a class of nonlinear hybrid time-delay systems. Nonlinear Anal Hybrid Syst 3: 1–10CrossRefGoogle Scholar
  27. 27.
    Phat VN, Niamsupb P (2010) A novel exponential stability condition of hybrid neural networks with time-varying delay. Vietnam J Math 38: 341–351MathSciNetMATHGoogle Scholar
  28. 28.
    Zames G (1966) On the input–output stability of time-varying nonlinear feedback systems, part I. IEEE Trans Autom Control 11(2): 228–238CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2012

Authors and Affiliations

  1. 1.State Key Laboratory of Power Transmission Equipment and System Security and New Technology, College of AutomationChongqing UniversityChongqingPeople’s Republic of China
  2. 2.School of Software EngineeringChongqing UniversityChongqingPeople’s Republic of China

Personalised recommendations