Skip to main content
SpringerLink
Log in
Book cover

Numerical Analysis and Optimization

NAO 2017: Numerical Analysis and Optimization pp 155–171Cite as

A Time-Delay Neural Network Model for Unconstrained Nonconvex Optimization

  • Conference paper
  • First Online:

  • 971 Accesses

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 235))

Abstract

In this paper, we first present some theoretical properties which include the convergence and bounds of both the trajectory and energy function for general neural network models for unconstrained nonconvex optimization problems. Based on this analysis, a novel time-delay neural network model is proposed for unconstrained nonconvex optimization. The simulation results of the new neural network on two examples indicate that the new neural network is quite efficient and outperforms the gradient neural network.

The first author was supported in part by grants from Hong Kong Baptist University (FRG) and General Research Fund (GRF) of Hong Kong. The second author was supported in part by the Chinese NSFC Grant (Nos. 11631013, 11331012 and 71331001) and the Key Project of Chinese National Programs for Fundamental Research and Development Grant (No. 2015CB856002).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)

    Article  MathSciNet  Google Scholar 

  2. Bouzerdoum, A., Pattison, T.R.: Neural network for quadratic optimization with bound constraints. IEEE Trans. Neural Netw. 4, 293–304 (1993)

    Article  Google Scholar 

  3. Chen, Y.H., Fang, S.C.: Solving convex programming problem with equality constraints by neural networks. Comput. Math. Appl. 36, 41–68 (1998)

    Article  MathSciNet  Google Scholar 

  4. Cichocki, A., Unbehauen, R.: Neural Networks for Optimization and Signal Processing. Wiley, Stuttgart (1993)

    Google Scholar 

  5. Cichocki, A., Unbehauen, R., Weinzierl, K., Holzel, R.: A new neural network for solving linear programming problems. Eur. J. Oper. Res. 93, 244–256 (1996)

    Article  Google Scholar 

  6. Chua, L.O., Lin, G.-N.: Nonlinear programming without computation. IEEE Trans. Circ. Syst. 31, 182–188 (1984)

    Article  MathSciNet  Google Scholar 

  7. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational ability. Proc. Natl. Acad. Sci. 79, 2554–2558 (1982)

    Article  MathSciNet  Google Scholar 

  8. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. 81, 3088–3092 (1984)

    Article  Google Scholar 

  9. Hopfield, J.J., Tank, D.W.: Neural computation of decisions in optimization problems. Biol. Cybern. 52, 141–152 (1985)

    MATH  Google Scholar 

  10. Hou, Z.-G., Wu, C.-P., Bao, P.: A neural network for hierarchical optimization of nonlinear large-scale systems. Int. J. Syst. Sci. 29(2), 159–166 (1998)

    Article  Google Scholar 

  11. Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4, 373–395 (1984)

    Article  MathSciNet  Google Scholar 

  12. Kennedy, M.P., Chua, L.O.: Neural networks for nonlinear programming. IEEE Trans. Circ. Syst. 35, 554–562 (1988)

    Article  MathSciNet  Google Scholar 

  13. Liao, L.-Z., Qi, H.D., Qi, L.Q.: Neurodynamic optimization. J. Glob. Optim. 28, 175–195 (2004)

    Article  Google Scholar 

  14. Lillo, W.E., Loh, M.-H., Hui, S., Żak, S.H.: On solving constrained optimization problems with neural networks: a penalty method approach. IEEE Trans. Neural Netw. 4, 931–940 (1993)

    Article  Google Scholar 

  15. Maa, C.Y., Shanblatt, M.A.: Linear and quadratic programming neural network analysis. IEEE Trans. Neural Netw. 3(4), 580–594 (1992)

    Article  Google Scholar 

  16. Maa, C.Y., Shanblatt, M.A.: A two-phase optimization neural network. IEEE Trans. Neural Netw. 3, 1003–1009 (1992)

    Article  Google Scholar 

  17. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    Article  MathSciNet  Google Scholar 

  18. Qian X., Liao L.-Z., Sun J., Zhu H.: The convergent generalized central paths for linearly constrained convex programming. SIAM J. Optim. (2018) (to appear)

    Google Scholar 

  19. Rodríguez-Vázquez, A., Domínguez-Castro, R., Rueda, A., Huertas, J.L., Sánchez-Sinencio, E.: Nonlinear switched-capacitor ‘neural’ networks for optimization problems. IEEE Trans. Circ. Syst. 37, 384–398 (1990)

    Article  MathSciNet  Google Scholar 

  20. Tank, D.W., Hopfield, J.J.: Simple ‘neural’ optimization network: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans. Circ. Syst. 33, 533–541 (1986)

    Article  Google Scholar 

  21. Wang, J.: Recurrent neural networks for computing pseudoinverses of rank-deficient matrices. SIAM J. Sci. Comput. 18, 1479–1493 (1997)

    Article  MathSciNet  Google Scholar 

  22. Wang, J., Xia, Y.: Analysis and design of primal-dual assignment networks. IEEE Trans. Neural Netw. 9, 183–194 (1998)

    Article  Google Scholar 

  23. Wu, X.-Y., Xia, Y., Li, J., Chen, W.-K.: A high-performance neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7, 643–651 (1996)

    Article  Google Scholar 

  24. Xia, Y.: A new neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7, 1544–1547 (1996)

    Article  Google Scholar 

  25. Zak, S.H., Upatising, V., Lillo, W.E., Hui, S.: A dynamical systems approach to solving linear programming problems. In: Elworthy, K.D., Everitt, W.N., Lee, E.B. (eds.) Differential Equations, Dynamical Systems, and Control Science, pp. 913–946. Marcel Dekker, New York (1994)

    Google Scholar 

  26. Zak, S.H., Upatising, V., Hui, S.: Solving linear programming problems with neural networks: a comparative study. IEEE Trans. Neural Netw. 6, 94–104 (1995)

    Article  Google Scholar 

  27. Zhang, H., Liu, Z., Huang, G.B., Wang, Z.: Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay. Trans. Neural Netw. 21(1), 91–106 (2010)

    Article  Google Scholar 

  28. Zhang, S., Constantinides, A.G.: Lagrange programming neural networks. IEEE Trans. Circ. Syst. 2 Analog Digital Sig. Process. 39, 441–452 (1992)

    Article  Google Scholar 

  29. Zhang, S., Zhu, X., Zou, L.-H.: Second-order neural nets for constrained optimization. IEEE Trans. Neural Netw. 3, 1021–1024 (1992)

    Article  Google Scholar 

  30. Zuo, Z., Yang, C., Wang, Y.: A new method for stability analysis of recurrent neural networks with interval time-varying delay. IEEE Trans. Neural Netw. 21(2), 339–344 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Kowloon, Hong Kong

    Li-Zhi Liao

  2. LSEC, Academy of Mathematics and Systems Sciences Chinese Academy of Sciences, Box 2719, Beijing, 100190, China

    Yu-Hong Dai

  3. School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

    Yu-Hong Dai

Authors
  1. Li-Zhi Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu-Hong Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu-Hong Dai .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Statistics, Sultan Qaboos University, Muscat, Oman

    Mehiddin Al-Baali

  2. Dipartimento di Ingegneria Informatica, Modellistica, Elettronica e Sistemistica, Calabria University, Arcavacata, Italy

    Lucio Grandinetti

  3. Department Mathematics and Statistics, Sultan Qaboos University, Muscat, Oman

    Anton Purnama

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liao, LZ., Dai, YH. (2018). A Time-Delay Neural Network Model for Unconstrained Nonconvex Optimization. In: Al-Baali, M., Grandinetti, L., Purnama, A. (eds) Numerical Analysis and Optimization. NAO 2017. Springer Proceedings in Mathematics & Statistics, vol 235. Springer, Cham. https://doi.org/10.1007/978-3-319-90026-1_7

Download citation

Publish with us

Policies and ethics

Search

Navigation