One of state-of-the-art recurrent neural networks (RNNs) is dual neural network (DNN). It can solve QP in real time. The dual neural network is of simple piecewise-linear dynamics and has global (exponential) convergence to optimal solutions. In this chapter, we first introduce the QP problem formulation and its online solution based on RNN. Some related concepts and definitions are also given. Second, we present the DNN and its design method. In addition to the general design method, for nondiagonal, nonanalytical, and/or time-varying cases, a matrix-inverse neural network could be combined into such a design procedure of the DNN for online computation of its matrix-inverse related term. Third, we show the analysis results of the DNN. Finally, we present a numerical simulation and illustrative example of using the DNN to solve static QP problems.


  1. 1.
    Hopfield JJ, Tank DW (1985) Neural computation of decision in optimization problems. Biol Cybern 52:122–141 MathSciNetGoogle Scholar
  2. 2.
    Kennedy MP, Chua LO (1988) Neural networks for nonlinear programming. IEEE Trans Circuits Syst 35(5):554–562 MathSciNetCrossRefGoogle Scholar
  3. 3.
    Zhang S, Constantinides AG (1992) Lagrange programming neural networks. IEEE Trans Circuits Syst 39(7):441–452 zbMATHCrossRefGoogle Scholar
  4. 4.
    Gong D, Gen M, Yamazaki G, Xu W (1996) A modified ANN for convex programming with linear constraints. In: Proceedings of IEEE international conference on neural networks, pp 537–542 Google Scholar
  5. 5.
    Xia Y (1996) A new neural network for solving linear and quadratic programming problems. IEEE Trans Neural Netw 7(6):1544–1547 CrossRefGoogle Scholar
  6. 6.
    Tao Q, Cao J, Xue M, Qiao H (2001) A high performance neural network for solving nonlinear programming problems with hybrid constraints. Phys Lett A 288(2):88–94 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Zhang Y (2002) Analysis and design of recurrent neural networks and their applications to control and robotic systems. PhD dissertation, Chinese University of Hong Kong, Hong Kong Google Scholar
  8. 8.
    Zhang Y, Wang J, Xia Y (2003) A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits. IEEE Trans Neural Netw 14(3):658–667 CrossRefGoogle Scholar
  9. 9.
    Zhang Y, Wang J (2002) A dual neural network for convex quadratic programming subject to linear equality and inequality constraints. Phys Lett A 298:271–278 MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Zhang Y, Ge SS, Lee TH (2004) A unified quadratic programming based dynamical system approach to joint torque optimization of physically constrained redundant manipulators. IEEE Trans Syst Man Cybern, Part B, Cybern 34(5):2126–2132 CrossRefGoogle Scholar
  11. 11.
    Zhang Y, Ma S (2007) Minimum-energy redundancy resolution of robot manipulators unified by quadratic programming and its online solution. In: Proceedings of IEEE international conference on mechatronics and automation, pp 3232–3237 Google Scholar
  12. 12.
    Zhang Y, Li Z, Tan H (2006) Inequality-based manipulator-obstacle avoidance using the LVI-based primal–dual neural network. In: Proceedings of IEEE international conference on robotics and biomimetics, pp 1459–1464 Google Scholar
  13. 13.
    Zhang Y (2005) On the LVI-based primal–dual neural network for solving online linear and quadratic programming problems. In: Proceedings of American control conference, pp 1351–1356 Google Scholar
  14. 14.
    Zhang Y (2006) Inverse-free computation for infinity-norm torque minimization of robot manipulators. Mechatronics 16:177–184 CrossRefGoogle Scholar
  15. 15.
    Zhang Y (2005) Revisit the analog computer and gradient-based neural system for matrix inversion. In: Proceedings of IEEE international symposium on intelligent control, pp 1411–1416 Google Scholar
  16. 16.
    Zhang Y, Heng PA, Fu A (1999) Estimate of exponential convergence rate and exponential stability for neural networks. IEEE Trans Neural Netw 10(6):1487–1493 CrossRefGoogle Scholar
  17. 17.
    Zhang Y, Wang J, Xu Y (2002) A dual neural network for bi-criteria kinematic control of redundant manipulators. IEEE Trans Robot Autom 18(6):923–931 CrossRefGoogle Scholar
  18. 18.
    Zhang Y, Wang J (2002) A dual neural network for constrained joint torque optimization of kinematically redundant manipulators. IEEE Trans Syst Man Cybern, Part B, Cybern 32(5):654–662 CrossRefGoogle Scholar
  19. 19.
    Bazaraa MS, Sherali HD, Shetty CM (1993) Nonlinear programming—theory and algorithms. Wiley, New York zbMATHGoogle Scholar
  20. 20.
    Wang J, Xia Y (1999) A dual neural network solving quadratic programming problems. In: Proceedings of international joint conference on neural networks, pp 588–593 Google Scholar
  21. 21.
    Mangasarian OL (1979) Solution of symmetric linear complementarity problems by iterative methods. J Optim Theory Appl 22(2):465–485 MathSciNetGoogle Scholar
  22. 22.
    Bertsekas DP (1989) Parallel and distributed computation: numerical methods. Prentice-Hall, New Jersey zbMATHGoogle Scholar
  23. 23.
    Li W, Swetits J (1997) A new algorithm for solving strictly convex quadratic programs. SIAM J Optim 7(3):595–619 MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Zhang Y, Ge SS (2005) Design and analysis of a general recurrent neural network model for time-varying matrix inversion. IEEE Trans Neural Netw 16(6):1477–1490 CrossRefGoogle Scholar
  25. 25.
    Mead C (1989) Analog VLSI and neural systems. Addison-Wesley, Reading zbMATHCrossRefGoogle Scholar
  26. 26.
    Zhang Y (2007) Recurrent neural networks: design, analysis, applications to control and robotic systems. Lambert Academic, Germany Google Scholar
  27. 27.
    Zhang Y, Leithead WE, Leith DJ (2005) Time-series Gaussian process regression based on Toeplitz computation of O(N 2) operations and O(N)-level storage. In: Proceedings of the 44th IEEE conference on decision and control, pp 3711–3716 CrossRefGoogle Scholar
  28. 28.
    Zhang Y, Jiang D, Wang J (2002) A recurrent neural network for solving Sylvester equation with time-varying coefficients. IEEE Trans Neural Netw 13(5):1053–1063 CrossRefGoogle Scholar
  29. 29.
    Zhang Y, Wang J (2002) Global exponential stability of recurrent neural networks for synthesizing linear feedback control systems via pole assignment. IEEE Trans Neural Netw 13(3):633–644 CrossRefGoogle Scholar
  30. 30.
    Zhang Y (2006) A set of nonlinear equations and inequalities arising in robotics and its online solution via a primal neural network. Neurocomputing 70(1–3):513–524 CrossRefGoogle Scholar
  31. 31.
    Pang JS (1987) A posterior error bounds for the linearly-constrained variational inequality problem. Math Oper Res 12(3):474–484 MathSciNetCrossRefGoogle Scholar
  32. 32.
    Pang JS, Yao YC (1995) On a generalization of a normal map and equation. SIAM J Control Optim 33:168–184 MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    The MathWorks, Inc (2004) MATLAB 7. Natick, MA Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Yunong Zhang
    • 1
  • Zhijun Zhang
    • 1
  1. 1.Sun Yat-sen UniversityGuangzhouPeople’s Republic of China

Personalised recommendations