Journal of Optimization Theory and Applications

, Volume 159, Issue 3, pp 698–720 | Cite as

A New Steepest Descent Differential Inclusion-Based Method for Solving General Nonsmooth Convex Optimization Problems

  • Alireza Hosseini
  • S. M. Hosseini


In this paper, we investigate a steepest descent neural network for solving general nonsmooth convex optimization problems. The convergence to optimal solution set is analytically proved. We apply the method to some numerical tests which confirm the effectiveness of the theoretical results and the performance of the proposed neural network.


Steepest descent neural network Differential inclusion-based methods General nonsmooth convex optimization Convergence of trajectories 



The authors would like to thank the editor and the reviewers of the paper for their instructive comments. Certainly, their meticulous reading and fruitful comments enriched the content and the structure of this paper.


  1. 1.
    Mordukhovich, B.: Variations Analysis and Generalized Differentiation, I: Basic Theory. Springer, Berlin (2006) Google Scholar
  2. 2.
    Mordukhovich, B.: Variations Analysis and Generalized Differentiation, II: Applications. Springer, Berlin (2006) Google Scholar
  3. 3.
    Soleimani-damaneh, M.: Nonsmooth optimization using Mordukhovich’s subdifferential. SIAM J. Control Optim. 48, 3403–3432 (2010) MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Shor, N.: Minimization Methods for Non-differentiable Functions. Springer, Berlin (1985) CrossRefMATHGoogle Scholar
  5. 5.
    Kelley, J.E.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8, 703–712 (1960) MathSciNetCrossRefGoogle Scholar
  6. 6.
    Goffin, J.L., Haurie, A., Vial, J.P.: Decomposition and nondifferentiable optimization with the projective algorithm. Manag. Sci. 38, 284–302 (1992) CrossRefMATHGoogle Scholar
  7. 7.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms: Part 2: Advanced Theory and Bundle Methods. Springer, Berlin (2010) Google Scholar
  8. 8.
    Solodov, M.V.: On approximations with finite precision in bundle methods for nonsmooth optimization. J. Optim. Theory Appl. 119, 151–165 (2003) MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Zhao, X., Luh, P.B.: New bundle methods for solving lagrangian relaxation dual problems. J. Optim. Theory Appl. 113, 373–397 (2002) MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Forti, M., Nistri, P., Quincampoix, M.: Generalized neural network for nonsmooth nonlinear programming problems. IEEE Trans. Circuits Syst. I 51, 1741–1754 (2004) MathSciNetCrossRefGoogle Scholar
  11. 11.
    Bian, W., Xue, X.: Subgradient-based neural networks for nonsmooth nonconvex optimization problems. IEEE Trans. Neural Netw. 20, 1024–1038 (2009) CrossRefGoogle Scholar
  12. 12.
    Li, G., Song, S., Guan, X.: Subgradient-based feedback neural networks for non-differentiable convex optimization problems. Sci. China Ser. F, Inf. Sci. 20, 421–435 (2006) CrossRefGoogle Scholar
  13. 13.
    Li, G., Song, S., Wu, C.: Generalized gradient projection neural networks for nonsmooth optimization problems. Sci. China Ser. F, Inf. Sci. 53, 990–1004 (2010) MathSciNetCrossRefGoogle Scholar
  14. 14.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69, 111–147 (1995) CrossRefMATHGoogle Scholar
  15. 15.
    Mäkelä, M.M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17(1), 1–29 (2001) CrossRefGoogle Scholar
  16. 16.
    Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics. Springer, Berlin (1985) MATHGoogle Scholar
  17. 17.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization. World Scientific, Singapore (1992) MATHGoogle Scholar
  18. 18.
    Xia, Y., Leung, H., Wang, J.: A projection neural network and its application to constrained optimization problems. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. 49, 442–457 (2002) MathSciNetGoogle Scholar
  19. 19.
    Xia, Y., Wang, J.: A general projection neural network for solving monotone variational inequalities and related optimization problems. IEEE Trans. Neural Netw. 15, 318–328 (2004) MathSciNetCrossRefGoogle Scholar
  20. 20.
    Xia, Y., Wang, J.: Neural network for solving linear programming problems with bound variables. IEEE Trans. Neural Netw. 6, 515–519 (1995) CrossRefGoogle Scholar
  21. 21.
    Xia, Y.: A new neural network for solving linear programming problems and its application. IEEE Trans. Neural Netw. 7, 525–529 (1996) CrossRefGoogle Scholar
  22. 22.
    Wang, J., Hu, Q., Jiang, D.: A Lagrangian network for kinematic control of redundant robot manipulators. IEEE Trans. Neural Netw. 10, 1123–1132 (1999) CrossRefGoogle Scholar
  23. 23.
    Aubin, J.P., Cellina, A.: Differential Inclusion: Set-Valued Maps and Viability Theory. Springer, Berlin (1984) CrossRefMATHGoogle Scholar
  24. 24.
    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957–972 (1977) MathSciNetCrossRefGoogle Scholar
  25. 25.
    Defeng, S., Womersley, R.S., Houduo, Q.: A feasible semismooth asymptotically Newton method for mixed complementarity problems. Math. Program. 94, 167–187 (2002) MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Society for Industrial and Applied Mathematics, Philadelphia (1983) MATHGoogle Scholar
  27. 27.
    Filippov, A.F.: Differential Equations with Discontinuous Right-Hand Sides: Control Systems. Kluwer, Boston (1988) CrossRefGoogle Scholar
  28. 28.
    Hosseini, A., Hosseini, S.M., Soleimani-damaneh, M.: A differential inclusion-based approach for solving nonsmooth convex optimization problems. Optimization (2011). doi: 10.1080/02331934.2011.613993 Google Scholar
  29. 29.
    Borwein, J.M., Moors, W.B.: Essentially strictly differentiable Lipschitz functions. J. Funct. Anal. 149, 305–351 (1997) MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Liu, Q., Wang, J.: A one-layer recurrent neural network for constrained nonsmooth optimization. IEEE Trans. Syst. Man Cybern., Part B 41, 1323–1333 (2011) CrossRefGoogle Scholar
  31. 31.
    Lukšan, L., Vlček, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Institute of Computer Science, Academy of Science of the Czech Republic, Technical report, 798 (2000) Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Department of MathematicsTarbiat Modares UniversityTehranIran

Personalised recommendations