Generalized Gradient Methods of Nondifferentiable Optimization Employing Space Dilatation Operations

  • N. Z. Shor

Abstract

A broad spectrum of complex problems of mathematical programming can be rather easily reduced to problems of minimization of nondifferentiable functions without constraints or with simple constraints. Thus, when decomposition schemes are used to solve structured optimization problems of large dimension or with a large number of constraints, the coordination problems with respect to linking variables (or dual estimates of linking constraints) as a rule prove to be problems of nondifferentiable optimization. The use of exact nonsmooth penalty functions in problems of nonlinear programming, maximum functions to estimate discrepancies in constraints, piecewise smooth approximation of technical-economic characteristics in practical problems of optimal planning and design, minimax compromise functions in problems of multi-criterion optimization, all of these generate problems of nondifferentiable optimization. Therefore, the efficiency of procedures for the solution of various complex problems of mathematical programming greatly depends on the efficiency of the algorithms employed to minimize nondifferentiable functions.

Keywords

Manifold Transportation Assure Hull Vasil 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    A. A. Bakaev, V. S. Mikhalevich, S. V. Branovickaja, N. Z. Shor. Methodology and Attempted Solution of Large Network Transportation Problems by Computer. - In the Book: Mathematical Methods for Production Problems, M., 1963, pp. 247–257.Google Scholar
  2. [2]
    L. V. Beljaeva, N. T. Zhurbenko, N. Z. Shor. Concerning Methods of Solution of One Class of Dynamic Distribution Problems. - Economics and Mathematical Methods, 1978, Vol. 14, Issue 1, pp. 137–146.Google Scholar
  3. [3]
    V. A. Bulavskij. Iterative Method of Solution of General Problem of Linear Programming. - In Coll.: Numerical Methods of Optimal Planning, Issue 1, Econ.- Math. Ser., Siberian Branch of Ac. Sci. U.S.S.R., 1962, pp. 35–64.Google Scholar
  4. [4]
    V. S. Mikhalevich, N. Z. Shor, L. A. Galustova, et al. Computational Methods of Choice of Optimal Design Decisions. - Kiev, Naukova Dumka, 1977, 178 p.Google Scholar
  5. [5]
    V. I. Gershovich. About a Cutting Method with Linear Space Transformations. In the Book: Theory of Optimal Solutions, Kiev, Inst, of Cybernetics. Ac. Sci. Ukr.S.S.R., 1979, pp. 15–23.Google Scholar
  6. [6]
    V. I. Gershovich. One Way of Representing Space Transformation Operators in High-Speed Versions of Generalized Gradient Methods. In the Book: Numerical Methods of Nonlinear Programming. Summaries of Papers at the III. All-Union Seminar. - Kharkov, 1979, pp. 64–66.Google Scholar
  7. [7]
    V. I. Gershovich. Efficient Algorithm for Solving Problems of Nonsmooth Optimization that Combines Cutting Methods and Linear Transformations in Variable Space. - In the Book: Systems of Software Support for Solution of Optimal Planning Problems. Summaries of Papers at the IV. Ail-Union Symposium.-M., 1980, pp. 50–51.Google Scholar
  8. [8]
    V. I. Gershovich. One Optimization Method Using Linear Space Transformations. - In the Book: Theory of Optimal Solutions. Kiev, Inst, of Cybernetics, Ac. Sci. Ukr. S.S.R., 1980, pp. 38–45.Google Scholar
  9. [9]
    V. I. Gershovich. About an Ellipsoid Algorithm. In the Book: Some Algorithms of Nonsmooth Optimization and Discrete Programming. Preprint 81-6 Inst, of Cybernetics, Ac. Sci. Ukr.S.S.R., 1981, pp. 8–13.Google Scholar
  10. [10]
    V. I. Gershovich. Quadratic Smoothing in Iterative Decomposition Algorithms of Solution of Large-Scale Linear-Programming Problems. In the Book: II. Republican Symposium on Methods of Solution of Nonlinear Equations and Optimization Problems. Tallin, 1981, pp. 188–190.Google Scholar
  11. [11]
    A. M. Gupal. Stochastic Methods of Solution of Nonsmooth Extremum Problems. - Kiev, Naukova Dumka, 1979, 149 p.Google Scholar
  12. [12]
    V. V. Dem’janov, V. N. Malozemov. Introduction to Minimax - M., Nauka, 1972, 368 p.Google Scholar
  13. [13]
    V. F. Dem’janov, L. V. Vasil’ev. Non-Differentiable Optimization. - M., Nauka, 1981, 384 p.Google Scholar
  14. [14]
    I. I. Eremin. Iterative Method for Tchebycheff Approximations of Inconsistent Systems of Linear Inequalities. - Dokl. AN S.S.S.R., 1962, Vol. 143, No. 6, pp. 1254–1256.MathSciNetGoogle Scholar
  15. [15]
    I. I. Eremin. About “Penalty ” Method in Convex Programming. - Kybernetika, 1967, No. 4, pp. 63–67.Google Scholar
  16. [16]
    I. I. Eremin. Methods of Fejer Approximations in Convex Programming. Mathem. Notes, 1968, Vol. 3, No. 2, pp. 217–234.MathSciNetGoogle Scholar
  17. [17]
    Ju. M. Ermol’jev. Stochastic Programming Methods, M., Nauka, 1976, 240 p.Google Scholar
  18. [18]
    Ju. M. Ermol’jev. Methods of Solution of Nonlinear Extremum Problems. - Kiber- netika, 1966, No. 4, pp. 1–17.Google Scholar
  19. [19]
    A. Ju. Levin. On an Algorithm for Minimization of Convex Functions. - Dokl. AN S.S.S.R., 1965, Vol. 160, No. 6, pp. 1244–1247.Google Scholar
  20. [20]
    A. S. Nemirovskij, D. B. Judin. Optimization Methods Adaptive to “Considerable ” Dimension of a Problem. - Avtomatika i Telemekhanika, 1977, No. 4, pp. 75–87.Google Scholar
  21. [21]
    N. S. Nemirovskij, D. B. Judin. Complexity of Problems and Efficiency of Optimization Methods. - M., Nauka, 1979, 383 p.Google Scholar
  22. [22]
    E. A. Nurminskij. Numerical Methods of Solution of Deterministic and Stochastic Minimax Problems. - Kiev. Naukova Dumka, 1979, 159 p.Google Scholar
  23. [23]
    E. A. Nurminskij. About Continuity of e-Subgradient Mappings. - Kibernetika, 1977, No. 5, pp. 148–149.Google Scholar
  24. [24]
    B. T. Poljak. A General Method of Solution of Extremum Problems. - Dokl. AN S.S.S.R., 1967, Vol. 174, No. 1, pp. 33–36.MathSciNetGoogle Scholar
  25. [25]
    B. T. Poljak. Minimization of Unsmooth Functionals. - J. Vychisl. Matem. i Mat. Fiziki, 1969, Vol. 9, No. 3, pp. 509–521.MathSciNetMATHGoogle Scholar
  26. [26]
    B. N. Pshenichnyj, Ju. M. Danilin. Numerical Methods for Extremum Problems.–M., Nauka, 1975, 319 p.Google Scholar
  27. [27]
    S. V. Rzhevskij. About a Unified Approach to Solution of Unconditional Minimization Problems. - J. Vychisl. Matem. i Matem. Fiziki, 1969, Vol. 20, No. 4, pp. 857–863.Google Scholar
  28. [28]
    S. V. Rzhevskij. e-Subgradient Method of Solution of Convex Programming Problem. - J. Vychisl. Matem. i Matem. Fiziki, Vol. 21, No. 5, 1981, pp. 1126–1132.MATHGoogle Scholar
  29. [29]
    S. V. Rzhevskij. Monotone Algorithm of Search for Saddle Point of Nonsmooth Function. - Kibernetika, 1982, No. 1, pp. 95–98.Google Scholar
  30. [30]
    V. A. Skokov. Note on Minimization Methods Using Operation of Space Dilatation. - Kibernetika, 1974, No. 4, pp. 115–117.Google Scholar
  31. [31]
    L. G. Khachijan. Polynomial Algorithm in Linear Programming. - Dokl. AN S.S.S.R., 1979, Vol. 244, No. 5, pp. 1093–1096.Google Scholar
  32. [32]
    L. G. Khachijan. Polynomial Algorithms in Linear Programming. - J. Vychisl. Matem. i Matem. Fiziki, 1980, Vol. 20, No. 1, pp. 51–68.Google Scholar
  33. [33]
    M. K. Kozlov, S. P. Tarasov, L. G. Khachijan. Polynomial Solvability of Convex Quadratic Programming. - Dokl. AN S.S.S.R., 1979, Vol. 248, No. 5.Google Scholar
  34. [34]
    N. Z. Shor. Application of Gradient Descent Method for Solution of Network Transportation Problems. In the Book: Notes, Scientific Seminar on Theory and Application of Cybernetics and Operations Research. Nauchn. Sovet po Kibernetika AN U.S.S.R, Kiev, 1962, Issue 1, pp. 9–17.Google Scholar
  35. [35]
    N. Z. Shor. Rate of Convergence of Generalized Gradient Descent Method. Kibernetika, 1968, No. 3, pp. 98–99.Google Scholar
  36. [36]
    N. Z. Shor. Utilization of Space Dilatation Operation in Minimization of Convex Functions. - Kibernetika, 1970, No. 1, pp. 6–12.Google Scholar
  37. [37]
    N. Z. Shor. About Rate of Convergence of Method of Generalized Gradient Descent with Space Dilatation. - Kibernetika, 1970, No. 2, pp. 80–85.Google Scholar
  38. [38]
    N. Z. Shor, N. G. Zhurbenko. Minimization Method Using Operation of Space Dilatation in the Direction of Difference of Two Sequential Gradients. - Kibernetika, 1971, No. 3, pp. 51–59.Google Scholar
  39. [39]
    N. Z. Shor, L. P. Shabashova. Solution of Minimax Problems by Generalized Gradient Descent Method with Space Dilatation. - Kibernetika, 1972, No. 1, pp. 82– 88.Google Scholar
  40. [40]
    N. Z. Shor, V. I. Gershovich. Family of Algorithms for Solving Problems of Convex Programming. - Kibernetika, 1979, No. 4, pp. 62–67.MathSciNetGoogle Scholar
  41. [41]
    N. Z. Shor. Method of Minimization of Almost Differentiable Functions. - Kibernetika, 1972, No. 4, pp. 65–70.Google Scholar
  42. [42]
    N. Z. Shor, V. I. Gershovich. About One Modification of Gradient-Type Algorithms with Space Dilatation for Solution of Large-Scale Problems. - Kibernetika, 1981, No. 5, pp. 67–70.MathSciNetGoogle Scholar
  43. [43]
    N. Z. Shor. Convergence of Gradient-Type Method with Space Dilatation in the Direction of Difference of Two Sequential Gradients. - Kibernetika, 1975, No. 4, pp. 48–53.Google Scholar
  44. [44]
    N. Z. Shor. Cutting Method with Space Dilatation for Solving Problems of Convex Programming. - Kibernetika, 1977, No. 1, pp. 94–95.Google Scholar
  45. [45]
    N. Z. Shor. New Directions in Development of Nonsmooth Optimization Methods. - Kibernetika, 1977, No. 6, pp. 87–91.Google Scholar
  46. [46]
    N. Z. Shor. Methods of Optimization of Non-Differentiable Functions and Their Applications. - Kiev, Naukova Dumka, 1979, 200 p.Google Scholar
  47. [47]
    D. B. Judin, A. S. Nemirovskij. Evaluation of Information Complexity and Effective Methods for Solving of Complex Extremum Problems. - Ekonomika i Matema- ticheskie Metody, 1976, Vol. 12, Issue 2, pp. 357–369.MathSciNetGoogle Scholar
  48. [48]
    D. Agmon. The Relaxation Method for Linear Inequalities. - Can. J. Math. 1954, Vol. 6, No. 2, pp. 381–392.MathSciNetGoogle Scholar
  49. [49]
    B. Aspvall, G. Shiloach. A Polynomial Time Algorithm for Solving Systems of Linear Inequalities with Two Variables per Inequality. SIAM J. Comput., Vol. 9, No. 4, Nov. 1980.Google Scholar
  50. [50]
    R. Bland, D. Goldfarb, M. Todd. The Ellipsoid Method: A Survey. Cornell University, Ithaca, New York, Technical Report No. 476, 79 p.Google Scholar
  51. [51]
    F. Clarke. Generalized Gradients and Applications. - Trans. Amer. Math. Soc., 1975, Vol. 205, pp. 247–262.MathSciNetMATHCrossRefGoogle Scholar
  52. [52]
    M. Grötschel, L. Lovâsz, A. Schrijver. The Ellipsoid Method and its Consequences in Combinatorial optimization. - Combinatorica, 1981, Vol. I, No. 2, pp. 169–197.Google Scholar
  53. [53]
    M. Grötschel, L. Lovâsz, A. Schrijver. Polynomial Algorithms for Perfect Graphs. Report No. 81176-OR, Bonn University, Febr. 1981.Google Scholar
  54. [54]
    J. Kelley. The “Cutting Plane” Method for Solving Convex Programs. - SIAM Journal, 1960, Vol. 8, No. 4, pp. 703–712.MathSciNetGoogle Scholar
  55. [55]
    B. Körte, R. Schräder. A Note on Convergence Proofs for Shor-Khachijan Methods. - In “Optimization and Optimal Control”. Lecture Notes in Control and Information Sciences 30, Springer (1980), pp. 51–57.Google Scholar
  56. [56]
    C. Lemarechal. An Extension of Davidon Methods to Nondifferentiable Problems. - Math. Program. Study 3, (1975), pp. 95–100.MathSciNetGoogle Scholar
  57. [57]
    L. Lovâsz. On the Shannon Capacity of a Graph. IEEE Trans. Inform. Theory 11-25 (1979), pp. 1–7.Google Scholar
  58. [58]
    J. F. Maurras, K. Truemper, M. Akgül. Polynomial Algorithms for a Class of Linear Programs. Math. Progr., 21 (1981), pp. 121–136.MATHCrossRefGoogle Scholar
  59. [59]
    R. Mifflin. Semismooth and Semiconvex Functions in Constrained Optimization. RR-76-21. HAS A. - Laxenburg, Austria, 1976, 23 p.Google Scholar
  60. [60]
    R. Mifflin. An Algorithm for Constrained Optimization with Semismooth Functions. RR-73-3. IIASA. - Laxenburg, Austria, 1977, 32 p.Google Scholar
  61. [61]
    T. Motzkin, I. Schoenberg. The Relaxation Method for Linear Inequalities. - Can. J. Math., 1954, Vol. 6, No. 2, pp. 393–404.MathSciNetMATHGoogle Scholar
  62. [62]
    R. Schräder. Ellipsoid Methods. In the Book: Modern Applied Mathematics: Optimization and Operations Research, B. Körte (ed.), North-Holland, Amsterdam, 1982, pp. 265–311.Google Scholar
  63. [63]
    B. Schwartz. Zur Minimierung konvexer sub differenzierbarer Funktionen über dem Rn. Math. Operationsforschung und Statistik, Series Optimization, 1978, 9, No. 4, pp. 545–557.MathSciNetMATHGoogle Scholar
  64. [64]
    P. Wolfe. A Method of Conjugate Subgradients for Minimizing Nondifferentiable Functions. - Math. Program. Study 3, (1975), pp. 145–173.MathSciNetGoogle Scholar
  65. [65]
    P. Wolfe. A Bibliography for the Ellipsoid Algorithm. IBM Research Center, York-town Heights. New York, 1980, 6 p.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1983

Authors and Affiliations

  • N. Z. Shor
    • 1
  1. 1.V. M. Glushkov, Institute of CyberneticsAcademy of Sciences of the Ukrainian SSRKiev - 207USSR

Personalised recommendations