Advertisement

Descent Methods

  • Simeon Reich
  • Alexander J. Zaslavski
Part of the Developments in Mathematics book series (DEVM, volume 34)

Abstract

In Chap. 8 we study discrete and continuous descent methods for minimizing a convex (Lipschitz) function on a general Banach space. We consider a space of vector fields V such that for any point x in the Banach space, the directional derivative in the direction Vx is nonpositive. This space of vector fields is equipped with a complete metric. Each vector field generates two gradient type algorithms (discrete descent methods) and a flow which consists of the solutions of the corresponding evolution equation (continuous descent method). We show that most (in the sense of Baire category) vector fields produce algorithms for which values of the objective function tend to its infimum as t tends to infinity. Actually, we introduce the subclass of regular vector fields, show that the convergence property stated above holds for them and that a generic vector field is regular. We also show that this convergence property is stable under small perturbations of a given regular vector field.

Keywords

Banach Space Vector Field Natural Number Descent Method Weak Topology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Aizicovici, S., Reich, S., & Zaslavski, A. J. (2005). Archiv der Mathematik, 85, 268–277. MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Alber, Y. I., Iusem, A. N., & Solodov, M. V. (1997). Journal of Convex Analysis, 4, 235–255. MathSciNetzbMATHGoogle Scholar
  3. 19.
    Ben-Tal, A., & Zibulevsky, M. (1997). SIAM Journal on Optimization, 7, 347–366. MathSciNetCrossRefzbMATHGoogle Scholar
  4. 41.
    Clarke, F. H. (1983). Optimization and nonsmooth analysis. New York: Wiley. zbMATHGoogle Scholar
  5. 44.
    Correa, R., & Lemaréchal, C. (1993). Mathematical Programming, 62, 261–275. MathSciNetCrossRefzbMATHGoogle Scholar
  6. 47.
    Curry, H. B. (1944). Quarterly of Applied Mathematics, 2, 258–261. MathSciNetzbMATHGoogle Scholar
  7. 69.
    Gowda, M. S., & Teboulle, M. (1990). SIAM Journal on Control and Optimization, 28, 925–935. MathSciNetCrossRefzbMATHGoogle Scholar
  8. 73.
    Hiriart-Urruty, J.-B., & Lemaréchal, C. (1993). Convex analysis and minimization algorithms. Berlin: Springer. Google Scholar
  9. 89.
    Lebourg, G. (1979). Transactions of the American Mathematical Society, 256, 125–144. MathSciNetCrossRefzbMATHGoogle Scholar
  10. 103.
    Neuberger, J. W. (2010). Lecture notes in mathematics: Vol. 1670. Sobolev gradients and differential equations (2nd ed.). Berlin: Springer. CrossRefzbMATHGoogle Scholar
  11. 135.
    Reich, S., & Zaslavski, A. J. (2000). Mathematics of Operations Research, 25, 231–242. MathSciNetCrossRefzbMATHGoogle Scholar
  12. 136.
    Reich, S., & Zaslavski, A. J. (2001). SIAM Journal on Optimization, 11, 1003–1018. MathSciNetCrossRefzbMATHGoogle Scholar
  13. 141.
    Reich, S., & Zaslavski, A. J. (2001). Nonlinear Analysis, 47, 3247–3258. MathSciNetCrossRefzbMATHGoogle Scholar
  14. 148.
    Reich, S., & Zaslavski, A. J. (2003). Electronic Journal of Differential Equations, 2003, 1–11. MathSciNetGoogle Scholar
  15. 163.
    Reich, S., & Zaslavski, A. J. (2008). Taiwanese Journal of Mathematics, 12, 1165–1176. MathSciNetzbMATHGoogle Scholar
  16. 176.
    Zaslavski, A. J. (1981). Siberian Mathematical Journal, 22, 63–68. CrossRefGoogle Scholar
  17. 182.
    Zaslavski, A. J. (2010). Optimization on metric and normed spaces. New York: Springer. CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Simeon Reich
    • 1
  • Alexander J. Zaslavski
    • 1
  1. 1.Department of MathematicsTechnion-Israel Institute of TechnologyHaifaIsrael

Personalised recommendations