Skip to main content
SpringerLink
Log in

Dual Convergence Estimates for a Family of Greedy Algorithms in Banach Spaces

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Big Data (MOD 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10710))

Included in the following conference series:

Abstract

The paper examines four weak relaxed greedy algorithms for finding approximate sparse solutions of convex optimization problems in a Banach space. First, we present a review of primal results on the convergence rate of the algorithms based on the geometric properties of the objective function. Then, using the ideas of [16], we define the duality gap and prove that the duality gap is a certificate for the current approximation to the optimal solution. Finally, we find estimates of the dependence of the duality gap values on the number of iterations for weak greedy algorithms.

This work was supported by the Russian Fund for Basic Research, projects 16-01-00507, 18-01-00408.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barron, A.R., Cohen, A., Dahmen, W., DeVore, R.A.: Approximation and learning by greedy algorithms. Ann. Stat. 36(1), 64–94 (2008)

    MathSciNet  MATH  Google Scholar 

  2. Blumensath, T., Davies, M.E.: Gradient pursuits. IEEE Trans. Signal Process. 56, 2370–2382 (2008)

    MathSciNet  MATH  Google Scholar 

  3. Blumensath, T., Davies, M.: Stagewise weak gradient pursuits. IEEE Trans. Signal Process. 57, 4333–4346 (2009)

    MathSciNet  MATH  Google Scholar 

  4. Bubeck, S.: Convex optimization: algorithms and complexity. Found. Trends Mach. Learn. 8(3–4), 231–358 (2015)

    MATH  Google Scholar 

  5. Clarkson, K.L.: Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. ACM Trans. Algorithms 6(4), 1–30 (2010)

    MathSciNet  MATH  Google Scholar 

  6. Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximation. Constr. Approx. 13, 57–98 (1997)

    MathSciNet  MATH  Google Scholar 

  7. Demyanov, V., Rubinov, A.: Approximate Methods in Optimization Problems. American Elsevier Publishing Co., New York (1970)

    Google Scholar 

  8. Dereventsov, A.V.: On the approximate weak Chebyshev greedy algorithm in uniformly smooth banach spaces. J. Math. Anal. Appl. 436(1), 288–304 (2016)

    MathSciNet  MATH  Google Scholar 

  9. DeVore, R.A., Temlyakov, V.N.: Some remarks on greedy algorithms. Adv. Comput. Math. 5, 173–187 (1996)

    MathSciNet  MATH  Google Scholar 

  10. DeVore, R.A., Temlyakov, V.N.: Convex optimization on Banach spaces. Found. Comput. Math. 16(2), 369–394 (2016)

    MathSciNet  MATH  Google Scholar 

  11. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Res. Logis. Quart. 3, 95–110 (1956)

    MathSciNet  Google Scholar 

  12. Freund, R.M., Grigas, P.: New analysis and results for the Frank-Wolfe method. Math. Program. 155(1), 199–230 (2016)

    MathSciNet  MATH  Google Scholar 

  13. Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)

    MathSciNet  MATH  Google Scholar 

  14. Georgiev, P.G., Sánchez-González, L., Pardalos, P.M.: Construction of pairs of reproducing kernel Banach spaces. In: Demyanov, V., Pardalos, P., Batsyn, M. (eds.) Constructive Nonsmooth Analysis and Related Topics. SOIA, vol. 87, pp. 39–57. Springer, New York (2014). https://doi.org/10.1007/978-1-4614-8615-2_4

    Google Scholar 

  15. Huber, P.J.: Projection pursuit. Ann. Statist. 13, 435–525 (1985)

    MathSciNet  MATH  Google Scholar 

  16. Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 427–435 (2013)

    Google Scholar 

  17. Jones, L.: On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Statist. 15, 880–882 (1987)

    MathSciNet  MATH  Google Scholar 

  18. Konyagin, S.V., Temlyakov, V.N.: A remark on greedy approximation in Banach spaces. East J. Approx. 5(3), 365–379 (1999)

    MathSciNet  MATH  Google Scholar 

  19. Levitin, E.S., Polyak, B.T.: Constrained minimization methods. USSR Comp. Math. M. Phys. 6(5), 1–50 (1966)

    Google Scholar 

  20. Nemirovski, A.: Optimization II: Numerical methods for nonlinear continuous optimization. Lecture Notes, Israel Institute of Technology (1999)

    Google Scholar 

  21. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, Boston (2004)

    MATH  Google Scholar 

  22. Nguyen, H., Petrova, G.: Greedy strategies for convex optimization. Calcolo 41(2), 1–18 (2016)

    MATH  Google Scholar 

  23. Polyak, B.T.: Introduction to Optimization. Optimization Software Inc., New York (1987)

    MATH  Google Scholar 

  24. Temlyakov, V.N.: Greedy approximation in convex optimization. Constr. Approx. 41(2), 269–296 (2015)

    MathSciNet  MATH  Google Scholar 

  25. Temlyakov, V.N.: Dictionary descent in optimization. Anal. Mathematica 42(1), 69–89 (2016)

    MathSciNet  MATH  Google Scholar 

  26. Zhang, H., Zhang, J.: Learning with reproducing Kernel Banach spaces. In: Dang, P., Ku, M., Qian, T., Rodino, L.G. (eds.) New Trends in Analysis and Interdisciplinary Applications. TM, pp. 417–423. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-48812-7_53

    Google Scholar 

  27. Zhang, Z., Shwartz, S., Wagner, L., Miller, W.: A greedy algorithm for aligning DNA sequences. J. Comput. Biol. 7(1–2), 203–214 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saratov State University, Saratov, Russian Federation

    S. P. Sidorov, S. V. Mironov & M. G. Pleshakov

Authors
  1. S. P. Sidorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. V. Mironov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. G. Pleshakov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. P. Sidorov .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  3. University of Catania, Catania, Italy

    Giovanni Giuffrida

  4. Harvard University, Cambridge, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sidorov, S.P., Mironov, S.V., Pleshakov, M.G. (2018). Dual Convergence Estimates for a Family of Greedy Algorithms in Banach Spaces. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R. (eds) Machine Learning, Optimization, and Big Data. MOD 2017. Lecture Notes in Computer Science(), vol 10710. Springer, Cham. https://doi.org/10.1007/978-3-319-72926-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-72926-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-72925-1

  • Online ISBN: 978-3-319-72926-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation