Abstract
We consider an incremental approximation method for solving variational problems in infinite-dimensional separable Hilbert spaces, where in each step a randomly and independently selected subproblem from an infinite collection of subproblems is solved. We show that convergence rates for the expectation of the squared error can be guaranteed under weaker conditions than previously established in Griebel and Oswald (Constr Approx 44(1):121–139, 2016).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barron, A., Cohen, A., Dahmen, W., DeVore, R.: Approximation and learning by greedy algorithms. Ann. Stat. 3, 64–94 (2008)
Binev, P., Cohen, A., Mula, O., Nichols, J.: Greedy algorithms for optimal measurements selection in state estimation using reduced models. SIAM/ASA J. Uncertain. Quantif. 6(3), 1101–1126 (2018)
Bogachev, V.: Gaussian Measures. AMS, Providence (1998)
Da Prato, G., Zabczyk, J.: Stochastic Equations in Infinite Dimensions. Cambridge University Press, Cambridge (1992)
Dieuleveut, A., Bach, F.: Nonparametric stochastic approximation with large step-sizes. Ann. Stat. 44(4), 1363–1399 (2016)
Fercoq, O., Richtarik, P.: Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent. SIAM Rev. 58(4), 739–771 (2016)
Griebel, M., Oswald, P.: On the abstract theory of additive and multiplicative Schwarz algorithms. Numer. Math. 70, 163–180 (1995)
Griebel, M., Oswald, P.: Greedy and randomized versions of the multiplicative Schwarz method. Linear Algebra Appl. 437(7), 1596–1610 (2012)
Griebel, M., Oswald, P.: Schwarz iterative methods: infinite space splittings. Constr. Approx. 44(1), 121–139 (2016)
Lin, J., Zhou, D.-X.: Learning theory of randomized Kaczmarz algorithm. J. Mach. Learn. Res. 16, 3341–3365 (2015)
Tarrés, P., Yao, Y.: Online learning as stochastic approximation of regularization paths: optimality and almost-sure convergence. IEEE Trans. Inf. Theory 60(9), 5716–5735 (2014)
Temlyakov, V.: Greedy Approximation. Cambridge University Press, Cambridge (2012)
Smale, S., Zhou, D.-X.: Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press, Cambridge (2007)
Wendland, H.: Scattered Data Approximation. Cambridge University Press, Cambridge (2010)
Xu, J.: Iterative methods by space decomposition and subspace correction. SIAM Rev. 34(4), 581–613 (1992)
Acknowledgements
M. Griebel was partially supported by the project EXAHD of the DFG priority program 1648 Software for Exascale Computing (SPPEXA) and by the Sonderforschungsbereich 1060 The Mathematics of Emergent Effects funded by the Deutsche Forschungsgemeinschaft. This paper was written while P. Oswald held a Bonn Research Chair sponsored by the Hausdorff Center for Mathematics at the University of Bonn funded by the Deutsche Forschungsgemeinschaft. He is grateful for this support.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Albert Cohen.
Rights and permissions
About this article
Cite this article
Griebel, M., Oswald, P. Stochastic Subspace Correction in Hilbert Space. Constr Approx 48, 501–521 (2018). https://doi.org/10.1007/s00365-018-9447-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00365-018-9447-1
Keywords
- Infinite space splitting
- Subspace correction
- Multiplicative Schwarz
- Block coordinate descent
- Greedy
- Randomized
- Convergence rates
- Online learning