Skip to main content
SpringerLink
Log in

A Kaczmarz algorithm for sequences of projections, infinite products, and applications to frames in IFS \(L^{2}\) spaces

  • Original Paper
  • Published:
Advances in Operator Theory Aims and scope Submit manuscript

Abstract

We show that an idea, originating initially with a fundamental recursive iteration scheme (usually referred as “the” Kaczmarz algorithm), admits important applications in such infinite-dimensional, and non-commutative, settings as are central to spectral theory of operators in Hilbert space, to optimization, to large sparse systems, to iterated function systems (IFS), and to fractal harmonic analysis. We present a new recursive iteration scheme involving as input a prescribed sequence of selfadjoint projections. Applications include random Kaczmarz recursions, their limits, and their error-estimates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Alpay, D., Jorgensen, P., Lewkowicz, I.: \(W\)-Markov measures, transfer operators, wavelets and multiresolutions. In: Frames and harmonic analysis, volume 706 of Contemp. Math., pp. 293–343. Amer. Math. Soc., Providence, RI (2018)

  2. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950)

    MathSciNet  MATH  Google Scholar 

  3. Baranov, A., Sarason, D.: Quotient representations of inner functions. In: Recent Trends in Analysis, volume 16 of Theta Ser. Adv. Math., pp. 35–46. Theta, Bucharest (2013)

  4. Barnsley, M.F., Hutchinson, J.E., Stenflo, Ö.: \(V\)-variable fractals: fractals with partial self similarity. Adv. Math. 218(6), 2051–2088 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Bauschke, H.H.: A norm convergence result on random products of relaxed projections in Hilbert space. Trans. Am. Math. Soc. 347(4), 1365–1373 (1995)

    MathSciNet  MATH  Google Scholar 

  6. Bemrose, T., Casazza, P.G., Kaftal, V., Lynch, R.G.: The unconditional constants for Hilbert space frame expansions. Linear Algebra Appl. 521, 1–18 (2017)

    MathSciNet  MATH  Google Scholar 

  7. Candès, E.J., Eldar, Y.C., Strohmer, T., Voroninski, V.: Phase retrieval via matrix completion [reprint of MR3032952]. SIAM Rev. 57(2), 225–251 (2015)

    MathSciNet  MATH  Google Scholar 

  8. Casazza, P.G., Haas, J.I.: On Grassmannian frames with spectral constraints. Sample Theory Signal Image Process. 17(1), 17–28 (2018)

    MathSciNet  MATH  Google Scholar 

  9. Casazza, P.G., Kutyniok, G.: Frames of subspaces. In: Wavelets, Frames and Operator Theory, volume 345 of Contemp. Math., pp. 87–113. Amer. Math. Soc., Providence, RI (2004)

  10. Casazza, P.G., Kutyniok, G.: Robustness of fusion frames under erasures of subspaces and of local frame vectors. In: Radon Transforms, Geometry, and Wavelets, volume 464 of Contemp. Math., pp. 149–160. Amer. Math. Soc., Providence, RI (2008)

  11. Casazza, P.G., Kutyniok, G., Li, S.: Fusion frames and distributed processing. Appl. Comput. Harmon. Anal. 25(1), 114–132 (2008)

    MathSciNet  MATH  Google Scholar 

  12. Chen, X.: The Kaczmarz algorithm, row action methods, and statistical learning algorithms. In: Frames and harmonic analysis, volume 706 of Contemp. Math., pp. 115–127. Amer. Math. Soc., Providence, RI (2018)

  13. Chen, Y., Candès, E.J.: The projected power method: an efficient algorithm for joint alignment from pairwise differences. Comm. Pure Appl. Math. 71(8), 1648–1714 (2018)

    MathSciNet  MATH  Google Scholar 

  14. Czaja, W., Tanis, J.H.: Kaczmarz algorithm and frames. Int. J. Wavelets Multiresolut. Inf. Process 11(5), 1350036 (2013)

    MathSciNet  MATH  Google Scholar 

  15. Dutkay, D.E., Jorgensen, P.E.T.: Fourier frequencies in affine iterated function systems. J. Funct. Anal. 247(1), 110–137 (2007)

    MathSciNet  MATH  Google Scholar 

  16. Ehler, M., Okoudjou, K.A.: Probabilistic frames: an overview. In: Finite frames, Appl. Numer. Harmon. Anal., pp. 415–436. Birkhäuser/Springer, New York (2013)

  17. Eldar, Y.C., Needell, D.: Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer. Algorithms 58(2), 163–177 (2011)

    MathSciNet  MATH  Google Scholar 

  18. Evans, D.J., Popa, C.: Projections and preconditioning for inconsistent least-squares problems. Int. J. Comput. Math. 78(4), 599–616 (2001)

    MathSciNet  MATH  Google Scholar 

  19. Fickus, M., Johnson, B.D., Kornelson, K., Okoudjou, K.A.: Convolutional frames and the frame potential. Appl. Comput. Harmon. Anal. 19(1), 77–91 (2005)

    MathSciNet  MATH  Google Scholar 

  20. Friedlander, B.: A subspace method for space time adaptive processing. IEEE Trans. Signal Process. 53(1), 74–82 (2005)

    MathSciNet  MATH  Google Scholar 

  21. Führ, H., Lemvig, J.: System bandwidth and the existence of generalized shift-invariant frames. J. Funct. Anal. 276(2), 563–601 (2019)

    MathSciNet  MATH  Google Scholar 

  22. Haller, R., Szwarc, R.: Kaczmarz algorithm in Hilbert space. Stud. Math. 169(2), 123–132 (2005)

    MathSciNet  MATH  Google Scholar 

  23. Han, D., Kornelson, K., Larson, D., Weber, E.: Frames for undergraduates. Student Mathematical Library, vol. 40. American Mathematical Society, Providence, RI (2007)

  24. Han, D., Larson, D.R., Liu, R.: Dilations of operator-valued measures with bounded \(p\)-variations and framings on Banach spaces. J. Funct. Anal. 274(5), 1466–1490 (2018)

    MathSciNet  MATH  Google Scholar 

  25. Herr, J.E., Jorgensen, P.E.T., Weber, E.S.: A matrix characterization of boundary representations of positive matrices in the Hardy space. In: Frames and harmonic analysis, volume 706 of Contemp. Math., pp. 255–270. Amer. Math. Soc., Providence, RI, (2018)

  26. Herr, J.E., Jorgensen, P.E.T., Weber, E.S.: Positive matrices in the Hardy space with prescribed boundary representations via the Kaczmarz algorithm. J. Anal. Math. 138(1), 209–234 (2019)

    MathSciNet  MATH  Google Scholar 

  27. Herr, J.E., Jorgensen, P.E.T., Weber, E.S.: A characterization of boundary representations of positive matrices in the hardy space via the abel product. Linear Algebra Appl. 576, 51–66 (2019)

    MathSciNet  MATH  Google Scholar 

  28. Hida, T.: Brownian motion, volume 11 of Applications of Mathematics. Springer-Verlag, New York-Berlin, 1980. Translated from the Japanese by the author and T. P. Speed

  29. Hotovy, R., Larson, D.R., Scholze, S.: Binary frames. Houston J. Math. 41(3), 875–899 (2015)

    MathSciNet  MATH  Google Scholar 

  30. Huang, J., Huang, T.-Z.: A nonstationary accelerating alternating direction method for frame-based Poissonian image deblurring. J. Comput. Appl. Math. 352, 181–193 (2019)

    MathSciNet  MATH  Google Scholar 

  31. Hutchinson, J.E.: Fractals and self-similarity. Indiana Univ. Math. J. 30(5), 713–747 (1981)

    MathSciNet  MATH  Google Scholar 

  32. Hutchinson, J.E.: Fractals: a mathematical framework. Complex. Int. 2, 14 (1995)

    MathSciNet  MATH  Google Scholar 

  33. Ivanov, A.I., Zhdanov, A.I.: Kaczmarz algorithm for Tikhonov regularization problem. Appl. Math. E-Notes 13, 270–276 (2013)

    MathSciNet  MATH  Google Scholar 

  34. Jorgensen, P., Tian, F.: Non-commutative Analysis. World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2017. With a foreword by Wayne Polyzou

  35. Jorgensen, P.E.T., Song, M.-S.: Entropy encoding, Hilbert space, and Karhunen–Loève transforms. J. Math. Phys. 48(10), 103503 (2007)

    MathSciNet  MATH  Google Scholar 

  36. Jorgensen, P.E.T., Song, M.-S.: Infinite-dimensional measure spaces and frame analysis. Acta Appl. Math. 155, 41–56 (2018)

    MathSciNet  MATH  Google Scholar 

  37. Jorgensen, P.E.T., Song, M.-S.: Markov chains and generalized wavelet multiresolutions. J. Anal. 26(2), 259–283 (2018)

    MathSciNet  MATH  Google Scholar 

  38. Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bulletin International de l’Académie Polonaise des Sciences et des Lettres 35, 355–357 (1937)

    MATH  Google Scholar 

  39. Kaftal, V., Larson, D.R.: Admissible sequences of positive operators. Trans. Am. Math. Soc. 371(5), 3721–3742 (2019)

    MathSciNet  MATH  Google Scholar 

  40. Kakutani, S.: Notes on infinite product measure spaces. II. Proc. Imp. Acad. Tokyo 19, 184–188 (1943)

    MathSciNet  MATH  Google Scholar 

  41. Kakutani, S.: On equivalence of infinite product measures. Ann. Math. 2(49), 214–224 (1948)

    MathSciNet  MATH  Google Scholar 

  42. Khader, M.M., Adel, M.: Introducing the windowed Fourier frames technique for obtaining the approximate solution of the coupled system of differential equations. J. Pseudo-Differ. Oper. Appl. 10(1), 241–256 (2019)

    MathSciNet  MATH  Google Scholar 

  43. Kolmogorov, A. N.: On logical foundations of probability theory. In Probability theory and mathematical statistics (Tbilisi, 1982), volume 1021 of Lecture Notes in Math., pp. 1–5. Springer, Berlin, (1983)

  44. Kwapień, S., Mycielski, J.: Erratum to the paper: “On the Kaczmarz algorithm of approximation in infinite-dimensional spaces” [Studia Math. 148 (2001), no. 1, 75–86; mr1881441]. Studia Math., 176(1):93, (2006)

  45. Lin, J., Zhou, D.-X.: Learning theory of randomized Kaczmarz algorithm. J. Mach. Learn. Res. 16, 3341–3365 (2015)

    MathSciNet  MATH  Google Scholar 

  46. Ling, S., Strohmer, T.: Self-calibration and biconvex compressive sensing. Inverse Probl. 31(11), 115002 (2015)

    MathSciNet  MATH  Google Scholar 

  47. Needell, D., Srebro, N., Ward, R.: Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm. Math. Program. 155(1–2), 549–573 (2016)

    MathSciNet  MATH  Google Scholar 

  48. Netyanun, A., Solmon, D.C.: Iterated products of projections in Hilbert space. Am. Math. Mon 113(7), 644–648 (2006)

    MathSciNet  MATH  Google Scholar 

  49. Okoudjou, K.A.: Preconditioning techniques in frame theory and probabilistic frames. In Finite frame theory, volume 73 of Proc. Sympos. Appl. Math., pp. 105–142. Amer. Math. Soc., Providence, RI, (2016)

  50. Okoudjou, K.A., Strichartz, R.S.: Asymptotics of eigenvalue clusters for Schrödinger operators on the Sierpiński gasket. Proc. Am. Math. Soc. 135(8), 2453–2459 (2007)

    MATH  Google Scholar 

  51. Popa, C.: Oblique projections as preconditioner in Kaczmarz-like algorithms. In Proceedings of the Ninth Symposium of Mathematics and its Applications, pp. 118–122. Rom. Acad., Timişoara, (2001)

  52. Popa, C.: A hybrid Kaczmarz-conjugate gradient algorithm for image reconstruction. Math. Comput. Simul. 80(12), 2272–2285 (2010)

    MathSciNet  MATH  Google Scholar 

  53. Popa, C.: Convergence rates for Kaczmarz-type algorithms. Numer. Algorithms 79(1), 1–17 (2018)

    MathSciNet  MATH  Google Scholar 

  54. Ruelle, D.: Analycity properties of the characteristic exponents of random matrix products. Adv. Math. 32(1), 68–80 (1979)

    MathSciNet  MATH  Google Scholar 

  55. Ruelle, D.: Characteristic exponents and invariant manifolds in Hilbert space. Ann. Math. 115(2), 243–290 (1982)

    MathSciNet  MATH  Google Scholar 

  56. Ruelle, D.: Thermodynamic formalism. Cambridge Mathematical Library. Cambridge University Press, Cambridge, second edition, 2004. The mathematical structures of equilibrium statistical mechanics

  57. Ruelle, D., Takens, F.: On the nature of turbulence. Comm. Math. Phys. 20, 167–192 (1971)

    MathSciNet  MATH  Google Scholar 

  58. Sarason, D.: Sub-Hardy Hilbert spaces in the unit disk, volume 10 of University of Arkansas Lecture Notes in the Mathematical Sciences. Wiley, New York, 1994. A Wiley-Interscience Publication

  59. Soltanolkotabi, M., Elhamifar, E., Candès, E.J.: Robust subspace clustering. Ann. Stat. 42(2), 669–699 (2014)

    MathSciNet  MATH  Google Scholar 

  60. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15(2), 262–278 (2009)

    MathSciNet  MATH  Google Scholar 

  61. Szwarc, R.: Kaczmarz algorithm in Hilbert space and tight frames. Appl. Comput. Harmon. Anal. 22(3), 382–385 (2007)

    MathSciNet  MATH  Google Scholar 

  62. Wickman, C., Okoudjou, K.: Duality and geodesics for probabilistic frames. Linear Algebra Appl. 532, 198–221 (2017)

    MathSciNet  MATH  Google Scholar 

  63. Zhang, J.-J.: A new greedy Kaczmarz algorithm for the solution of very large linear systems. Appl. Math. Lett. 91, 207–212 (2019)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The co-authors thank the following colleagues for helpful and enlightening discussions: Professors Daniel Alpay, Sergii Bezuglyi, Ilwoo Cho, Wayne Polyzou, Eric S. Weber, and members in the Math Physics seminar at The University of Iowa.

Author information

Authors and Affiliations

  1. The University of Iowa, Iowa City, IA, 52242-1419, USA

    Palle Jorgensen

  2. Department of Mathematics, Southern Illinois University Edwardsville, Edwardsville, IL, 62026, USA

    Myung-Sin Song

  3. Mathematical Reviews, 416 4th Street, Ann Arbor, MI, 48103-4816, USA

    James Tian

Authors
  1. Palle Jorgensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myung-Sin Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James Tian.

Additional information

Communicated by Uwe Franz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jorgensen, P., Song, MS. & Tian, J. A Kaczmarz algorithm for sequences of projections, infinite products, and applications to frames in IFS \(L^{2}\) spaces. Adv. Oper. Theory 5, 1100–1131 (2020). https://doi.org/10.1007/s43036-020-00079-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43036-020-00079-1

Keywords

Mathematics Subject Classification

Search

Navigation