Journal of Scientific Computing

, Volume 75, Issue 2, pp 859–888 | Cite as

Flows Generating Nonlinear Eigenfunctions

Article
  • 61 Downloads

Abstract

Linear eigenvalue analysis has provided a fundamental framework for many scientific and engineering disciplines. Consequently, vast research was devoted to numerical schemes for computing eigenfunctions. In recent years, new research in image processing and machine-learning has shown the applicability of nonlinear eigenvalue analysis, specifically based on operators induced by convex functionals. This has provided new insights, better theoretical understanding and improved image-processing, clustering and classification algorithms. However, the theory of nonlinear eigenvalue problems is still very preliminary. We present a new class of nonlinear flows that can generate nonlinear eigenfunctions of the form \(T(u)=\lambda u\), where T(u) is a nonlinear operator and \(\lambda \in \mathbb {R} \) is the eigenvalue. We develop the theory where T(u) is a subgradient element of a regularizing one-homogeneous functional, such as total-variation or total-generalized-variation. We focus on a forward flow which simultaneously smooths the solution (with respect to the regularizer) while increasing the 2-norm. An analog discrete flow and its normalized version are formulated and analyzed. The flows translate to a series of convex minimization steps. In addition we suggest an indicator to measure the affinity of a function to an eigenfunction and relate it to pseudo-eigenfunctions in the linear case.

Keywords

Nonlinear eigenfunctions Variational methods Nonlinear flows Total-variation Nonlinear spectral theory One-homogeneous functionals 

Notes

Acknowledgements

We would like to thank Martin Benning, Nicolas Papadakis and Jean-Francois Aujol for stimulating discussions and helpful comments. We would further like to thank the two anonymous reviewers for their suggestions. We acknowledge support by the Israel Science Foundation (Grant No. 718/15).

References

  1. 1.
    Appell, J., De Pascale, E., Vignoli, A.: Nonlinear spectral theory, vol. 10. Walter de Gruyter, Berlin (2004)CrossRefMATHGoogle Scholar
  2. 2.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math. Program. 137, 91–129 (2013)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Aubert, G., Aujol, J.-F.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68, 925–946 (2008)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Aujol, J., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition—modeling, algorithms, and parameter selection. Int. J. Comput. Vision 67, 111–136 (2006)CrossRefMATHGoogle Scholar
  5. 5.
    Aujol, J.-F., Gilboa, G., Papadakis, N.: Theoretical analysis of flows estimating eigenfunctions of one-homogeneous functionals for segmentation and clustering. Preprint. HAL-01563922. (2017)Google Scholar
  6. 6.
    Bellettini, G., Caselles, V., Novaga, M.: The total variation flow in r n. J. Differ. Equ. 184, 475–525 (2002)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Benning, M., Brune, C., Burger, M., Müller, J.: Higher-order tv methodsenhancement via bregman iteration. J. Sci. Comput. 54, 269–310 (2013)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Benning, M., Burger, M.: Ground states and singular vectors of convex variational regularization methods. Methods Appl. Anal. 20, 295–334 (2013)MathSciNetMATHGoogle Scholar
  9. 9.
    Börm, S., Mehl, C.: Numerical Methods for Eigenvalue Problems. Walter de Gruyter, Berlin (2012)CrossRefMATHGoogle Scholar
  10. 10.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3, 492–526 (2010)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Bresson, X., Laurent, T., Uminsky, D., Brecht, J.: Convergence and energy landscape for cheeger cut clustering. In: Advances in Neural Information Processing Systems, pp. 1385–1393 (2012)Google Scholar
  12. 12.
    Bresson, X., Laurent, T., Uminsky, D., Von Brecht, J.: Multiclass total variation clustering. In: Advances in Neural Information Processing Systems, pp. 1421–1429 (2013)Google Scholar
  13. 13.
    Bresson, X., Szlam, A.: Total variation, cheeger cuts. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 1039–1046 (2010)Google Scholar
  14. 14.
    Bresson, X., Tai, X.-C., Chan, T.F., Szlam, A.: Multi-class transductive learning based on 1 relaxations of Cheeger cut and Mumford-Shah-Potts model. J. Math. Imaging Vis. 49, 191–201 (2014)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Brinkmann, E.-M., Burger, M., Rasch, J., Sutour, C.: Bias-reduction in variational regularization, arXiv preprint arXiv:1606.05113 (2016)
  16. 16.
    Burger, M., Gilboa, G., Moeller, M., Eckardt, L., Cremers, D.: Spectral decompositions using one-homogeneous functionals, arXiv preprint arXiv:1601.02912 (2016)
  17. 17.
    Burger, M., Gilboa, G., Osher, S., Xu, J., et al.: Nonlinear inverse scale space methods. Commun. Math. Sci. 4, 179–212 (2006)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Burger, M., He, L., Schönlieb, C.-B.: Cahn–Hilliard inpainting and a generalization for grayvalue images. SIAM J. Imaging Sci. 2, 1129–1167 (2009)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Burger, M., Osher, S.: A guide to the tv zoo. In: Level Set and PDE Based Reconstruction Methods in Imaging, Springer, pp. 1–70 (2013)Google Scholar
  20. 20.
    Chambolle, A.: An algorithm for total variation minimization and applications. JMIV 20, 89–97 (2004)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theor. Found. Numer. Methods Sparse Recovery 9, 227 (2010)MathSciNetMATHGoogle Scholar
  22. 22.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Chatelin, F.: The spectral approximation of linear operators with applications to the computation of eigenelements of differential and integral operators. SIAM Rev. 23, 495–522 (1981)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Cuppen, J.J.M.: A divide and conquer method for the symmetric tridiagonal eigenproblem. Numer. Math. 36, 177–195 (1980)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16, 2080–2095 (2007)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Deledalle, C.-A., Papadakis, N., Salmon, J.: On debiasing restoration algorithms: applications to total-variation and nonlocal-means. In: International Conference on Scale Space and Variational Methods in Computer Vision, Springer, pp. 129–141 (2015)Google Scholar
  27. 27.
    Dong, B., Ji, H., Li, J., Shen, Z., Xu, Y.: Wavelet frame based blind image inpainting. Appl. Comput. Harmonic Anal. 32, 268–279 (2012)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Ekeland, I., Temam, R.: Convex Analysis and 9 Variational Problems. SIAM, New Delhi (1976)MATHGoogle Scholar
  29. 29.
    Elmoataz, A., Lezoray, O., Bougleux, S.: Nonlocal discrete regularization on weighted graphs: a framework for image and manifold processing. IEEE Trans. Image Process. 17, 1047–1060 (2008)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Francis, J.G.F.: The qr transformation a unitary analogue to the lr transformationpart 1. Comput. J. 4, 265–271 (1961)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Gilboa, G.: A total variation spectral framework for scale and texture analysis. SIAM J. Imaging Sci. 7, 1937–1961 (2014)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7, 1005–1028 (2009)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Hein, M., Bühler, T.: An inverse power method for nonlinear eigenproblems with applications in 1-spectral clustering and sparse pca. In: Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23, pp. 847–855. Curran Associates, Inc., New York (2010)Google Scholar
  34. 34.
    Jung, M., Peyré, G., Cohen, L.D.: Nonlocal active contours. SIAM J. Imaging Sci. 5, 1022–1054 (2012)MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Kindermann, S., Osher, S., Jones, P.W.: Deblurring and denoising of images by nonlocal functionals. Multiscale Model. Simul. 4, 1091–1115 (2005)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Knoll, F., Bredies, K., Pock, T., Stollberger, R.: Second order total generalized variation (TGV) for MRI. Magn. Reson. Med. 65, 480–491 (2011)CrossRefGoogle Scholar
  37. 37.
    Landau, H.J.: On Szegö’s eingenvalue distribution theorem and non-hermitian kernels. Journal d’Analyse Mathématique 28, 335–357 (1975)CrossRefMATHGoogle Scholar
  38. 38.
    Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. In: International Conference on Scale Space and Variational Methods in Computer Vision, Springer, pp. 150–162 (2009)Google Scholar
  39. 39.
    Louchet, C., Moisan, L.: Total variation as a local filter. SIAM J. Imaging Sci. 4, 651–694 (2011)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Meyer, Y.: Oscillating patterns in image processing and in some nonlinear evolution equations, March 2001. The 15th Dean Jacquelines B. Lewis Memorial Lectures. (2001)Google Scholar
  41. 41.
    Papafitsoros, K., Bredies, K.: A study of the one dimensional total generalised variation regularisation problem. arXiv preprint arXiv:1309.5900 (2013)
  42. 42.
    Pöschl, C., Scherzer, O.: Exact solutions of one-dimensional tgv. arXiv preprint arXiv:1309.7152 (2013)
  43. 43.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Saad, Y.: Numerical Methods for Large Eigenvalue Problems. Society for Industrial and Applied Mathematics, Philadelphia (2011)CrossRefMATHGoogle Scholar
  45. 45.
    Sawatzky, A., Tenbrinck, D., Jiang, X., Burger, M.: A variational framework for region-based segmentation incorporating physical noise models. J. Math. Imaging Vis. 47, 179–209 (2013)MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Schmaltz, C., Rosenhahn, B., Brox, T., Weickert, J.: Region-based pose tracking with occlusions using 3d models. Mach. Vis. Appl. 23, 557–577 (2012)CrossRefGoogle Scholar
  47. 47.
    Schmidt, M.F., Benning, M., Schönlieb, C.-B.: Inverse scale space decomposition, arXiv preprint arXiv:1612.09203 (2016)
  48. 48.
    Trefethen, L.N.: Approximation theory and numerical linear algebra. In: Algorithms for approximation II, Springer, pp. 336–360 (1990)Google Scholar
  49. 49.
    Trefethen, L.N.: Pseudospectra of matrices. Numer. Anal 91, 234–266 (1991)MATHGoogle Scholar
  50. 50.
    Trefethen, L.N., Bau III, D.: Numerical Linear Algebra, vol. 50. Siam, New Delhi (1997)CrossRefMATHGoogle Scholar
  51. 51.
    Trefethen, L.N., Embree, M.: Spectra and Pseudospectra: The Behavior of Nonnormal Matrices and Operators. Princeton University Press, Princeton (2005)MATHGoogle Scholar
  52. 52.
    Varah, J.M.: On the separation of two matrices. SIAM J. Numer. Anal. 16, 216–222 (1979)MathSciNetCrossRefMATHGoogle Scholar
  53. 53.
    Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in pde-based computation of image motion. Int. J. Comput. Vis. 45, 245–264 (2001)CrossRefMATHGoogle Scholar
  54. 54.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 21, pp. 1753–1760. Curran Associates, Inc., New York (2009)Google Scholar
  55. 55.
    Werlberger, M., Pock, T., Unger, M., Bischof, H.: Optical flow guided tv-l1 video interpolation and restoration. In: International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, Springer, pp. 273–286 (2011)Google Scholar
  56. 56.
    Wilkinson, J.H., Wilkinson, J.H.: The Algebraic Eigenvalue Problem, vol. 87. Clarendon Press, Oxford (1965)MATHGoogle Scholar
  57. 57.
    Yang, M., Liang, J., Zhang, J., Gao, H., Meng, F., Xingdong, L., Song, S.-J.: Non-local means theory based perona-malik model for image denosing. Neurocomputing 120, 262–267 (2013)CrossRefGoogle Scholar
  58. 58.
    Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: 2011 International Conference on Computer Vision, IEEE, pp. 479–486 (2011)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Electrical EngineeringTechnion - Israel Institute of TechnologyHaifaIsrael

Personalised recommendations