Learning Filter Functions in Regularisers by Minimising Quotients

  • Martin Benning
  • Guy Gilboa
  • Joana Sarah GrahEmail author
  • Carola-Bibiane Schönlieb
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10302)


Learning approaches have recently become very popular in the field of inverse problems. A large variety of methods has been established in recent years, ranging from bi-level learning to high-dimensional machine learning techniques. Most learning approaches, however, only aim at fitting parametrised models to favourable training data whilst ignoring misfit training data completely. In this paper, we follow up on the idea of learning parametrised regularisation functions by quotient minimisation as established in [3]. We extend the model therein to include higher-dimensional filter functions to be learned and allow for fit- and misfit-training data consisting of multiple functions. We first present results resembling behaviour of well-established derivative-based sparse regularisers like total variation or higher-order total variation in one-dimension. Our second and main contribution is the introduction of novel families of non-derivative-based regularisers. This is accomplished by learning favourable scales and geometric properties while at the same time avoiding unfavourable ones.


Regularisation learning Non-linear eigenproblem Sparse regularisation Generalised inverse power method 



MB acknowledges support from the Leverhulme Trust early career fellowship “Learning from mistakes: a supervised feedback-loop for imaging applications” and the Newton Trust. GG acknowledges support by the Israel Science Foundation (grant 718/15). JSG acknowledges support by the NIHR Cambridge Biomedical Research Centre. CBS acknowledges support from Leverhulme Trust project Breaking the non-convexity barrier, EPSRC grant EP/M00483X/1, EPSRC centre EP/N014588/1, the Cantab Capital Institute for the Mathematics of Information, and from CHiPS (Horizon 2020 RISE project grant).

Data Statement. The corresponding MATLAB\(^{\textregistered }\) code is publicly available on Apollo - University of Cambridge Repository (


  1. 1.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)CrossRefGoogle Scholar
  2. 2.
    Benning, M., Burger, M.: Ground states and singular vectors of convex variational regularization methods. Methods Appl. Anal. 20(4), 295–334 (2013)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Benning, M., Gilboa, G., Schönlieb, C.-B.: Learning parametrised regularisation functions via quotient minimisation. PAMM 16(1), 933–936 (2016)CrossRefGoogle Scholar
  4. 4.
    Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bresson, X., Laurent, T., Uminsky, D., Brecht, J.V.: Convergence and energy landscape for Cheeger cut clustering. In: Advances in Neural Information Processing Systems (2012)Google Scholar
  6. 6.
    Brox, T., Kleinschmidt, O., Cremers, D.: Efficient nonlocal means for denoising of textural patterns. IEEE Trans. Image Process. 17(7), 1083–1092 (2008)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    De Los Reyes, J.C., Schönlieb, C.-B.: Image denoising: learning the noise model via nonsmooth PDE-constrained optimization. Inverse Probl. Imaging 7(4), 1139–1155 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    De Los Reyes, J.C., Schönlieb, C.-B., Valkonen, T.: Bilevel parameter learning for higher-order total variation regularisation models. J. Math. Imaging Vis., 1–25 (2016)Google Scholar
  10. 10.
    Gilboa, G.: Expert regularizers for task specific processing. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 24–35. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38267-3_3 CrossRefGoogle Scholar
  11. 11.
    Grah, J.S., Harrington, J., Koh, S.B., Pike, J., Schreiner, A., Burger, M., Schönlieb, C.-B., Reichelt, S.: Mathematical Imaging Methods for Mitosis Analysis in Live-Cell Phase Contrast Microscopy. arXiv preprint arXiv:1609.04649 (2016)
  12. 12.
    Grant, M., Boyd, S.: CVX: Matlab software for disciplined convex programming, version 2.1 (2016).
  13. 13.
    Hein, M., Bühler, T.: An inverse power method for nonlinear eigenproblems with applications in 1-spectral clustering and sparse PCA. In: Advances in Neural Information Processing Systems (2010)Google Scholar
  14. 14.
    Kunisch, K., Pock, T.: A bilevel optimization approach for parameter learning in variational models. SIAM J. Imaging Sci. 6(2), 938–983 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Kurdyka, K.: On gradients of functions definable in o-minimal structures. Annales de l’institut Fourier 48(3), 769–783 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. In: Les Équations aux Dérivées Partielles, pp. 87–89 (1963)Google Scholar
  17. 17.
    Papyan, V., Romano, Y., Elad, M.: Convolutional Neural Networks Analyzed via Convolutional Sparse Coding. arXiv preprint arXiv:1607.08194 (2016)
  18. 18.
    Schmidt, M.F., Benning, M., Schönlieb, C.-B.: Inverse Scale Space Decomposition. arXiv preprint arXiv:1612.09203 (2016)
  19. 19.
    Zeune, L., van Dalum, G., Terstappen, L., van Gils, S.A., Brune, C.: Multiscale Segmentation via Bregman Distances and Nonlinear Spectral Analysis. arXiv preprint arXiv:1604.06665 (2016)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Martin Benning
    • 1
  • Guy Gilboa
    • 2
  • Joana Sarah Grah
    • 1
    Email author
  • Carola-Bibiane Schönlieb
    • 1
  1. 1.Department of Applied Mathematics and Theoretical PhysicsUniversity of CambridgeCambridgeUK
  2. 2.Electrical Engineering DepartmentTechnion - Israel Institute of TechnologyHaifaIsrael

Personalised recommendations