FSI Schemes: Fast Semi-Iterative Solvers for PDEs and Optimisation Methods

  • David HafnerEmail author
  • Peter Ochs
  • Joachim Weickert
  • Martin Reißel
  • Sven Grewenig
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


Many tasks in image processing and computer vision are modelled by diffusion processes, variational formulations, or constrained optimisation problems. Basic iterative solvers such as explicit schemes, Richardson iterations, or projected gradient descent methods are simple to implement and well-suited for parallel computing. However, their efficiency suffers from severe step size restrictions. As a remedy we introduce a simple and highly efficient acceleration strategy, leading to so-called Fast Semi-Iterative (FSI) schemes that extrapolate the basic solver iteration with the previous iterate. To derive suitable extrapolation parameters, we establish a recursion relation that connects box filtering with an explicit scheme for 1D homogeneous diffusion. FSI schemes avoid the main drawbacks of recent Fast Explicit Diffusion (FED) and Fast Jacobi techniques, and they have an interesting connection to the heavy ball method in optimisation. Our experiments show their benefits for anisotropic diffusion inpainting, nonsmooth regularisation, and Nesterov’s worst case problems for convex and strongly convex optimisation.


Elliptic Problem Time Step Size Explicit Scheme Linear Diffusion Diffusion Evolution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Our research has been partially funded by the Deutsche Forschungsgemeinschaft (DFG) through a Gottfried Wilhelm Leibniz Prize for Joachim Weickert and by the Graduate School of Computer Science at Saarland University. This is gratefully acknowledged.


  1. 1.
    Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Prob. 10, 1217–1229 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bertero, M., Poggio, T.A., Torre, V.: Ill-posed problems in early vision. Proc. IEEE 76(8), 869–889 (1988)CrossRefGoogle Scholar
  3. 3.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)MathSciNetGoogle Scholar
  4. 4.
    Gentzsch, W., Schlüter, A.: Über ein Einschrittverfahren mit zyklischer Schrittweitenänderung zur Lösung parabolischer Differentialgleichungen. Zeitschrift für Angewandte Mathematik und Mechanik 58, T415–T416 (1978). in GermanzbMATHGoogle Scholar
  5. 5.
    Golub, G.H., Varga, R.S.: Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods. Part I. Numerische Mathematik 3(1), 147–156 (1961)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Hellwig, G.: Partial Differential Equations. Teubner, Stuttgart (1977)zbMATHGoogle Scholar
  7. 7.
    van der Houwen, P.J., Sommeijer, B.P.: On the internal stability of explicit, m-stage Runge-Kutta methods for large m-values. Zeitschrift für Angewandte Mathematik und Mechanik 60(10), 479–485 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Iijima, T.: Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bull. Electrotech. Lab. 26, 368–388 (1962). in JapaneseGoogle Scholar
  9. 9.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, Applied Optimization, vol. 87. Kluwer, Boston (2004)zbMATHGoogle Scholar
  10. 10.
    Ochs, P., Brox, T., Pock, T.: iPiasco: inertial proximal algorithm for strongly convex optimization. J. Math. Imaging Vis. 53(2), 171–181 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)CrossRefGoogle Scholar
  12. 12.
    Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)CrossRefGoogle Scholar
  13. 13.
    Richardson, L.F.: The approximate arithmetical solution by finite differences of physical problems involving differential equation, with an application to the stresses in a masonry dam. Philos. Trans. R. Soc. A 210, 307–357 (1911)CrossRefzbMATHGoogle Scholar
  14. 14.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation, chap. 8. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)Google Scholar
  16. 16.
    Saul’yev, V.K.: Integration of Equations of Parabolic Type by the Method of Nets. Elsevier, Pergamon, Oxford (1964)zbMATHGoogle Scholar
  17. 17.
    Setzer, S., Steidl, G., Morgenthaler, J.: A cyclic projected gradient method. Comput. Optim. Appl. 54(2), 417–440 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of 30th International Conference on Machine Learning, pp. 1139–1147. Atlanta, GA, June 2013Google Scholar
  19. 19.
    Varga, R.S.: A comparison of the successive overrelaxation method and semi-iterative methods using Chebyshev polynomials. J. Soc. Ind. Appl. Math. 5(2), 39–46 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Weickert, J.: Theoretical foundations of anisotropic diffusion in image processing. Comput. Suppl. 11, 221–236 (1996)CrossRefGoogle Scholar
  21. 21.
    Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner, Stuttgart (1998)zbMATHGoogle Scholar
  22. 22.
    Weickert, J., Grewenig, S., Schroers, C., Bruhn, A.: Cyclic schemes for PDE-based image analysis. Int. J. Comput. Vis. 118(3), 275–299 (2016)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wells, W.M.: Efficient synthesis of Gaussian filters by cascaded uniform filters. IEEE Trans. Pattern Anal. Mach. Intell. 8(2), 234–239 (1986)CrossRefGoogle Scholar
  24. 24.
    Young, D.M.: On Richardson’s method for solving linear systems with positive definite matrices. J. Math. Phys. 32(1), 243–255 (1954)zbMATHGoogle Scholar
  25. 25.
    Chzhao-Din, Y.: Some difference schemes for the solution of the first boundary value problem for linear differential equations with partial derivatives. Ph.D. thesis, Moscow State University (1958). in RussianGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • David Hafner
    • 1
    Email author
  • Peter Ochs
    • 1
  • Joachim Weickert
    • 1
  • Martin Reißel
    • 2
  • Sven Grewenig
    • 1
  1. 1.Mathematical Image Analysis Group, Faculty of Mathematics and Computer Science, Campus E1.7Saarland UniversitySaarbrückenGermany
  2. 2.Fachbereich Medizintechnik und TechnomathematikFachhochschule AachenJülichGermany

Personalised recommendations