Journal of Scientific Computing

, Volume 65, Issue 3, pp 895–919 | Cite as

A New Steplength Selection for Scaled Gradient Methods with Application to Image Deblurring



Gradient methods are frequently used in large scale image deblurring problems since they avoid the onerous computation of the Hessian matrix of the objective function. Second order information is typically sought by a clever choice of the steplength parameter defining the descent direction, as in the case of the well-known Barzilai and Borwein rules. In a recent paper, a strategy for the steplength selection approximating the inverse of some eigenvalues of the Hessian matrix has been proposed for gradient methods applied to unconstrained minimization problems. In the quadratic case, this approach is based on a Lanczos process applied every \(m\) iterations to the matrix of the gradients computed in the previous \(m\) iterations, but the idea can be extended to a general objective function. In this paper we extend this rule to the case of scaled gradient projection methods applied to constrained minimization problems, and we test the effectiveness of the proposed strategy in image deblurring problems in both the presence and the absence of an explicit edge-preserving regularization term.


Image deconvolution Constrained optimization Scaled gradient projection methods Ritz values 

Mathematics Subject Classification

65K05 65R32 68U10 90C06 



This work has been partially supported by the Italian Spinner 2013 Ph.D. Project “High-complexity inverse problems in biomedical applications and social systems” and by MIUR (Italian Ministry for University and Research), under the projects FIRB—Futuro in Ricerca 2012, contract RBFR12M3AC, and PRIN 2012, contract 2012MTE38N. The Italian GNCS—INdAM (Gruppo Nazionale per il Calcolo Scientifico—Istituto Nazionale di Alta Matematica) is also acknowledged.


  1. 1.
    Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 1217–1229 (2004)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bardsley, J.M., Goldes, J.: Regularization parameter selection methods for ill-posed Poisson maximum likelihood estimation. Inverse Probl. 25(9), 095005 (2009)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988)MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Bertero, M., Boccacci, P., Talenti, G., Zanella, R., Zanni, L.: A discrepancy principle for Poisson data. Inverse Probl. 26(10), 105004 (2010)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bertero, M., Lantéri, H., Zanni, L.: Iterative image reconstruction: a point of view. In: Censor, Y., Jiang, M., Louis, A.K. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy, pp. 37–63. Edizioni della Normale, Pisa (2008)Google Scholar
  6. 6.
    Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1999)MATHGoogle Scholar
  7. 7.
    Bertsekas, D.: Convex Optimization Theory. Supplementary Chapter 6 on Convex Optimization Algorithms. Athena Scientific, Belmont (2009)Google Scholar
  8. 8.
    Birgin, E.G., Martinez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539–559 (2003)MATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Bonettini, S., Landi, G., Loli Piccolomini, E., Zanni, L.: Scaling techniques for gradient projection-type methods in astronomical image deblurring. Int. J. Comput. Math. 90(1), 9–29 (2013)MATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    Bonettini, S., Prato, M.: Nonnegative image reconstruction from sparse Fourier data: a new deconvolution algorithm. Inverse Probl. 26(9), 095001 (2010)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Bonettini, S., Prato, M.: Accelerated gradient methods for the X-ray imaging of solar flares. Inverse Probl. 30(5), 055004 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bonettini, S., Prato, M.: A new general framework for gradient projection methods (2014). arXiv:1406.6601
  13. 13.
    Bonettini, S., Ruggiero, V.: An alternating extragradient method for total variation based image restoration from Poisson data. Inverse Probl. 27(9), 095001 (2011)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Bonettini, S., Ruggiero, V.: On the convergence of primal–dual hybrid gradient algorithms for total variation image restoration. J. Math. Imaging Vis. 44(3), 236–253 (2012)MATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    Bonettini, S., Zanella, R., Zanni, L.: A scaled gradient projection method for constrained image deblurring. Inverse Probl. 25(1), 015002 (2009)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Carlavan, M., Blanc-Féraud, L.: Regularizing parameter estimation for Poisson noisy image restoration. In: International ICST Workshop on New Computational Methods for Inverse Problems, May 2011, Paris, FranceGoogle Scholar
  17. 17.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)MathSciNetGoogle Scholar
  18. 18.
    Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6(2), 418–445 (1996)MATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    Cornelio, A., Porta, F., Prato, M., Zanni, L.: On the filtering effect of iterative regularization algorithms for discrete inverse problems. Inverse Probl. 29(12), 125013 (2013)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Dai, Y.H., Yuan, Y.X.: Alternate minimization gradient method. IMA J. Numer. Anal. 23(3), 377–393 (2003)MATHMathSciNetCrossRefGoogle Scholar
  22. 22.
    Daube-Witherspoon, M.E., Muehllener, G.: An iterative image space reconstruction algorithm suitable for volume ECT. IEEE Trans. Med. Imaging 5(2), 61–66 (1986)CrossRefGoogle Scholar
  23. 23.
    De Asmundis, R., Di Serafino, D., Riccio, F., Toraldo, G.: On spectral properties of steepest descent methods. IMA J. Numer. Anal. 33(4), 1416–1435 (2013)MATHMathSciNetCrossRefGoogle Scholar
  24. 24.
    De Asmundis, R., Di Serafino, D., Hager, W.W., Toraldo, G., Zhang, H.: An efficient gradient method using the Yuan steplength. Comput. Optim. Appl. 59(3), 541–563 (2014)MATHMathSciNetCrossRefGoogle Scholar
  25. 25.
    Fletcher, R.: A limited memory steepest descent method. Math. Program. 135(1–2), 413–436 (2012)MATHMathSciNetCrossRefGoogle Scholar
  26. 26.
    Frassoldati, G., Zanghirati, G., Zanni, L.: New adaptive stepsize selections in gradient methods. J. Ind. Manage. Optim. 4(2), 299–312 (2008)MATHMathSciNetCrossRefGoogle Scholar
  27. 27.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996)MATHGoogle Scholar
  28. 28.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23(4), 707–716 (1986)MATHMathSciNetCrossRefGoogle Scholar
  29. 29.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1997)MATHGoogle Scholar
  30. 30.
    Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)CrossRefGoogle Scholar
  31. 31.
    Harmany, Z.T., Marcia, R.F., Willett, R.M.: This is spiral-tap: sparse Poisson intensity reconstruction algorithms–theory and practice. IEEE Trans. Image Process. 3(21), 1084–1096 (2012)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Lantéri, H., Roche, M., Aime, C.: Penalized maximum likelihood image restoration with positivity constraints: multiplicative algorithms. Inverse Probl. 18(5), 1397–1419 (2002)MATHCrossRefGoogle Scholar
  33. 33.
    Lantéri, H., Roche, M., Cuevas, O., Aime, C.: A general method to devise maximum likelihood signal restoration multiplicative algorithms with non-negativity constraints. Signal Process. 81(5), 945–974 (2001)MATHCrossRefGoogle Scholar
  34. 34.
    Lucy, L.: An iterative technique for the rectification of observed distributions. Astron. J. 79(6), 745–754 (1974)CrossRefGoogle Scholar
  35. 35.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)MATHGoogle Scholar
  36. 36.
    Porta, F., Zanella, R., Zanghirati, G., Zanni, L.: Limited-memory scaled gradient projection methods for real-time image deconvolution in microscopy. Commun. Nonlinear Sci. Numer. Simul. 21, 112–127 (2015)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Prato, M., Cavicchioli, R., Zanni, L., Boccacci, P., Bertero, M.: Efficient deconvolution methods for astronomical imaging: algorithms and IDL-GPU codes. Astron. Astrophys. 539, A133 (2012)CrossRefGoogle Scholar
  38. 38.
    Prato, M., La Camera, A., Bonettini, S., Bertero, M.: A convergent blind deconvolution method for post-adaptive-optics astronomical imaging. Inverse Probl. 29(6), 065017 (2013)CrossRefGoogle Scholar
  39. 39.
    Richardson, W.H.: Bayesian based iterative method of image restoration. J. Opt. Soc. Am. 62(1), 55–59 (1972)CrossRefGoogle Scholar
  40. 40.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)MATHCrossRefGoogle Scholar
  41. 41.
    Ruggiero, V., Zanni, L.: A modified projection algorithm for large strictly-convex quadratic programs. J. Optim. Theory Appl. 104(2), 281–299 (2000)MATHMathSciNetCrossRefGoogle Scholar
  42. 42.
    Setzer, S., Steidl, G., Teuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21(3), 193–199 (2010)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)MATHCrossRefGoogle Scholar
  44. 44.
    Yuan, Y.: A new stepsize for the steepest descent method. J. Comput. Math. 24, 149–156 (2006)MATHMathSciNetGoogle Scholar
  45. 45.
    Zanella, R., Boccacci, P., Zanni, L., Bertero, M.: Efficient gradient projection methods for edge-preserving removal of Poisson noise. Inverse Probl. 25(4), 045010 (2009)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Zanella, R., Zanghirati, G., Cavicchioli, R., Zanni, L., Boccacci, P., Bertero, M., Vicidomini, G.: Towards real-time image deconvolution: application to confocal and sted microscopy. Sci. Rep. 3, 2523 (2013)CrossRefGoogle Scholar
  47. 47.
    Zhou, B., Gao, L., Dai, Y.H.: Gradient methods with adaptive step-sizes. Comput. Optim. Appl. 35(1), 69–86 (2006)MATHMathSciNetCrossRefGoogle Scholar
  48. 48.
    Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total-variation-regularized image restoration. Comput. Optim. Appl. 47(3), 377–400 (2008)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Dipartimento di Scienze Fisiche, Informatiche e MatematicheUniversità degli Studi di Modena e Reggio EmiliaModenaItaly

Personalised recommendations