Large-Scale Inverse Problems in Imaging

  • Julianne Chung
  • Sarah Knepper
  • James G. Nagy


Large-scale inverse problems arise in a variety of significant applications in image processing, and efficient regularization methods are needed to compute meaningful solutions. This chapter surveys three common mathematical models including a linear, a separable nonlinear, and a general nonlinear model. Techniques for regularization and large-scale implementations are considered, with particular focus on algorithms and computations that can exploit structure in the problem. Examples from image deconvolution, multi-frame blind deconvolution, and tomosynthesis illustrate the potential of these algorithms. Much progress has been made in the field of large-scale inverse problems, but many challenges still remain for future research.


Inverse Problem Singular Value Decomposition Regularization Parameter Point Spread Function Tikhonov Regularization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References and Further Reading

  1. 1.
    Andrews HC, Hunt BR (1977) Digital image restoration. Prentice-Hall, Englewood CliffsGoogle Scholar
  2. 2.
    Bachmayr M, Burger M (2009) Iterative total variation schemes for nonlinear inverse problems. Inverse Prob 25:105004MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bardsley JM (2008) An efficient computational method for total variation-penalized Poisson likelihood estimation. Inverse Prob Imaging 2(2):167–185MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Bardsley JM (2008) Stopping rules for a nonnegatively constrained iterative method for illposed Poisson imaging problems. BIT 48(4):651–664MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Bardsley JM, Vogel CR (2003) A nonnegatively constrained convex programming method for image reconstruction. SIAM J Sci Comput 25(4):1326–1343MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Barzilai J, Borwein JM (1988) Two-point step size gradient methods. IMA J Numer Anal 8(1):141–148MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Björck Å (1988) A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT, 28(3):659–670MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Björck Å (1996) Numerical methods for least squares problems. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  9. 9.
    Björck Å, Grimme E, van Dooren P (1994) An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34(4):510–534MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Brakhage H (1987) On ill-posed problems and the method of conjugate gradients. In: Engl HW, Groetsch CW (eds) Inverse and ill-posed problems. Academic, Boston, pp 165–175Google Scholar
  11. 11.
    Calvetti D, Reichel L (2003) Tikhonov regularization of large linear problems. BIT 43(2):263–283MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Calvetti D, Somersalo E (2007) Introduction to Bayesian scientific computing. Springer, New YorkMATHGoogle Scholar
  13. 13.
    Candès EJ, Romberg JK, Tao T (2006) Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math 59(8):1207–1223MATHCrossRefGoogle Scholar
  14. 14.
    Carasso AS (2001) Direct blind deconvolution. SIAM J Appl Math 61(6):1980–2007MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Chadan K, Colton D, Päivärinta L, Rundell W (1997) An introduction to inverse scattering and inverse spectral problems. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  16. 16.
    Chan TF, Shen J (2005) Image processing and analysis: variational, PDE, wavelet, and stochastic methods. SIAM, PhiladelphiaMATHGoogle Scholar
  17. 17.
    Cheney M, Borden B (2009) Fundamentals of radar imaging. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  18. 18.
    Chung J, Haber E, Nagy J (2006) Numerical methods for coupled super-resolution. Inverse Prob 22(4):1261–1272MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Chung J, Nagy J (2010) An efficient iterative approach for large-scale separable nonlinear inverse problems. SIAM J Sci Comput 31(6):4654–4674MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Chung J, Nagy J, Sechopoulos I (2010) Numerical algorithms for polyenergetic digital breast tomosynthesis reconstruction. SIAM J Imaging Sci 3(1):133–152MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Chung J, Nagy JG, O’Leary DP (2008) A weighted GCV method for Lanczos hybrid regularization. Elec Trans Numer Anal 28: 149–167MathSciNetGoogle Scholar
  22. 22.
    Chung J, Sternberg P, Yang C (2010) High performance 3-d image reconstruction for molecular structure determination. Int J High Perform Comput Appl 24(2):117–135CrossRefGoogle Scholar
  23. 23.
    De Man B, Nuyts J, Dupont P, Marchal G, Suetens P (2001) An iterative maximumlikelihood polychromatic algorithm for CT. IEEE Trans Med Imaging 20(10):999–1008CrossRefGoogle Scholar
  24. 24.
    Diaspro A, Corosu M, Ramoino P, Robello M (1999) Two-photon excitation imaging based on a compact scanning head. IEEE Eng Med Biol 18(5):18–30CrossRefGoogle Scholar
  25. 25.
    Dobbins JT III, Godfrey DJ (2003) Digital X-ray tomosynthesis: current state of the art and clinical potential. Phys Med Biol 48(19):R65–R106CrossRefGoogle Scholar
  26. 26.
    Easley GR, Healy DM, Berenstein CA (2009) Image deconvolution using a general ridgelet and curvelet domain. SIAM J Imaging Sci 2(1):253–283MATHCrossRefGoogle Scholar
  27. 27.
    Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658CrossRefGoogle Scholar
  28. 28.
    Engl HW, Hanke M, Neubauer A (2000) Regularization of inverse problems. Kluwer, DordrechtGoogle Scholar
  29. 29.
    Engl HW, Kügler P (2005) Nonlinear inverse problems: theoretical aspects and some industrial applications. In: Capasso V, Périaux J (eds) Multidisciplinary methods for analysis optimization and control of complex systems. Springer, Berlin, pp 3–48CrossRefGoogle Scholar
  30. 30.
    Engl HW, Kunisch K, Neubauer A (1989) Convergence rates for Tikhonov regularisation of nonlinear ill-posed problems. Inverse Prob 5(4):523–540MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Engl HW, Louis AK, Rundell W (eds) (1996) Inverse problems in geophysical applications. SIAM, PhiladelphiaGoogle Scholar
  32. 32.
    Eriksson J, Wedin P (2004) Truncated Gauss-Newton algorithms for ill-conditioned nonlinear least squares problems. Optim Meth Softw 19(6):721–737MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Faber TL, Raghunath N, Tudorascu D, Votaw JR (2009) Motion correction of PET brain images through deconvolution: I. Theoretical development and analysis in software simulations. Phys Med Biol 54(3):797–811CrossRefGoogle Scholar
  34. 34.
    Figueiredo MAT, Nowak RD, Wright SJ (2007) Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J Sel Top Signal Process 1(4):586–597CrossRefGoogle Scholar
  35. 35.
    Frank J (2006) Three-dimensional electron microscopy of macromolecular assemblies. Oxford University Press, New YorkCrossRefGoogle Scholar
  36. 36.
    Golub GH, Heath M, Wahba G (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2):215–223MathSciNetMATHGoogle Scholar
  37. 37.
    Golub GH, Luk FT, Overton ML (1981) A block Lanczos method for computing the singular values and corresponding singular vectors of a matrix. ACM Trans Math Softw 7(2): 149–169MathSciNetMATHCrossRefGoogle Scholar
  38. 38.
    Golub GH, Pereyra V (1973) The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J Numer Anal 10(2):413–432MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Golub GH, Pereyra V (2003) Separable nonlinear least squares: the variable projection method and its applications. Inverse Prob 19: R1–R26MathSciNetMATHCrossRefGoogle Scholar
  40. 40.
    Haber E, Ascher UM, Oldenburg D (2000) On optimization techniques for solving nonlinear inverse problems. Inverse Prob 16(5):1263–1280MathSciNetMATHCrossRefGoogle Scholar
  41. 41.
    Haber E, Oldenburg D (2000) A GCV based method for nonlinear ill-posed problems. Comput Geosci 4(1):41–63MathSciNetMATHCrossRefGoogle Scholar
  42. 42.
    Hammerstein GR, Miller DW, White DR, Masterson ME, Woodard HQ, Laughlin JS (1979) Absorbed radiation dose in mammography. Radiology 130(2):485–491Google Scholar
  43. 43.
    Hanke M (1995) Conjugate gradient type methods for ill-posed problems. Pitman research notes in mathematics, Longman Scientific & Technical, HarlowGoogle Scholar
  44. 44.
    Hanke M (1996) Limitations of the L-curve method in ill-posed problems. BIT 36(2):287–301MathSciNetMATHCrossRefGoogle Scholar
  45. 45.
    Hanke M (2001) On Lanczos based methods for the regularization of discrete ill-posed problems. BIT 41(5):1008–1018MathSciNetCrossRefGoogle Scholar
  46. 46.
    Hansen PC (1992) Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev 34(4):561–580MathSciNetMATHCrossRefGoogle Scholar
  47. 47.
    Hansen PC (1992) Numerical tools for analysis and solution of Fredholm integral equations of the first kind. Inverse Prob 8(6):849–872MATHCrossRefGoogle Scholar
  48. 48.
    Hansen PC (1994) Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer Algorithms 6(1):1–35MathSciNetMATHCrossRefGoogle Scholar
  49. 49.
    Hansen PC (1998) Rank-deficient and discrete ill-posed problems. SIAM, PhiladelphiaCrossRefGoogle Scholar
  50. 50.
    Hansen PC (2010) Discrete inverse problems: insight and algorithms. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  51. 51.
    Hansen PC, Nagy JG, O’Leary DP (2006) Deblurring images: matrices, spectra and filtering. SIAM, PhiladelphiaMATHGoogle Scholar
  52. 52.
    Hansen PC, O’Leary DP (1993) The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14(6):1487–1503MathSciNetMATHCrossRefGoogle Scholar
  53. 53.
    Hardy JW (1994) Adapt Opt Sci Am 270(6): 60–65Google Scholar
  54. 54.
    Hofmann B (1993) Regularization of nonlinear problems and the degree of ill-posedness. In: Anger G, Gorenflo R, Jochmann H, Moritz H, Webers W (eds) Inverse problems: principles and applications in geophysics, technology, and medicine. Akademie Verlag, BerlinGoogle Scholar
  55. 55.
    Hohn M, Tang G, Goodyear G, Baldwin PR, Huang Z, Penczek PA, Yang C, Glaeser RM, Adams PD, Ludtke SJ (2007) SPARX, a new environment for Cryo-EM image processing. J Struct Biol 157(1):47–55CrossRefGoogle Scholar
  56. 56.
    Jain AK (1989) Fundamentals of digital image processing. Prentice-Hall, Englewood CliffsMATHGoogle Scholar
  57. 57.
    Kang MG, Chaudhuri S (2003) Super-resolution image reconstruction. IEEE Signal Process Mag 20(3):19–20CrossRefGoogle Scholar
  58. 58.
    Kaufman L (1975) A variable projection method for solving separable nonlinear least squares problems. BIT 15(1):49–57MathSciNetMATHCrossRefGoogle Scholar
  59. 59.
    Kilmer ME, Hansen PC, Español MI (2007) A projection-based approach to general-form Tikhonov regularization. SIAM J Sci Comput 29(1):315–330MathSciNetMATHCrossRefGoogle Scholar
  60. 60.
    Kilmer ME, O’Leary DP (2001) Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J Matrix Anal Appl 22(4):1204–1221MathSciNetMATHCrossRefGoogle Scholar
  61. 61.
    Landweber L (1951) An iteration formula for Fredholm integral equations of the first kind. Am J Math 73(3):615–624MathSciNetMATHCrossRefGoogle Scholar
  62. 62.
    Larsen RM (1998) Lanczos bidiagonalization with partial reorthogonalization. PhD thesis, Department of Computer Science, University of Aarhus, DenmarkGoogle Scholar
  63. 63.
    Lawson CL, Hanson RJ (1995) Solving least squares problems. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  64. 64.
    Löfdahl MG (2002) Multi-frame blind deconvolution with linear equality constraints. In: Bones PJ, Fiddy MA, Millane RP (eds) Image reconstruction from incomplete data II, vol 4792-21. SPIE, pp 146–155Google Scholar
  65. 65.
    Lohmann AW, Paris DP (1965) Space-variant image formation. J Opt Soc Am 55(8):1007–1013Google Scholar
  66. 66.
    Marabini R, Herman GT, Carazo JM (1998) 3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs). Ultramicroscopy 72(1–2):53–65CrossRefGoogle Scholar
  67. 67.
    Matson CL, Borelli K, Jefferies S, Beckner CC Jr, Hege EK, Lloyd-Hart M (2009) Fast and optimal multiframe blind deconvolution algorithm for high-resolution groundbased imaging of space objects. Appl Opt 48(1):A75–A92CrossRefGoogle Scholar
  68. 68.
    McNown SR, Hunt BR (1994) Approximate shift-invariance by warping shift-variant systems. In: Hanisch RJ, White RL (eds) The restoration of HST images and spectra II. Space Telescope Science Institute, Baltimore, MD, pp 181–187Google Scholar
  69. 69.
    Miller K (1970) Least squares methods for ill-posed problems with a prescribed bound. SIAM J Math Anal 1(1):52–74MathSciNetMATHCrossRefGoogle Scholar
  70. 70.
    Modersitzki J (2004) Numerical methods for image registration. Oxford University Press, OxfordMATHGoogle Scholar
  71. 71.
    Morozov VA (1966) On the solution of functional equations by the method of regularization. Sov Math Dokl 7:414–417MATHGoogle Scholar
  72. 72.
    Nagy JG, O’Leary DP (1997) Fast iterative image restoration with a spatially varying PSF. In: Luk FT (ed) Advanced signal processing: algorithms, architectures, and implementations VII, vol 3162. SPIE, pp 388–399Google Scholar
  73. 73.
    Nagy JG, O’Leary DP (1998) Restoring images degraded by spatially-variant blur. SIAM J Sci Comput 19(4):1063–1082MathSciNetMATHCrossRefGoogle Scholar
  74. 74.
    Natterer F (2001) The mathematics of computerized tomography. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  75. 75.
    Natterer F, Wübbeling F (2001) Mathematical methods in image reconstruction. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  76. 76.
    Nguyen N, Milanfar P, Golub G (2001) Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans Image Process 10(9):1299–1308MathSciNetMATHCrossRefGoogle Scholar
  77. 77.
    Nocedal J, Wright S (1999) Numerical optimization. Springer, New YorkMATHCrossRefGoogle Scholar
  78. 78.
    O’Leary DP, Simmons JA (1981) A bidiagonali- zation-regularization procedure for large scale discretizations of ill-posed problems. SIAM J Sci Stat Comput 2(4):474–489MathSciNetMATHCrossRefGoogle Scholar
  79. 79.
    Osborne MR (2007) Separable least squares, variable projection, and the Gauss-Newton algorithm. Elec Trans Numer Anal 28:1–15MathSciNetMATHGoogle Scholar
  80. 80.
    Paige CC, Saunders MA (1982) Algorithm 583 LSQR: Sparse linear equations and least squares problems. ACM Trans Math Softw 8(2): 195–209MathSciNetCrossRefGoogle Scholar
  81. 81.
    Paige CC, Saunders MA (1982) LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8(1): 43–71MathSciNetMATHCrossRefGoogle Scholar
  82. 82.
    Penczek PA, Radermacher M, Frank J (1992) Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1):33–53CrossRefGoogle Scholar
  83. 83.
    Phillips DL (1962) A technique for the numerical solution of certain integral equations of the first kind. J Assoc Comput Mach 9(1): 84–97MathSciNetMATHCrossRefGoogle Scholar
  84. 84.
    Raghunath N, Faber TL, Suryanarayanan S, Votaw JR (2009) Motion correction of PET brain images through deconvolution: II. Practical implementation and algorithm optimization. Phys Med Biol 54(3):813–829CrossRefGoogle Scholar
  85. 85.
    Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60:259–268MATHCrossRefGoogle Scholar
  86. 86.
    Ruhe A, Wedin P (1980) Algorithms for separable nonlinear least squares problems. SIAM Rev 22(3):318–337MathSciNetMATHCrossRefGoogle Scholar
  87. 87.
    Saad Y (1980) On the rates of convergence of the Lanczos and the block-Lanczos methods. SIAM J Numer Anal 17(5):687–706MathSciNetMATHCrossRefGoogle Scholar
  88. 88.
    Saban SD, Silvestry M, Nemerow GR, Stewart PL (2006) Visualization of α-helices in a 6-Ångstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments. J Virol 80(24): 49–59CrossRefGoogle Scholar
  89. 89.
    Tikhonov AN (1963) Regularization of incorrectly posed problems. Sov Math Dokl 4:1624–1627MATHGoogle Scholar
  90. 90.
    Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Sov Math Dokl 4:1035–1038Google Scholar
  91. 91.
    Tikhonov AN, Arsenin VY (1977) Solutions of ill-posed problems. Winston, WashingtonMATHGoogle Scholar
  92. 92.
    Tikhonov AN, Leonov AS, Yagola AG (1998) Nonlinear ill-posed problems, vol 1–2. Chapman and Hall, LondonMATHGoogle Scholar
  93. 93.
    Trussell HJ, Fogel S (1992) Identification and restoration of spatially variant motion blurs in sequential images. IEEE Trans Image Process 1(1):123–126CrossRefGoogle Scholar
  94. 94.
    Tsaig Y, Donoho DL (2006) Extensions of compressed sensing. Signal Process 86(3): 549–571MATHCrossRefGoogle Scholar
  95. 95.
    Varah JM (1983) Pitfalls in the numerical solution of linear ill-posed problems. SIAM J Sci Stat Comput 4(2):164–176MathSciNetMATHCrossRefGoogle Scholar
  96. 96.
    Vogel CR (1986) Optimal choice of a truncation level for the truncated SVD solution of linear first kind integral equations when data are noisy. SIAM J Numer Anal 23(1):109–117MathSciNetMATHCrossRefGoogle Scholar
  97. 97.
    Vogel CR (1987) An overview of numerical methods for nonlinear ill-posed problems. In: Engl HW, Groetsch CW (eds) Inverse and ill-posed problems. Academic Press, Boston, pp 231–245Google Scholar
  98. 98.
    Vogel CR (1996) Non-convergence of the L-curve regularization parameter selection method. Inverse Prob 12(4):535–547MATHCrossRefGoogle Scholar
  99. 99.
    Vogel CR (2002) Computational methods for inverse problems. SIAM, PhiladelphiaMATHCrossRefGoogle Scholar
  100. 100.
    Wagner FC, Macovski A, Nishimura DG (1988) A characterization of the scatter pointspread-function in terms of air gaps. IEEE Trans Med Imaging 7(4):337–344CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Julianne Chung
  • Sarah Knepper
  • James G. Nagy

There are no affiliations available

Personalised recommendations